Skip to content

Add support for real device testing#83

Open
longtimeno-c wants to merge 3 commits into
EvanBacon:mainfrom
longtimeno-c:main
Open

Add support for real device testing#83
longtimeno-c wants to merge 3 commits into
EvanBacon:mainfrom
longtimeno-c:main

Conversation

@longtimeno-c

@longtimeno-c longtimeno-c commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces a new "device mode" to the serve-sim tool, allowing users to stream the screen of a physical iOS device (iPhone/iPad) to their browser using WebDriverAgent (WDA). This is particularly useful for Intel Macs that can't run arm64-only simulators. The implementation includes a new relay server for device streaming, a minimal viewer UI, and a new CLI command for launching device mode.

Device streaming support:

  • Added a new device-server.ts module that implements an HTTP relay server to bridge a browser to a physical iOS device via WDA. It provides endpoints for MJPEG streaming, input events, device config, and health checks, along with a minimal HTML viewer for live device interaction.
  • Introduced a new debug logger debugDevice in debug.ts for real-device mode, aiding in troubleshooting device communication.

CLI enhancements:

  • Added a device command to the CLI (index.ts) that detects connected iOS devices, sets up and launches a WDA session, and starts the device relay server. The command provides helpful error messages, setup instructions, and clean shutdown handling.
  • Registered the new device command with the CLI, including documentation, usage examples, and environment variable overrides in the help text.

Summary by CodeRabbit

  • New Features

    • Added a new "serve-sim device" command to view and control physical iOS devices in a browser with real-time MJPEG streaming, pointer input (tap/drag/home), and automatic reconnects.
    • Automatic device detection, WebDriverAgent-backed session management, and local/network viewer URLs.
  • Documentation

    • Updated docs and examples with usage, one-time setup steps, and environment configuration for the new device workflow.

@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 12046406-a811-42d0-a0b6-f7272cd74492

📥 Commits

Reviewing files that changed from the base of the PR and between 00b7891 and b988575.

📒 Files selected for processing (1)
  • packages/serve-sim/src/device-server.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/serve-sim/src/device-server.ts

📝 Walkthrough

Walkthrough

Adds real-device mode: WDA lifecycle (build, launch, port forwarding, session, input), an HTTP relay server for MJPEG stream and input mapping with an embedded viewer, CLI command wiring (serve-sim device), and documentation updates.

Changes

Physical Device Support via WebDriverAgent

Layer / File(s) Summary
WDA Module Foundation & Device Discovery
packages/serve-sim/src/wda.ts (1–163)
Module docs, RealDevice/WdaConfig types, findPymobiledevice3, device listing/detection, team ID resolution, and default config helpers.
WdaSession Class & Build/Launch Lifecycle
packages/serve-sim/src/wda.ts (165–333)
WdaSetupError, WdaSession core state, runner product path, ensureBuilt() to build runner, and launchRunner() to start the on-device runner.
WDA Session Establishment & Communication
packages/serve-sim/src/wda.ts (334–402)
startForward() usbmux forwarding, pollStatusReady() health polling, openSession() POST /session, refreshWindowSize(), and full start() orchestration.
WDA Input Operations & Session Cleanup
packages/serve-sim/src/wda.ts (404–479)
tap, drag, pressButton methods, stop() teardown, delay() and wdaFetch() helpers.
Device Server Setup & Request Routing
packages/serve-sim/src/device-server.ts (1–118)
DeviceServer interface and startDeviceServer that binds an HTTP server and routes /health, /config, /stream.mjpeg, POST /input, and viewer endpoints.
MJPEG Stream Relay
packages/serve-sim/src/device-server.ts (120–139)
proxyMjpeg reverse-proxies upstream session MJPEG with correct headers, CORS, client/upstream lifecycle handling, and upstream error responses.
Input Event Handling & Coordinate Mapping
packages/serve-sim/src/device-server.ts (141–189)
handleInput guards request size, parses JSON PointerEventBody, clamps/scales normalized 0..1 coords to device points, calls WdaSession actions, and responds with structured JSON; includes clamp01() and json() helpers.
Embedded Viewer HTML & Frontend
packages/serve-sim/src/device-server.ts (206–309)
viewerHtml emits a self-contained page displaying /stream.mjpeg, normalizing pointer input, distinguishing tap vs drag, posting to /input, Home button, reconnect logic, and escapeHtml() usage.
CLI Command & Device Function
packages/serve-sim/src/debug.ts (13–15), packages/serve-sim/src/index.ts (1867–2117)
Exports debugDevice, adds serve-sim device command and device() flow that selects device, starts WdaSession, runs startDeviceServer, prints local/LAN URLs, and coordinates shutdown on signals.

Sequence Diagram

sequenceDiagram
  participant User as CLI User
  participant WdaSession
  participant Xcode as xcodebuild
  participant usbmux as pymobiledevice3
  participant WDA as WebDriverAgent
  participant DeviceServer as Device Server
  participant Browser as Browser/Viewer
  User->>WdaSession: create & start()
  WdaSession->>Xcode: build-for-testing (if needed)
  WdaSession->>Xcode: test-without-building (launch runner)
  WdaSession->>usbmux: usbmux forward control/mjpeg ports
  WdaSession->>WDA: GET /status (poll until ready)
  WdaSession->>WDA: POST /session
  WdaSession->>WDA: GET /session/{id}/window/size
  User->>DeviceServer: startDeviceServer(session)
  Browser->>DeviceServer: GET /stream.mjpeg
  DeviceServer->>WDA: GET /mjpeg (proxied)
  Browser->>DeviceServer: POST /input (tap/drag/button)
  DeviceServer->>WdaSession: tap/drag/pressButton
  WdaSession->>WDA: POST session command
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • EvanBacon/serve-sim#59: Both PRs touch CLI command registration in packages/serve-sim/src/index.ts; changes may intersect with commander refactor.

Poem

🐰 I hopped through build logs, ports, and streams,
To bring phones alive inside browser dreams,
MJPEG frames flutter, taps translate to deeds,
A tiny viewer carries developers' needs,
Hooray — real devices now answer our beams.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 52.94% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'Add support for real device testing' is vague and generic—it does not specifically convey the main change of adding WebDriverAgent-based device streaming via a new CLI subcommand. Consider a more specific title like 'Add device streaming via WebDriverAgent' or 'Add serve-sim device command for iOS real-device preview' to clearly convey the primary change.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
packages/serve-sim/src/wda.ts (1)

334-345: 💤 Low value

Consider adding error handling for port forwarding failures.

The forwarder processes are spawned with stdio ignored, so a failure (e.g., port 8100 already in use) will go unnoticed until pollStatusReady times out with a less specific error. Adding an error event listener could provide clearer diagnostics:

   private startForward(localPort: number, devicePort: number): ChildProcess {
     const pmd = findPymobiledevice3();
     if (!pmd) throw new WdaSetupError("pymobiledevice3 not found (needed for usbmux port forwarding).");
     const child = spawn(
       pmd,
       ["usbmux", "forward", String(localPort), String(devicePort), "--serial", this.device.udid],
-      { stdio: ["ignore", "ignore", "ignore"] },
+      { stdio: ["ignore", "ignore", "pipe"] },
     );
+    child.stderr?.on("data", (d) => debugDevice("forward stderr: %s", d.toString().trim()));
+    child.on("error", (err) => debugDevice("forward %d→%d spawn error: %s", localPort, devicePort, err.message));
     this.forwards.push(child);
     return child;
   }

This would surface port conflicts or process failures in debug logs rather than appearing as a generic timeout.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/serve-sim/src/wda.ts` around lines 334 - 345, The startForward
method currently spawns the usbmux child with stdio ignored so failures (like
port-in-use) are silent; modify the startForward function to attach listeners to
the spawned ChildProcess (the variable child returned by spawn) to handle
'error' and 'exit' events, log descriptive diagnostics (including localPort,
devicePort and child.pid) via the existing logger or throw a WdaSetupError when
the process fails to start or exits with a non-zero code, and ensure the child
is removed from this.forwards on exit to avoid stale references.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/serve-sim/src/device-server.ts`:
- Line 23: Remove the unused Server type from the import list so the import
reads only createServer, IncomingMessage and ServerResponse; the runtime
variable server (created via createServer) is already inferred so no explicit
Server annotation is needed—update the import line to drop Server and verify
there are no remaining references to the Server symbol (e.g., no type
annotations using Server).
- Around line 143-146: The data handler accumulating `raw` in the
`req.on("data", (chunk) => { ... })` callback currently calls `req.destroy()`
when `raw.length > 4096` without sending a response; change this so you first
send an HTTP 413 Payload Too Large to the client (e.g. set `res.statusCode =
413` or call `res.writeHead(413, ...)` and `res.end()` with a short message) and
then destroy the request to avoid the end handler race; update the
`req.on("data", ...)` branch that checks `raw.length > 4096` to perform these
steps instead of only calling `req.destroy()`.

In `@packages/serve-sim/src/wda.ts`:
- Line 30: The import list in packages/serve-sim/src/wda.ts includes an unused
symbol `execFile`; remove `execFile` from the named imports of "child_process"
so only `spawn`, `execFileSync`, and `type ChildProcess` remain (update the
import declaration that currently imports `spawn, execFile, execFileSync, type
ChildProcess`).

---

Nitpick comments:
In `@packages/serve-sim/src/wda.ts`:
- Around line 334-345: The startForward method currently spawns the usbmux child
with stdio ignored so failures (like port-in-use) are silent; modify the
startForward function to attach listeners to the spawned ChildProcess (the
variable child returned by spawn) to handle 'error' and 'exit' events, log
descriptive diagnostics (including localPort, devicePort and child.pid) via the
existing logger or throw a WdaSetupError when the process fails to start or
exits with a non-zero code, and ensure the child is removed from this.forwards
on exit to avoid stale references.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 525d5fd7-2240-40de-af85-a01973f34c38

📥 Commits

Reviewing files that changed from the base of the PR and between e0d98f6 and c226dd4.

📒 Files selected for processing (4)
  • packages/serve-sim/src/debug.ts
  • packages/serve-sim/src/device-server.ts
  • packages/serve-sim/src/index.ts
  • packages/serve-sim/src/wda.ts

Comment thread packages/serve-sim/src/device-server.ts Outdated
Comment thread packages/serve-sim/src/device-server.ts
Comment thread packages/serve-sim/src/wda.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant