feat: Apple TV support#76
Open
douglowder wants to merge 20 commits into
Open
Conversation
Captures the survey of iOS-specific code in serve-sim, the open design choices around Siri remote input transport and frame chrome, and a phased implementation sketch. Posted for discussion before any code lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Threads a `tv` option through findBootedDevice / pickDefaultDevice and the follow/detach/serve entry points. When --tv is passed, the auto- selection prefers booted tvOS sims and filters out the full-resolution 4K models to keep the streaming framebuffer at 1080p — the "(at 1080p)" 4K variants are kept since their framebuffer is 1080p. iOS-default behavior is unchanged; --tv is opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extend the runtime allowlist regex in listAllSimulators() to include tvOS, and add Apple TV / tvOS entries to deviceKind() and runtimeOrder() so the picker groups and sorts them sensibly. Apple TV falls after Vision Pro in the ordering — matches "less commonly used" positioning. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apple TV renders to a television, so the device frame has zero bezels and no inner radius — the stream fills the panel. SIMULATOR_SCREENS gains 1080p entries for plain "Apple TV" and the "(at 1080p)" 4K variants so aspect ratios resolve correctly without the iPhone fallback. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tvOS Siri-remote button cases to sendButton:
- remote_up / down / left / right — USB HID Keyboard arrow usages
(tvOS's focus engine listens to keyboard events directly)
- remote_select — USB HID Keyboard Return (0x28)
- remote_menu — Consumer Page Menu Escape (0x46)
- remote_tv — Consumer Page AC Home (0x223)
- remote_play_pause — Consumer Page Play/Pause (0xCD)
- remote_volume_up / volume_down — Consumer Page Volume Inc/Dec
(0xE9 / 0xEA)
- remote_siri — Consumer Page Voice Command (0xCF)
The Consumer-page buttons go through IndigoHIDMessageForHIDArbitrary
on target 0x15 (the tvOS remote routing target) — the path
Simulator.app uses to translate Mac media keys into simulator events.
This is the only transport that actually dispatches discrete Siri-
remote button events; the IndigoHIDMessageForButton source codes
extracted from Simulator.app's `appleRemote*ButtonAction:` selectors
all funnel through a generic back/home path and do not address the
individual buttons.
Verified live on a booted "Apple TV 4K (3rd generation) (at 1080p)"
sim: select opens apps, menu steps back, tv goes home, play_pause
toggles video playback, siri opens the voice command UI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SimulatorView gains a deviceType prop. When deviceType === "tv", a
window keydown listener maps ArrowKeys / Enter / Escape / PlayPause to
sendButton("remote_*"). client.tsx passes its existing deviceType
through so tvOS mode activates automatically when the picker lands on
an Apple TV.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends ButtonEventPayload with an optional holdMs field that plumbs from the `serve-sim button --hold [ms]` CLI flag through the WebSocket to the helper. The HIDInjector branches on holdMs in pressKey / pressConsumer — when set, the press is dispatched on buttonQueue so the press-sleep-release sequence doesn't block the input handler. The CLI also defers closing the WebSocket until hold-time-plus-50ms so the up event survives. Long-press is only meaningful for tvOS gestures, so the flag is honored by: - remote_select (focus context menu / wiggle mode) - remote_play_pause (Siri remote info / sleep on older remotes) - remote_menu (jump to home instead of stepping back) Other buttons silently ignore holdMs. Default duration when --hold is passed without an argument is 1500ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up the consolidated button mappings and --hold long-press plumbing in HIDInjector.swift, ClientManager.swift, main.swift, and Protocol.swift. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two issues, one commit: 1. Clicking the streamed image was firing touch events into the sim, which on tvOS the helper still forwards via the mouse-NSEvent path. That input was being interpreted as something the focus engine couldn't handle, blanking the screen and bouncing back to home. Fix: set pointerEvents="none" on the input overlay when deviceType === "tv", since Apple TV has no touchscreen anyway. 2. Pressing Enter fired no remote_select. The root cause was a duplicate-keystroke race: client.tsx's generic keyboard pass-through (which sends every key as a USB HID keystroke for typing into iOS apps) was firing in parallel with SimulatorView's tvOS remote-button handler. Both routed Enter to keyboard usage 0x28 — two press events for the same key, which the focus engine dropped as a duplicate. Arrow keys tolerated the dup; Enter didn't. Fix: short-circuit the client.tsx pass-through when deviceType === "tv". SimulatorView's remote handler is the single source of truth for tvOS navigation keys. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Apple TV / tvOS to the skill description and intro, extends the common-operations table to cover --tv startup and the remote_* button names + --hold flag, and adds a new "Apple TV (tvOS) support" section that documents: - Device selection (--tv flag, 1080p auto-pick) - The Siri-remote button set and its two transport routes (keyboard HID for directional/select, Consumer Page via HIDArbitrary for the discrete remote buttons) - Long-press behavior (--hold, default 1500ms) - Web-preview keyboard map - What's intentionally gated off on tvOS (mouse forwarding, keyboard pass-through) and why - What doesn't work / wasn't tested (tap, gesture, iOS buttons, camera, permissions, rotation, audible volume) Also updates the "do not invent button names" anti-pattern with the tvOS button set. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Threads tvOS coverage through the reference files that the SKILL.md overview points at: - buttons-rotation.md: full Siri-remote button table with effects and transports (keyboard HID vs Consumer-Page), a long-press section, and a tvOS-doesn't-rotate note in the rotation gotchas. - gestures.md: tvOS note at the top — tap and gesture don't reach an Apple TV; redirect to the remote_* buttons. - camera.md: tvOS callout — injection not supported / not validated. - permissions.md: tvOS warning — TCC schema differs, subcommand tested only against iOS. - endpoints.md: button event now documents the holdMs field, and the touch / multi-touch / orientation / keyboard rows are tagged with their tvOS applicability. - workflows.md: new Workflow 8 — driving a tvOS sim end to end, including what NOT to do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier "gate the iOS keystroke pass-through off entirely for tvOS" fix solved the duplicate-press race on Enter (3b2866c), but it also blocked letter / digit / Space / Backspace keys from reaching tvOS text-input modals — so the on-screen keyboard could only be typed by selecting each character. Narrow the gate: only the codes that SimulatorView's remote handler owns (ArrowUp/Down/Left/Right, Enter/NumpadEnter, Escape) are suppressed on tvOS. Every other key still passes through as a USB HID keyboard event, which tvOS routes into the focused text field. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The earlier docs said the keyboard pass-through was fully gated off for tvOS and that `type` was rarely useful — both no longer accurate after the previous commit. Update: - SKILL.md: the "Web preview keyboard map" section now notes that any non-navigation key is forwarded as a USB HID keystroke, and a new "Typing text" subsection covers both the web preview and `serve-sim type` as paths into a focused tvOS text field. The "doesn't work" list loses the `type` bullet. - endpoints.md: the `0x06` (JSON keyboard event) bullet now describes the selective filter — arrows / Enter / Escape suppressed on tvOS, everything else forwarded. - workflows.md: Workflow 8 gets a step demonstrating `serve-sim type` into a focused text field on tvOS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Top-level TV.md mirrors README.md's structure but focuses on the tvOS surface: --tv startup, Siri-remote button vocabulary with transports, --hold long-press, web-preview keyboard map, text input, end-to-end recipe, what doesn't work, and a transport-internals "How it works" section. Cross-links back to the agent skill docs. README.md's features bullet now mentions Apple TV with a TV.md pointer (using the same root-relative path style as the existing `skills/serve-sim` link, which only works because the README is symlinked at the repo root). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR adds comprehensive Apple TV (tvOS) simulator support to serve-sim, including device type definitions, tvOS runtime enumeration and device selection, Siri-remote button protocol implementation with configurable hold-press behavior, client-side input routing, CLI --tv and --hold option threading, and extensive user documentation covering tvOS-specific workflows and constraints. ChangesApple TV (tvOS) Support
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (5)
packages/serve-sim-client/src/simulator/SimulatorView.tsx (1)
1-1: 💤 Low valueFilename violates kebab-case guideline.
As per coding guidelines, TypeScript/JavaScript files should use kebab-case. This file should be named
simulator-view.tsxinstead ofSimulatorView.tsx.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/serve-sim-client/src/simulator/SimulatorView.tsx` at line 1, Rename the file from SimulatorView.tsx to simulator-view.tsx and update all import paths referencing "SimulatorView" to the new kebab-case filename (e.g., imports of SimulatorView or from '.../SimulatorView' should point to '.../simulator-view'); ensure the exported component name (SimulatorView) and any default export remain unchanged so internal symbols (component SimulatorView) still match existing references.packages/serve-sim-client/src/simulator/deviceFrames.tsx (2)
250-261: ⚡ Quick winDeviceFrameChrome lacks explicit tv case.
The switch statement doesn't handle
type === "tv"and will fall through to the defaultPhoneFrameChrome, which renders iPhone-specific chrome including a Dynamic Island. While the TV frame defines all bezels as 0 (making chrome invisible), explicitly returningnullor a dedicated borderless component for TV would be clearer and more maintainable.📐 Proposed fix to add explicit tv case
export function DeviceFrameChrome({ type = "iphone", streaming = false }: { type?: DeviceType; streaming?: boolean }) { switch (type) { case "ipad": return <IPadFrameChrome streaming={streaming} />; case "watch": return <WatchFrameChrome streaming={streaming} />; case "vision": return <VisionProFrameChrome streaming={streaming} />; + case "tv": + // Apple TV renders borderless; no chrome overlay needed. + return null; default: return <PhoneFrameChrome streaming={streaming} />; } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/serve-sim-client/src/simulator/deviceFrames.tsx` around lines 250 - 261, DeviceFrameChrome currently falls through to PhoneFrameChrome for unknown types, causing TV sims to render iPhone chrome; update the switch in DeviceFrameChrome to include an explicit "tv" case and return either null or a dedicated borderless TV component (e.g., a new TvFrameChrome or simply null) instead of PhoneFrameChrome so TV devices render without Dynamic Island/phone bezels; modify the switch that references DeviceFrameChrome and DeviceType to add the "tv" branch and ensure streaming prop is passed or ignored appropriately.
1-1: 💤 Low valueFilename violates kebab-case guideline.
As per coding guidelines, TypeScript/JavaScript files should use kebab-case. This file should be named
device-frames.tsxinstead ofdeviceFrames.tsx.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/serve-sim-client/src/simulator/deviceFrames.tsx` at line 1, The filename deviceFrames.tsx violates the kebab-case rule; rename the file to device-frames.tsx and update all imports/exports that reference deviceFrames.tsx (search for "deviceFrames" in import paths) so they point to device-frames.tsx, and ensure any tooling/config (exports index files or barrel files) that lists the old name is updated accordingly.TVOS-CONSIDERATIONS.md (1)
1-98: ⚡ Quick winConsider updating or archiving this planning document.
This document uses future tense and describes implementation phases, but based on the PR summary and other documentation, the tvOS support appears to be fully implemented. Consider either:
- Updating it to reflect the completed state (past tense, "what was done")
- Moving it to a historical/archive location
- Removing it if no longer needed
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TVOS-CONSIDERATIONS.md` around lines 1 - 98, The planning doc "TVOS-CONSIDERATIONS.md" is out of date; either convert it to a historical post-mortem or archive/remove it: update the prose from future to past tense and add a short "Status" section listing completed Phases (reference the Phase 1..Phase 5 headings and key changes like enumeration unlock and Siri-remote HID work), or move the file to an archive folder (or delete) and update any README/CLAUDE.md references to point to the new location or the completed implementation notes; ensure the file header clearly indicates "Archived" or "Post-mortem" and include links to the implemented artifacts (e.g., package names/sections mentioned in the document) so readers can find the actual code.TV.md (1)
152-164: ⚡ Quick winAdd language specifier to fenced code block.
The ASCII diagram should specify a language (or use
text) to comply with Markdown linting standards.📝 Proposed fix
-``` +```text ┌──────────────┐ simctl io ┌─────────────────┐ MJPEG / WS ┌─────────┐As per coding guidelines, the static analysis tool markdownlint-cli2 flagged this: "Fenced code blocks should have a language specified (MD040)."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@TV.md` around lines 152 - 164, The fenced ASCII diagram block lacks a language specifier and triggers MD040; update the opening fence for the diagram (the triple backticks above the ASCII art in TV.md) to include a language label such as "text" (e.g., change ``` to ```text) so the code block is explicitly marked and satisfies markdownlint; no other changes to the diagram content are needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/serve-sim-client/src/simulator/SimulatorView.tsx`:
- Around line 63-68: The JSDoc for deviceType mentions that "Space" is forwarded
to Siri-remote but the keyboard handler in SimulatorView that forwards
Enter/Escape/arrow keys does not include Space; either remove "Space" from the
deviceType JSDoc or update the key mapping in the SimulatorView keyboard handler
(the key-to-remote mapping that currently handles Enter, Escape and arrow keys)
to also recognize the Space key (e.g., handle ' ' and 'Space' key values and
forward them as the appropriate Siri-remote action).
In `@skills/serve-sim/references/workflows.md`:
- Line 261: Replace the hardcoded URL in the curl command "curl -s
http://localhost:3100/ax | jq '.children[0].AXLabel'" with the same dynamic port
discovery pattern used earlier in this document/workflows (i.e., resolve the
service port at runtime and interpolate it into the curl URL), so that the
command queries http://localhost:<discovered_port>/ax rather than always using
3100; update the line containing that curl invocation accordingly to use the
discovered port variable or lookup expression consistent with workflows 1–4.
- Line 272: The guidance in workflows.md conflicts with permissions.md: change
the sentence about "Treat `serve-sim permissions` results as authoritative on
tvOS" to state the opposite — that `serve-sim permissions` is unvalidated on
tvOS and its results should be treated as untested/unreliable because the TCC
layout differs; explicitly reference the `serve-sim permissions` subcommand and
align wording with the existing warning in permissions.md so readers know not to
treat the output as authoritative on tvOS.
- Line 260: Replace the invalid URL concatenation in the curl call: change the
string literal using the `%` character ("$URL%/.sim/api") to a proper shell-safe
form such as "${URL}/.sim/api" (or "$URL/.sim/api") so the curl command that
extracts '.streamUrl' with jq uses a valid URL; update the curl invocation in
the line with the comment "# discover the helper port" accordingly.
---
Nitpick comments:
In `@packages/serve-sim-client/src/simulator/deviceFrames.tsx`:
- Around line 250-261: DeviceFrameChrome currently falls through to
PhoneFrameChrome for unknown types, causing TV sims to render iPhone chrome;
update the switch in DeviceFrameChrome to include an explicit "tv" case and
return either null or a dedicated borderless TV component (e.g., a new
TvFrameChrome or simply null) instead of PhoneFrameChrome so TV devices render
without Dynamic Island/phone bezels; modify the switch that references
DeviceFrameChrome and DeviceType to add the "tv" branch and ensure streaming
prop is passed or ignored appropriately.
- Line 1: The filename deviceFrames.tsx violates the kebab-case rule; rename the
file to device-frames.tsx and update all imports/exports that reference
deviceFrames.tsx (search for "deviceFrames" in import paths) so they point to
device-frames.tsx, and ensure any tooling/config (exports index files or barrel
files) that lists the old name is updated accordingly.
In `@packages/serve-sim-client/src/simulator/SimulatorView.tsx`:
- Line 1: Rename the file from SimulatorView.tsx to simulator-view.tsx and
update all import paths referencing "SimulatorView" to the new kebab-case
filename (e.g., imports of SimulatorView or from '.../SimulatorView' should
point to '.../simulator-view'); ensure the exported component name
(SimulatorView) and any default export remain unchanged so internal symbols
(component SimulatorView) still match existing references.
In `@TV.md`:
- Around line 152-164: The fenced ASCII diagram block lacks a language specifier
and triggers MD040; update the opening fence for the diagram (the triple
backticks above the ASCII art in TV.md) to include a language label such as
"text" (e.g., change ``` to ```text) so the code block is explicitly marked and
satisfies markdownlint; no other changes to the diagram content are needed.
In `@TVOS-CONSIDERATIONS.md`:
- Around line 1-98: The planning doc "TVOS-CONSIDERATIONS.md" is out of date;
either convert it to a historical post-mortem or archive/remove it: update the
prose from future to past tense and add a short "Status" section listing
completed Phases (reference the Phase 1..Phase 5 headings and key changes like
enumeration unlock and Siri-remote HID work), or move the file to an archive
folder (or delete) and update any README/CLAUDE.md references to point to the
new location or the completed implementation notes; ensure the file header
clearly indicates "Archived" or "Post-mortem" and include links to the
implemented artifacts (e.g., package names/sections mentioned in the document)
so readers can find the actual code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1b92f915-512d-4b7a-a4b3-e828ffef98f1
📒 Files selected for processing (22)
TV.mdTVOS-CONSIDERATIONS.mdpackages/serve-sim-client/src/simulator/SimulatorView.tsxpackages/serve-sim-client/src/simulator/deviceFrames.tsxpackages/serve-sim/README.mdpackages/serve-sim/Sources/SimStreamHelper/ClientManager.swiftpackages/serve-sim/Sources/SimStreamHelper/HIDInjector.swiftpackages/serve-sim/Sources/SimStreamHelper/Protocol.swiftpackages/serve-sim/Sources/SimStreamHelper/main.swiftpackages/serve-sim/bin/serve-sim-binpackages/serve-sim/src/client/client.tsxpackages/serve-sim/src/client/utils/devices.tspackages/serve-sim/src/device.tspackages/serve-sim/src/index.tspackages/serve-sim/src/middleware.tsskills/serve-sim/SKILL.mdskills/serve-sim/references/buttons-rotation.mdskills/serve-sim/references/camera.mdskills/serve-sim/references/endpoints.mdskills/serve-sim/references/gestures.mdskills/serve-sim/references/permissions.mdskills/serve-sim/references/workflows.md
CodeRabbit's pre-merge "Docstring Coverage" check flagged the branch at 45.83% (threshold 80%). Document the functions added/modified for tvOS support: - HIDInjector.swift: sendButton (now covers the holdMs param and the remote_* button vocabulary); split the consumerTarget constant out with its own doc so the pressConsumer docstring attaches correctly. - client/utils/devices.ts: parseSimctlList, deviceKind, runtimeOrder. - deviceFrames.tsx: getDeviceType, simulatorAspectRatio, fallbackScreenSize, simulatorMaxWidth. - index.ts: button (holdMs behavior) and serve (--tv threading). Comment-only change; the helper binary is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DeviceFrameChrome fell through to PhoneFrameChrome for the "tv" type, overlaying iPhone bezels and a Dynamic Island on tvOS sims. Add an explicit "tv" case returning null — Apple TV outputs to a television, so the borderless DEVICE_FRAMES.tv geometry should render with no chrome. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- SimulatorView deviceType JSDoc: Space is not mapped to a Siri-remote button (it passes through as text input). Correct the doc to list only arrow keys / Enter / Escape as forwarded, and note other keys type normally. - workflows.md Workflow 8: fix the broken `"$URL%/.sim/api"` curl (stray %), and replace the hardcoded http://localhost:3100/ax with the runtime port-discovery pattern used in workflows 1-4. - workflows.md: reword the permissions bullet so it aligns with permissions.md — serve-sim permissions is unvalidated on tvOS and its results shouldn't be treated as authoritative. - TV.md: add a `text` language label to the "How it works" ASCII diagram fence to satisfy markdownlint MD040. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This reverts commit afe923a.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/serve-sim-client/src/simulator/SimulatorView.tsx (1)
64-68:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAlign tv keyboard docs with the actual key map.
This updated JSDoc now says Space is text input, but the inline comment at Line 550 still says Space is forwarded as a Siri-remote button. Please update that comment to match the real mapping.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/serve-sim-client/src/simulator/SimulatorView.tsx` around lines 64 - 68, The JSDoc in SimulatorView now states that Space is treated as text input for device kind "tv", but the inline comment near the key handling logic still claims Space is forwarded as a Siri-remote button; update that inline comment in the key-handling block (look for the handler function like onKeyDown / handleKeyDown or the switch that maps keys for device kind "tv") to reflect the actual mapping: Space should be documented as passing through as text input (not forwarded), while arrow keys, Enter and Escape remain Siri-remote button presses; keep the rest of the comment consistent with the JSDoc wording.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@packages/serve-sim-client/src/simulator/SimulatorView.tsx`:
- Around line 64-68: The JSDoc in SimulatorView now states that Space is treated
as text input for device kind "tv", but the inline comment near the key handling
logic still claims Space is forwarded as a Siri-remote button; update that
inline comment in the key-handling block (look for the handler function like
onKeyDown / handleKeyDown or the switch that maps keys for device kind "tv") to
reflect the actual mapping: Space should be documented as passing through as
text input (not forwarded), while arrow keys, Enter and Escape remain
Siri-remote button presses; keep the rest of the comment consistent with the
JSDoc wording.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 6f6c89d9-5192-4a07-9c9f-86b4e0a3a150
📒 Files selected for processing (4)
TV.mdpackages/serve-sim-client/src/simulator/SimulatorView.tsxpackages/serve-sim-client/src/simulator/deviceFrames.tsxskills/serve-sim/references/workflows.md
✅ Files skipped from review due to trivial changes (2)
- skills/serve-sim/references/workflows.md
- TV.md
The comment above the tvOS keydown effect still claimed Space was forwarded as a Siri-remote button. Align it with the actual map and the deviceType JSDoc: arrows / Enter / Escape (plus PlayPause and AppleTV media keys) are forwarded; Space and other keys pass through as text input. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
serve-sim for Apple TV (tvOS)
The
npx serveof Apple TV simulators.Drive a booted tvOS simulator from the browser or the CLI — focus navigation, the Siri remote's discrete buttons, long presses, and text input all over the same MJPEG stream + WebSocket channel that powers the iOS preview.
npx serve-sim --tv # → Preview at http://localhost:3200tvOS support is opt-in via a top-level
--tvflag. Apple TV simulators don't share much input surface with iOS (no touchscreen, different button transports), soserve-sim's iOS-default behaviors stay iOS-only unless--tvis passed.Features
--hold [ms]) for the buttons that have a discrete tvOS long-press gesture.serve-sim type "..."CLI as iOS, for scripted text input.Quick start
Boot an Apple TV simulator (Xcode → Open Developer Tool → Simulator, or
xcrun simctl boot "Apple TV (3rd generation)"etc.).Start serve-sim with
--tv:Open the preview URL in your browser, or drive the sim from the CLI.
When auto-selecting,
--tvprefers a 1080p Apple TV sim and skips full-resolution 4K variants — the 3840×2160 framebuffer chokes the H.264 encoder. The "(at 1080p)" 4K sims are kept because they output a 1080p framebuffer. Pass-d <name>to override.Without
--tv,findBootedDevice()still prefers an iOS sim, so existing iOS workflows are unaffected.Siri-remote buttons
remote_up/remote_down/remote_left/remote_rightremote_selectremote_menuremote_tvremote_play_pauseremote_volume_up/remote_volume_downremote_siriWhy two transports
Directional keys +
remote_selectride the keyboard HID path because tvOS's focus engine listens to USB-HID keyboard events natively. The discrete remote buttons (menu,tv,play_pause,volume_*,siri) go throughIndigoHIDMessageForHIDArbitraryon the Consumer Page with target0x15(the tvOS remote routing target).Long press
--hold [ms]holds the press forms(default1500) before releasing. Honored only by the buttons with a discrete tvOS long-press gesture; every other button silently ignores it.remote_selectremote_play_pauseremote_menuserve-sim button remote_select --hold # 1500ms hold serve-sim button remote_play_pause --hold 800Web preview controls
When the preview is showing a tvOS sim, these keys map to Siri-remote events:
remote_up/down/left/rightremote_selectremote_menuPlayPause(Mac media key)remote_play_pauseAppleTVremote_tvEvery other key (letters, digits, Space, Backspace, Tab, …) is forwarded as a USB HID keyboard event, so it lands in whatever text field has focus on the sim — the on-screen keyboard in Settings, account fields, search bars, etc.
Mouse/touch forwarding on the stream surface is turned off automatically for tvOS. Apple TV has no touchscreen, and clicks were being interpreted by the helper's mouse-NSEvent path and bouncing the sim back to the home screen.
Typing text
Both paths reach a focused text field on tvOS:
The web preview forwards every non-navigation key as a USB HID keystroke.
serve-sim type "..."sends the same keystrokes from the CLI without needing the browser:If a text field is not focused, the keystrokes still arrive at the helper but tvOS has nowhere to route them — navigate focus to the field with the remote buttons first.
End-to-end example
What doesn't work on tvOS
serve-sim tapandserve-sim gesture— no touchscreen on Apple TV.home,swipe_home,app_switcher,lock,siri,side_button) — they target iOS hardware buttons. Useremote_tvfor home andremote_sirifor Siri.serve-sim rotate— Apple TV is always landscape.serve-sim camera <bundle-id>— camera injection isn't supported on tvOS. tvOS apps rarely use AVFoundation, and the dylib injection path was validated against iOS sims only.serve-sim permissions— the TCC database schema and push-notification state layout on tvOS differ from iOS. The subcommand was tested only against iOS.remote_volume_up/remote_volume_down— the keystroke lands but the tvOS sim doesn't model speaker volume, so the press is a no-op.How it works
The same Swift helper drives every Apple simulator family. tvOS-specific input is layered on:
remote_*inHIDInjector.sendButton.pressConsumer(usage:)helper that callsIndigoHIDMessageForHIDArbitrary(target=0x15, usagePage=0x0C, usage, direction)— the only transport that dispatches per-button Siri-remote events into the sim.--holdsupport: the press / sleep / release sequence runs on a dedicated dispatch queue so the WebSocket handler thread isn't blocked.See also
skills/serve-sim/SKILL.md— agent-facing skill, with a dedicated "Apple TV (tvOS) support" section.skills/serve-sim/references/buttons-rotation.md— source-of-truth button vocabulary.skills/serve-sim/references/workflows.md— Workflow 8 is the end-to-end tvOS recipe.Summary by CodeRabbit
New Features
Documentation