diff --git a/Docs/multicam.md b/Docs/multicam.md new file mode 100644 index 00000000..4ac28bab --- /dev/null +++ b/Docs/multicam.md @@ -0,0 +1,217 @@ +# Multicam Director mode — architecture + +One phone (the **director**) drives up to four camera phones over the existing +Stormo/QUIC transport, showing a live preview of every angle and firing +**synchronized** photo capture and video recording across all of them. Each +camera saves full-resolution media locally; the footage then auto-collects to +the director, stamped with alignment metadata so any editor can line the angles +up. + +The shape is deliberately lopsided: **one director actor, N unchanged 1:1 +cameras.** A camera runs the same `SessionCoordinator` it always has — it never +learns it is one of several. Everything multicam lives on the director, in +`MulticamController`. A single-camera session never enters this code at all: the +scanner hands off to `MulticamController` only when two or more cameras connect, +and every wire message here is gated on a `supports_multicam` capability flag, +so a 9.0.x phone pairs as an ordinary single camera. + +> Free tier = 2 cameras, Pro = 4. The whole feature is behind +> `FeatureFlags.ENABLE_MULTICAM`. + +## Components & connections + +Four isolation domains. The director's UI is main-actor; `MulticamController` is +an actor with a single FIFO inbox; the transport is shared; each camera is its +own process on its own phone. + +```mermaid +flowchart LR + subgraph UI["Director UI · main actor"] + VC["MulticamViewController"] + VM["MulticamViewModel
lanes: [CameraLane]"] + LANE["CameraLane ×N
FrameStreamReceiver + FrameDisplayModel"] + VIEW["MulticamView · RigTray · Grid"] + end + + subgraph CTRL["MulticamController · actor"] + INBOX["tell → AsyncStream FIFO inbox → pump"] + LINKS["links: [PeerID: CameraLink]
the source of truth"] + SINKS["frameSinks: [PeerID: FrameSink]"] + end + + subgraph XPORT["Transport"] + MP["MultipeerService
one QUIC session, N peers"] + end + + subgraph CAMS["Camera phones ×N"] + CAM["SessionCoordinator (camera role)
1:1, unchanged"] + end + + VC -- "commands: capturePhoto, setFocusedPeer …
(nonisolated → tell)" --> INBOX + MP -- "transport events: frames, acks,
connect/disconnect (nonisolated → tell)" --> INBOX + INBOX --> LINKS + LINKS -- "one coalesced main-hop:
lane + rig snapshots" --> VM + VM --> LANE --> VIEW + LINKS -- "send(cmd, to: peer)" --> MP + SINKS -- "per-lane preview frame" --> LANE + MP <-- "QUIC" --> CAM +``` + +The load-bearing property: **both** UI commands and transport events enter the +actor the same way — a `nonisolated` method that calls `tell(_:)`, appending to +one `AsyncStream`. A single pump task (`for await msg in stream`) processes them +one at a time, in arrival order. There is no second door. See +[Design rules](#design-rules). + +## The camera link — one source of truth per camera + +`CameraLink` (a `final class`, actor-confined) is the only place a camera's +state is declared: status, capabilities, clock estimator, capture outcome, +recording flag, stream profile, collection progress. The UI is a projection of +it: + +``` +CameraLink (actor truth) + │ snapshot (one computed property) + ▼ +MulticamLaneInfo (Sendable value) + │ one coalesced hop to main + ▼ +CameraLane (@Published) → SwiftUI tile +``` + +Mutations never publish by hand. The controller mutates a `CameraLink` and marks +the lanes dirty; a single coalescing step turns the current links into +`[MulticamLaneInfo]` and hands them to the main actor once per pump message. A +state change can never be "forgotten" on the way to the screen, because +publishing is a consequence of mutation, not a separate call. + +## Live preview — five hops, four domains + +A preview frame crosses every isolation boundary exactly once, and lands on +exactly one tile: + +```mermaid +flowchart LR + A["didReceiveFrame
nonisolated · transport thread"] + -->|tell| B["inbox pump
actor · FIFO"] + B -->|"frameSinks[peerId]"| C["FrameStreamReceiver.receive
decode queue"] + C -->|"onImage → main"| D["FrameDisplayModel
main · @Published image"] + D -->|publish| E["LiveFrameView
this lane's tile only"] + B -->|"RequestFrame → source peer only"| F["credit-window ack (Seam B)"] +``` + +Two things make this robust: + +- **Rendering isolation.** Each lane owns its own `FrameStreamReceiver` and + `FrameDisplayModel`, so a frame from camera B publishes into camera B's model + and re-renders only camera B's tile — never camera A, never the chrome. +- **Sink routing, not display calls.** The actor routes frames through a + per-lane `@Sendable` closure (`MulticamFrameSink`) registered when the lane is + created. The actor never calls into a `UIViewController`; it calls a Sendable + sink that hops to its own decode queue. + +## Synced capture — the clock trick + +The director never says "everyone shoot now" (that skews by 10–40 ms as the +command arrives at each camera at a different time). It measures each camera's +clock offset and schedules a shutter instant expressed in *that camera's* clock. + +```mermaid +sequenceDiagram + participant UI as Director UI + participant MC as MulticamController (actor) + participant A as Camera A + participant B as Camera B + + Note over MC,B: every ~30s per lane
ClockSyncPing(t0) → Pong(echo, camClock)
→ offset ±2–5 ms (min-RTT of 5) + + UI->>MC: capturePhoto() [tell → inbox] + MC->>MC: fireAt = now + 150 ms · captureId + MC->>A: ScheduledCapture(fireAt + offsetA, captureId) + MC->>B: ScheduledCapture(fireAt + offsetB, captureId) + A-->>MC: ScheduledCaptureAck(captureId) + B-->>MC: ScheduledCaptureAck(captureId) + Note over MC: state = capturingPhoto(acksRemaining)
per-lane 3 s timeout → lane .failed, capture proceeds + Note over A,B: each fires at that instant on its OWN clock
→ sub-frame skew at 30 fps + A-->>MC: TakePicResp(stamped photo) + B-->>MC: TakePicResp(stamped photo) + MC->>MC: save RS___cam to Photos · lane .collected + MC-->>UI: lane snapshots (badges) +``` + +Video start and stop ride the same mechanism +(`ScheduledStartRecording` / `ScheduledStopRecording`), so every clip shares a +start and stop anchor and the lengths match. Photos come back inline in +`TakePicResp`; video clips transfer as resources afterward (see +[Known debts](#known-debts)). + +## Resilient camera — a drop mid-recording + +The one behavior a camera changes in a director session: if the director drops +while recording, the camera keeps rolling instead of stopping. It is gated on an +explicit `CameraDriver` state (`.solo` vs `.director`), latched when the first +scheduled multicam command arrives and cleared only when the session truly ends +— so a single-camera session is byte-identical. + +```mermaid +sequenceDiagram + participant MC as MulticamController + participant A as Camera A (recording) + participant B as Camera B (recording) + + Note over A,B: cameraDriver = .director (latched at scheduled start) + Note over MC: director drops B's link + MC->>MC: link[B].status = .reconnecting
armReconnect(B) · other lanes untouched + Note over B: DisconnectPeer while .director →
KEEP recording, enter reconnect (do NOT stop) + MC->>B: browser re-finds B → re-invite (only reconnecting lanes) + B-->>MC: reconnected · lane .linked + Note over B: clip saved when the scheduled stop fires,
or when the user stops on-device +``` + +## Design rules + +These are the invariants worth preserving through future changes. + +- **Single-entry inbox = arrival-order invariant.** Every input — UI command or + transport event — becomes a message through `tell`. Because there is one + queue, the actor always sees the world in the order things happened: a shutter + tap that arrives after a disconnect is processed after it, so it can never fan + a capture out to a camera the actor already knows is gone. (Contrast: direct + `await` command methods would interleave with queued events at every `await`.) +- **Commands are requests; results are state.** A command returns nothing and + cannot fail loudly — it enqueues. Outcomes (captured / failed / reconnecting / + transferring) arrive later as published `CameraLink` state and render as tile + badges. The UI is a pure function of that state, not of any command's return. +- **Framing belongs to a camera; the shot belongs to the rig.** Per-camera + controls (zoom, focus, flash, torch, lens, camera flip) address the *focused* + camera only. Rig controls (shutter, record, timer, quality/HDR) fan out to + all. Nothing per-camera ever broadcasts. +- **Rendering isolation per lane.** One decoder and one published image per + camera; a frame from one camera can only touch its own tile. +- **Additive, gated wire.** Every multicam action (`ClockSyncPing`, + `ScheduledCapture`, `ScheduledStartRecording`, `ScheduledStopRecording`, + `SetStreamProfile`, `RequestVideoResend`) is an appended FlatBuffers action + gated on the `supports_multicam` capability, so old peers pair normally and + never receive a message they would misread. + +## Known debts + +Honest list of what is deliberately first-draft, for whoever picks this up next. + +- **Fixed-delay transfer stagger.** Auto-collect delays each camera's clip send + by `(cameraIndex − 1) × staggerSeconds` so N×4K clips don't hit the link at + once (`SessionCoordinator.handleSendVideoResource`). It is open-loop: if one + transfer runs long, the next can still overlap. The real fix is + director-coordinated turn-taking (the director grants "your turn" per lane). + Worth doing once real-world 4-camera 4K load is observed post-release. +- **`RequestVideoResend` shape.** Retry re-sends `lastMulticamClipURL`, which + holds only the *last* clip; it reuses `capture_id` loosely and depends on the + file still existing. Fine for one-clip-at-a-time retry; it would need a real + per-capture store to retry an older clip. +- **`PeerSessionCore` extraction pending.** Reconnect, the per-peer version + gate, capability parsing, and the frame pump exist in both + `SessionCoordinator` (camera + 1:1 monitor) and `MulticamController`. They + have begun to diverge; extracting a shared core is the highest-value + robustness refactor, deferred until after the 9.1 release so it doesn't ride + the same release as the feature. diff --git a/RemoteCam/CameraControlling.swift b/RemoteCam/CameraControlling.swift index 9bc9da61..fc6de76b 100644 --- a/RemoteCam/CameraControlling.swift +++ b/RemoteCam/CameraControlling.swift @@ -32,6 +32,14 @@ protocol CameraControlling: AnyObject, Sendable { func takePicture(_ sendMediaToRemote: Bool) func startRecordingVideo() func stopRecordingVideo(_ shouldSendVideo: Bool) + /// Multicam only: the sync metadata for the recording about to start, so + /// the pipeline stamps the .mov and names it under the shared RS_ group. + /// Cleared (nil) for ordinary single-camera recording — never called + /// there, so that path is byte-identical. + func setVideoSyncMetadata(_ metadata: CaptureSyncMetadata?) + /// Multicam only: retune the live preview encoder (resolution/bitrate/fps) + /// for tiered director previews. Never called in a single-camera session. + func applyStreamProfile(_ profile: StreamProfile) func setZoom(zoomFactor: CGFloat) async throws -> (CGFloat, CameraLensType, RemoteCmd.ZoomRange) /// Sets the focus/exposure point of interest from a monitor tap. `x`/`y` are diff --git a/RemoteCam/CameraLink.swift b/RemoteCam/CameraLink.swift new file mode 100644 index 00000000..0796c33a --- /dev/null +++ b/RemoteCam/CameraLink.swift @@ -0,0 +1,105 @@ +// +// CameraLink.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import Foundation +import MPCCompat +import Stormo + +/// One camera in a multicam director session. The director holds one of these +/// per connected camera, keyed by `MCPeerID`; it is the multicam analog of the +/// single link that `SessionCoordinator` holds for a 1:1 monitor. (Frame +/// decoding lives entirely on the UI side, one `FrameStreamReceiver` per +/// `CameraLane`; this actor-domain link never touches pixels.) +/// +/// A reference type, not a struct: the `MulticamController` actor mutates it in +/// place as capabilities and clock samples arrive — a value type would force a +/// dictionary read-modify-write on every update. +final class CameraLink { + + /// Progress of one lane's footage transfer to the director after a take. + enum LaneCollectionState: Equatable { + case idle + case transferring(Double) // 0…1 + case collected + case failed + } + + enum Status: Equatable { + /// The session is up and frames are expected. + case linked + /// The link dropped; the director is re-browsing to invite it back. + /// The tile stays on screen (last frame frozen) rather than vanishing. + case reconnecting + /// The peer is gone for good (removed by the user, or version-refused). + case failed + } + + let peerID: MCPeerID + let displayName: String + var status: Status = .linked + /// Whether this lane is the focused one (the controller maintains it when + /// focus changes, so the snapshot is a pure read). + var isFocused = false + + /// The most recent capabilities the camera advertised, or nil until the + /// first exchange completes. `supportsMulticam` gates the multicam-only + /// wire messages (clock sync now; scheduled capture in later PRs). + var capabilities: RemoteCmd.CameraCapabilitiesResp? + var supportsMulticam: Bool { capabilities?.supportsMulticam ?? false } + + /// Rolling clock-offset estimate for this camera, fed by ClockSyncPong. + /// Stored here so a future synced capture can schedule on the camera's own + /// clock (PR4); PR3 only measures and surfaces it. + var clockEstimator = ClockOffsetEstimator() + var latestOffset: ClockOffsetSample? { clockEstimator.best } + + /// Monitor side: this lane has produced at least one VP9 frame, proving the + /// camera speaks VP9 — the gate for sending it `RequestKeyframe` (mirrors + /// `SessionCoordinator.monitorReceivedVP9Frame`, but per camera). + var sawVP9 = false + + /// How this camera answered the most recent synced capture (nil before the + /// first). Drives the tile's captured/failed badge. + var captureOutcome: CaptureOutcome? + + /// This camera is rolling as part of a synced recording — drives the tile's + /// REC badge. + var isRecording = false + + /// The preview profile most recently pushed to this camera, so the director + /// only re-sends `SetStreamProfile` when the tier actually changes. + var lastSentProfile: StreamProfile? + + /// A late joiner (or a device-switched camera) that cannot honor the + /// running rig video quality — its tile is badged and the tray offers a + /// re-match rather than silently changing the rig. + var needsQualityRematch = false + + /// Where this lane's footage is in the post-take auto-collect to the + /// director. Drives the tile's transfer progress / done / failed badge. + var collection: LaneCollectionState = .idle + + init(peerID: MCPeerID) { + self.peerID = peerID + self.displayName = peerID.displayName + } + + /// The single source of the UI snapshot for this lane — every displayed + /// field is declared exactly once, here. + var snapshot: MulticamLaneInfo { + MulticamLaneInfo( + peerID: peerID, + displayName: displayName, + status: status, + isFocused: isFocused, + clockOffsetMillis: latestOffset?.offsetMillis, + captureOutcome: captureOutcome, + isRecording: isRecording, + needsQualityRematch: needsQualityRematch, + collection: collection) + } +} diff --git a/RemoteCam/CameraRig.swift b/RemoteCam/CameraRig.swift index 9e117ee5..d27087d5 100644 --- a/RemoteCam/CameraRig.swift +++ b/RemoteCam/CameraRig.swift @@ -549,6 +549,14 @@ extension CameraRig: CameraControlling { pipeline.stopRecording(shouldSendVideo) } + func setVideoSyncMetadata(_ metadata: CaptureSyncMetadata?) { + pipeline.pendingSyncMetadata = metadata + } + + func applyStreamProfile(_ profile: StreamProfile) { + streamingCoordinator.applyStreamProfile(profile) + } + func updateTimerCountdown(value: Int) { OperationQueue.main.addOperation { if value > 0 { diff --git a/RemoteCam/CaptureEngine.swift b/RemoteCam/CaptureEngine.swift index 405eb268..76019483 100644 --- a/RemoteCam/CaptureEngine.swift +++ b/RemoteCam/CaptureEngine.swift @@ -938,6 +938,9 @@ final class CaptureEngine: NSObject, AVCapturePhotoCaptureDelegate { // This build understands SetCameraPreviewMode; advertise the current // persisted mode so the monitor reflects it from the first exchange. supportsPreviewMode: true, + // Tied to the flag so cameras start advertising multicam the same + // release the director UI ships. + supportsMulticam: FeatureFlags.ENABLE_MULTICAM, previewMode: CameraPreviewModeStore().load(), error: nil ) diff --git a/RemoteCam/CaptureSyncMetadata.swift b/RemoteCam/CaptureSyncMetadata.swift new file mode 100644 index 00000000..fc070342 --- /dev/null +++ b/RemoteCam/CaptureSyncMetadata.swift @@ -0,0 +1,139 @@ +// +// CaptureSyncMetadata.swift +// RemoteShutter +// +// Created by Dario Lencina on 2026. +// Copyright © 2026 Security Union. All rights reserved. +// + +import AVFoundation +import Foundation +import ImageIO +import UniformTypeIdentifiers + +/// Alignment metadata attached to every clip and photo captured in a multicam +/// director session. Each camera saves full-res media locally; these fields +/// are what let any editor (CapCut, FCP, Resolve) line the angles up without +/// the files ever leaving the phones. +/// +/// The `anchorMillis` timestamp is the scheduled fire time on the *director's* +/// clock, so it is identical across all N cameras' clips for one capture — +/// that shared value is the alignment key. `clockOffsetMillis`/ +/// `roundTripMillis` record how good this camera's clock estimate was when it +/// fired (a quality hint, not part of alignment). +struct CaptureSyncMetadata: Codable, Equatable { + /// One per multicam rig session (director generates it). + let sessionID: String + /// One per shutter press / record start, shared by every camera in the rig. + let captureID: String + /// Stable 1-based index of this camera in the rig, for humans and filenames. + let cameraIndex: Int + /// Scheduled fire time in ms on the director's clock — the alignment key. + let anchorMillis: UInt64 + /// This camera's estimated clock offset vs the director when it fired (ms). + let clockOffsetMillis: Int64 + /// RTT of the offset estimate (ms); smaller = tighter sync. + let roundTripMillis: Int64 + + /// QuickTime metadata keys, reverse-DNS in the `mdta` keyspace. + enum QuickTimeKey { + static let anchor = "com.remoteshutter.syncAnchorMs" + static let capture = "com.remoteshutter.captureId" + static let session = "com.remoteshutter.sessionId" + static let offset = "com.remoteshutter.clockOffsetMs" + } + + /// `RS___cam` — groups one capture's files across cameras + /// when they land in a shared folder or an editor's media bin. Uses the + /// first UUID group so names stay readable. + var filenamePrefix: String { + "RS_\(Self.shortID(sessionID))_\(Self.shortID(captureID))_cam\(cameraIndex)" + } + + /// Items for `AVAssetWriter.metadata` so the values travel inside the + /// .mov itself and survive export/AirDrop. + func quickTimeMetadataItems() -> [AVMetadataItem] { + [ + Self.item(key: QuickTimeKey.anchor, value: NSNumber(value: anchorMillis)), + Self.item(key: QuickTimeKey.capture, value: captureID as NSString), + Self.item(key: QuickTimeKey.session, value: sessionID as NSString), + Self.item(key: QuickTimeKey.offset, value: NSNumber(value: clockOffsetMillis)), + ] + } + + /// JSON blob for the photo path (EXIF UserComment) and the Documents + /// sidecar. Sorted keys so output is deterministic and testable. + func jsonString() -> String? { + let encoder = JSONEncoder() + encoder.outputFormatting = [.sortedKeys] + guard let data = try? encoder.encode(self) else { return nil } + return String(data: data, encoding: .utf8) + } + + static func fromJSONString(_ string: String) -> CaptureSyncMetadata? { + guard let data = string.data(using: .utf8) else { return nil } + return try? JSONDecoder().decode(CaptureSyncMetadata.self, from: data) + } + + /// Re-encode `imageData` with this shot's sync fields embedded in EXIF, so + /// the alignment travels inside the photo itself (survives export/AirDrop). + /// The alignment key (`anchorMillis`) rides in `UserComment` as opaque JSON; + /// `DateTimeOriginal`/`SubSecTimeOriginal` are the camera's **wall clock at + /// capture** (`capturedAt`) — EXIF's meaning is "when the photo was taken", + /// and `anchorMillis` is monotonic uptime, not a wall-clock date. Format + /// (JPEG/HEIC) is preserved. Returns the original data unchanged if + /// re-encoding isn't possible, so a stamping failure never costs the photo. + func stamped(_ imageData: Data, capturedAt: Date = Date()) -> Data { + guard let source = CGImageSourceCreateWithData(imageData as CFData, nil), + let type = CGImageSourceGetType(source) else { return imageData } + + var properties = (CGImageSourceCopyPropertiesAtIndex(source, 0, nil) + as? [CFString: Any]) ?? [:] + var exif = (properties[kCGImagePropertyExifDictionary] as? [CFString: Any]) ?? [:] + + if let json = jsonString() { + exif[kCGImagePropertyExifUserComment] = json + } + exif[kCGImagePropertyExifDateTimeOriginal] = Self.exifDateFormatter.string(from: capturedAt) + let subsec = Int((capturedAt.timeIntervalSince1970.truncatingRemainder(dividingBy: 1)) * 1000) + exif[kCGImagePropertyExifSubsecTimeOriginal] = String(format: "%03d", subsec) + properties[kCGImagePropertyExifDictionary] = exif + + let output = NSMutableData() + guard let dest = CGImageDestinationCreateWithData( + output, type, 1, nil) else { return imageData } + CGImageDestinationAddImageFromSource(dest, source, 0, properties as CFDictionary) + guard CGImageDestinationFinalize(dest) else { return imageData } + return output as Data + } + + /// A Photos `originalFilename` for this shot, e.g. + /// `RS___cam2.heic`. Groups a capture's files across cameras. + func photoFilename(isHEIC: Bool) -> String { + "\(filenamePrefix).\(isHEIC ? "heic" : "jpg")" + } + + /// A Photos `originalFilename` for this clip, e.g. `RS___cam2.mov`. + func videoFilename() -> String { "\(filenamePrefix).mov" } + + /// EXIF wants `yyyy:MM:dd HH:mm:ss` in the local zone. + private static let exifDateFormatter: DateFormatter = { + let f = DateFormatter() + f.dateFormat = "yyyy:MM:dd HH:mm:ss" + f.locale = Locale(identifier: "en_US_POSIX") + return f + }() + + private static func shortID(_ uuidString: String) -> String { + String(uuidString.prefix(8)).lowercased() + } + + private static func item(key: String, value: NSCopying & NSObjectProtocol) -> AVMetadataItem { + let item = AVMutableMetadataItem() + item.identifier = AVMetadataItem.identifier(forKey: key, keySpace: .quickTimeMetadata) + item.keySpace = .quickTimeMetadata + item.key = key as NSString + item.value = value + return item + } +} diff --git a/RemoteCam/ClockOffsetEstimator.swift b/RemoteCam/ClockOffsetEstimator.swift new file mode 100644 index 00000000..2bc4c8e4 --- /dev/null +++ b/RemoteCam/ClockOffsetEstimator.swift @@ -0,0 +1,74 @@ +// +// ClockOffsetEstimator.swift +// RemoteShutter +// +// Created by Dario Lencina on 2026. +// Copyright © 2026 Security Union. All rights reserved. +// + +import Foundation + +/// The millisecond clock both ends of a clock-sync exchange read. Monotonic +/// while the app runs (never jumps with NTP/wall-clock changes), but it +/// pauses in deep sleep and restarts per boot — which is why offsets are +/// re-estimated on every foreground and never persisted. +enum SyncClock { + static func nowMillis() -> UInt64 { + DispatchTime.now().uptimeNanoseconds / 1_000_000 + } +} + +/// One ping/pong exchange reduced to an offset estimate. +struct ClockOffsetSample: Equatable { + /// camera clock − director clock, in ms. Adding this to a director + /// timestamp yields the same instant on the camera's clock. + let offsetMillis: Int64 + /// Round trip of the exchange; smaller = tighter estimate (the error + /// bound is the RTT asymmetry, at most rtt/2). + let roundTripMillis: Int64 +} + +/// NTP-style offset estimation over the session link: keep a short window of +/// exchanges and trust the minimum-RTT one — a queued or retransmitted +/// exchange has a large RTT and an unreliable midpoint, so it should never +/// outvote a clean one. +/// +/// Pure value type: callers supply every timestamp, so tests need no clocks. +struct ClockOffsetEstimator { + static let windowSize = 5 + + private(set) var samples: [ClockOffsetSample] = [] + + /// Record one exchange: director sent at `t0Millis`, camera stamped + /// `cameraClockMillis` at receipt, director received the pong at + /// `t3Millis` (all in each side's own `SyncClock`). Returns the sample, + /// or nil for a nonsensical exchange (pong before ping — a stale reply + /// from before a clock reset). + @discardableResult + mutating func recordExchange(t0Millis: UInt64, + cameraClockMillis: UInt64, + t3Millis: UInt64) -> ClockOffsetSample? { + guard t3Millis >= t0Millis else { return nil } + let rtt = Int64(t3Millis - t0Millis) + let directorMidpoint = Int64(bitPattern: t0Millis) + rtt / 2 + let sample = ClockOffsetSample( + offsetMillis: Int64(bitPattern: cameraClockMillis) - directorMidpoint, + roundTripMillis: rtt) + samples.append(sample) + if samples.count > Self.windowSize { + samples.removeFirst(samples.count - Self.windowSize) + } + return sample + } + + /// The trusted estimate: the minimum-RTT sample in the window. + var best: ClockOffsetSample? { + samples.min { $0.roundTripMillis < $1.roundTripMillis } + } + + /// Drop everything — called when the clocks may have moved under us + /// (either side backgrounded, or the peer reconnected). + mutating func reset() { + samples.removeAll() + } +} diff --git a/RemoteCam/DeviceScannerView.swift b/RemoteCam/DeviceScannerView.swift index c7c86c7a..cf4532e2 100644 --- a/RemoteCam/DeviceScannerView.swift +++ b/RemoteCam/DeviceScannerView.swift @@ -14,6 +14,15 @@ struct DeviceScannerView: View { let onShareApp: () -> Void let onOpenSettings: () -> Void let onHelp: () -> Void + /// Multicam only: select every discovered camera up to the tier cap (pure). + var onSelectAll: (() -> Void)? = nil + /// Multicam only: connect the selected cameras (the bottom CTA). + var onConnectSelected: (() -> Void)? = nil + + /// The scanner is in multicam edit-mode selection (monitor role, flag on). + private var isMulticamScanner: Bool { + FeatureFlags.ENABLE_MULTICAM && viewModel.role == .monitor + } /// Peer-link state; the reconnect overlay is a function of it. @ObservedObject var peerLink: PeerLinkStatus = .shared @@ -34,10 +43,38 @@ struct DeviceScannerView: View { connectingOverlay } + if isMulticamScanner, let onConnectSelected { + connectSelectedButton(onConnectSelected) + } + PeerLinkOverlay(status: peerLink) } } + /// The bottom "Connect (N)" CTA: fires the invites for the selected set. + /// Disabled at N = 0; hidden entirely during the connecting phase. + @ViewBuilder + private func connectSelectedButton(_ action: @escaping () -> Void) -> some View { + if viewModel.multicamPhase == .selecting { + VStack { + Spacer() + Button(action: action) { + Text(String(format: NSLocalizedString("Connect (%d)", comment: "connect N selected cameras"), + viewModel.multicamSelectionCount)) + .font(.headline) + .foregroundColor(.white) + .frame(maxWidth: .infinity) + .padding() + .background(viewModel.canConnectMulticam ? AppTheme.accent : Color.gray) + .clipShape(RoundedRectangle(cornerRadius: 14)) + } + .disabled(!viewModel.canConnectMulticam) + .padding(.horizontal, 24) + .padding(.bottom, 24) + } + } + } + // MARK: - Peer List private var peerList: some View { @@ -46,37 +83,13 @@ struct DeviceScannerView: View { statusBadge .padding(.top, 8) + if isMulticamScanner && viewModel.showsMulticamSelectAll { selectAllRow } + ForEach(viewModel.connectedPeers, id: \.self) { peer in Button { onSelectPeer(peer) } label: { - HStack(spacing: 14) { - Image(systemName: "iphone.radiowaves.left.and.right") - .font(.title3) - .foregroundColor(AppTheme.accent) - .frame(width: 40, height: 40) - .background(AppTheme.accentSubtle) - .clipShape(Circle()) - - Text(peer.displayName) - .font(.body) - .fontWeight(.medium) - .foregroundColor(.primary) - - Spacer() - - Text(NSLocalizedString("Connect", comment: "")) - .font(.subheadline) - .fontWeight(.medium) - .foregroundColor(AppTheme.accent) - } - .padding(14) - .background(.ultraThinMaterial) - .clipShape(RoundedRectangle(cornerRadius: 14)) - .overlay( - RoundedRectangle(cornerRadius: 14) - .strokeBorder(AppTheme.glassBorder, lineWidth: 0.5) - ) + peerRow(peer) } .buttonStyle(GlassPressStyle()) } @@ -85,8 +98,124 @@ struct DeviceScannerView: View { .padding(.top, 8) } .padding(.horizontal, 20) - .padding(.bottom, 40) + .padding(.bottom, isMulticamScanner ? 100 : 40) // room for Connect (N) + } + } + + /// One discovered-camera row. In multicam it carries a leading selection + /// circle (Apple edit-mode idiom); otherwise it keeps the classic + /// icon + "Connect" affordance. + private func peerRow(_ peer: MCPeerID) -> some View { + HStack(spacing: 14) { + if isMulticamScanner { + multicamSelectionCircle(peer) + } else { + Image(systemName: "iphone.radiowaves.left.and.right") + .font(.title3) + .foregroundColor(AppTheme.accent) + .frame(width: 40, height: 40) + .background(AppTheme.accentSubtle) + .clipShape(Circle()) + } + + Text(peer.displayName) + .font(.body) + .fontWeight(.medium) + .foregroundColor(.primary) + + Spacer() + + peerRowTrailing(peer) + } + .padding(14) + .background(.ultraThinMaterial) + .clipShape(RoundedRectangle(cornerRadius: 14)) + .overlay( + RoundedRectangle(cornerRadius: 14) + .strokeBorder(rowBorderColor(peer), lineWidth: rowBorderWidth(peer)) + ) + } + + /// The edit-mode leading circle. Selecting: empty ↔ filled check. Connecting + /// phase adds spinner (in flight), filled check (connected), warning (failed). + @ViewBuilder + private func multicamSelectionCircle(_ peer: MCPeerID) -> some View { + switch viewModel.multicamRowState(peer) { + case .selected, .connected: + Image(systemName: "checkmark.circle.fill") + .font(.title2) + .foregroundColor(AppTheme.accent) + .frame(width: 40, height: 40) + case .connecting: + ProgressView() + .frame(width: 40, height: 40) + case .failed: + Image(systemName: "exclamationmark.circle.fill") + .font(.title2) + .foregroundColor(.orange) + .frame(width: 40, height: 40) + case .unselected: + Image(systemName: "circle") + .font(.title2) + .foregroundColor(.secondary) + .frame(width: 40, height: 40) + } + } + + @ViewBuilder + private func peerRowTrailing(_ peer: MCPeerID) -> some View { + if isMulticamScanner { + // While selecting, an over-cap unselected row shows a lock; tapping + // it opens the paywall (handled by the host). + if viewModel.multicamRowLocked(peer, maxCameras: StoreManager.shared.maxCameras()) { + Image(systemName: "lock.fill") + .font(.subheadline) + .foregroundColor(.secondary) + } + } else { + Text(NSLocalizedString("Connect", comment: "")) + .font(.subheadline) + .fontWeight(.medium) + .foregroundColor(AppTheme.accent) + } + } + + private func rowBorderColor(_ peer: MCPeerID) -> Color { + guard isMulticamScanner else { return AppTheme.glassBorder } + switch viewModel.multicamRowState(peer) { + case .selected, .connected, .connecting: return AppTheme.accent + default: return AppTheme.glassBorder + } + } + + private func rowBorderWidth(_ peer: MCPeerID) -> CGFloat { + guard isMulticamScanner else { return 0.5 } + switch viewModel.multicamRowState(peer) { + case .selected, .connected, .connecting: return 2 + default: return 0.5 + } + } + + /// "Select All" — picks every discovered, not-yet-selected camera up to the + /// cap (pure). Offered only while selecting and something is unselected. + @ViewBuilder + private var selectAllRow: some View { + Button { + onSelectAll?() + } label: { + HStack(spacing: 10) { + Image(systemName: "checklist") + .font(.title3) + Text(NSLocalizedString("Select All", comment: "select every discovered camera")) + .fontWeight(.semibold) + Spacer() + } + .foregroundColor(AppTheme.accent) + .padding(14) + .background(AppTheme.accentSubtle) + .clipShape(RoundedRectangle(cornerRadius: 14)) } + .buttonStyle(GlassPressStyle()) } // MARK: - Camera Waiting State diff --git a/RemoteCam/DeviceScannerViewController.swift b/RemoteCam/DeviceScannerViewController.swift index 720e08bc..ed69fdcd 100644 --- a/RemoteCam/DeviceScannerViewController.swift +++ b/RemoteCam/DeviceScannerViewController.swift @@ -109,6 +109,11 @@ public class DeviceScannerViewController: UIViewController { remoteCamSession.setFrameSender(frameSender) self.remoteCamSession ! SetScannerLobby(lobby: self) scannerViewModel.role = role + // Multicam director collecting: only the monitor role, only behind the + // flag. Off, the coordinator's scanning path is byte-identical. + if FeatureFlags.ENABLE_MULTICAM && role == .monitor { + remoteCamSession ! UICmd.SetMulticamCollecting(on: true) + } // The reconnect overlay's only action, routed like every other UI // command; the overlay itself is pure state (PeerLinkStatus). PeerLinkStatus.shared.onCancel = { [weak self] in @@ -174,8 +179,12 @@ public class DeviceScannerViewController: UIViewController { }, onSelectPeer: { [weak self] peer in guard let self = self else { return } - self.remoteCamSession ! ConnectToDevice(peer: peer, sender: nil) - self.scannerViewModel.connectingToPeer() + if FeatureFlags.ENABLE_MULTICAM && self.role == .monitor { + self.handleMulticamRowTap(peer) + } else { + self.remoteCamSession ! ConnectToDevice(peer: peer, sender: nil) + self.scannerViewModel.connectingToPeer() + } }, onCancelConnect: { [weak self] in self?.remoteCamSession ! UICmd.CancelConnect(sender: nil) @@ -188,7 +197,13 @@ public class DeviceScannerViewController: UIViewController { }, onHelp: { [weak self] in self?.showHelpModal() - } + }, + onSelectAll: (FeatureFlags.ENABLE_MULTICAM && role == .monitor) + ? { [weak self] in self?.handleSelectAll() } + : nil, + onConnectSelected: (FeatureFlags.ENABLE_MULTICAM && role == .monitor) + ? { [weak self] in self?.handleConnectSelected() } + : nil ) swiftUIHostingController = embedSwiftUIView(scannerView) @@ -331,6 +346,75 @@ public class DeviceScannerViewController: UIViewController { } } + /// Multicam "Start (N)": one camera runs the classic 1:1 monitor + /// (unchanged); two or more hands the live transport to a + /// `MulticamController` and pushes the director screen. + /// A tap on a discovered-camera row while SELECTING. Pure — it toggles the + /// checkmark and fires zero network. An over-cap unselected row is locked + /// and routes to the paywall instead. + private func handleMulticamRowTap(_ peer: MCPeerID) { + let vm = scannerViewModel + if vm.multicamRowLocked(peer, maxCameras: StoreManager.shared.maxCameras()) { + presentMulticamPaywall() + return + } + vm.toggleMulticamSelection(peer) + } + + /// "Select All": pick every discovered camera up to the cap. Pure. + private func handleSelectAll() { + scannerViewModel.selectAllMulticam(maxCameras: StoreManager.shared.maxCameras()) + } + + /// "Connect (N)": leave selecting, fire an invite per selected camera, and + /// hand off once every invite has settled. + private func handleConnectSelected() { + let peers = scannerViewModel.beginMulticamConnecting() + guard !peers.isEmpty else { return } + for peer in peers { + remoteCamSession ! ConnectToDevice(peer: peer, sender: nil) + } + } + + /// Every selected camera has connected or failed. Hand off to the right + /// screen, or fall back to the scanner if none connected. + private func finishMulticamConnectIfSettled() { + let vm = scannerViewModel + guard vm.multicamConnectSettled else { return } + let connected = vm.multicamConnectedPeers + switch MulticamHandoff.decide( + connected: connectedPeersInSelectionOrder(connected)) { + case .none: + vm.resetMulticamToSelecting() + presentScanningError() + case .classicMonitor: + Task { @MainActor in + if await remoteCamSession.promoteSingleCollectedToConnected() { goToRole() } + } + case .director: + Task { @MainActor in + guard let handoff = await remoteCamSession.detachTransportForMulticam() else { return } + let controller = MulticamController() + await controller.install(transport: handoff.transport, + initialPeers: handoff.peers, mode: .photo) + let directorVC = MulticamViewController(controller: controller) + navigationController?.pushViewController(directorVC, animated: true) + } + } + } + + /// The connected cameras in discovered-list order (stable handoff order). + private func connectedPeersInSelectionOrder(_ connected: Set) -> [MCPeerID] { + scannerViewModel.connectedPeers.filter { connected.contains($0) } + } + + /// The shared Settings/paywall sheet — the same one the monitor gates use. + private func presentMulticamPaywall() { + let ctrl = UIHostingController(rootView: SettingsView()) + ctrl.modalPresentationStyle = .pageSheet + present(ctrl, animated: true) + } + func goToAppSettings() { #if targetEnvironment(macCatalyst) // Local-network permission lives in System Settings on the Mac. @@ -377,6 +461,18 @@ extension DeviceScannerViewController: ScannerLobby { navigationController?.popToViewController(self, animated: true) } + /// Multicam collecting: reconcile the per-row selection + the "Start (N)" + /// count from the coordinator's effective set. + func didCollectMulticamCameras(_ peers: [MCPeerID]) { + scannerViewModel.reconcileMulticamConnected(peers) + finishMulticamConnectIfSettled() + } + + func didFailMulticamCamera(_ peer: MCPeerID) { + scannerViewModel.markMulticamFailed(peer) + finishMulticamConnectIfSettled() + } + func presentScanningError() { let alert = UIAlertController( title: NSLocalizedString("Scanning Error", comment: ""), diff --git a/RemoteCam/DeviceScannerViewModel.swift b/RemoteCam/DeviceScannerViewModel.swift index 687bfa0c..d9bd42d4 100644 --- a/RemoteCam/DeviceScannerViewModel.swift +++ b/RemoteCam/DeviceScannerViewModel.swift @@ -4,6 +4,23 @@ import Stormo import Network import dnssd +/// Where a settled multicam connect hands off to, decided purely by how many +/// cameras actually connected: none → stay (error), one → the classic 1:1 +/// monitor (unchanged), two or more → the director screen. +enum MulticamHandoff: Equatable { + case none + case classicMonitor(MCPeerID) + case director([MCPeerID]) + + static func decide(connected: [MCPeerID]) -> MulticamHandoff { + switch connected.count { + case 0: return .none + case 1: return .classicMonitor(connected[0]) + default: return .director(connected) + } + } +} + /// Classifies the pre-scan local-network probe: one browse whose only job is /// to learn whether the user denied Local Network permission, so the app can /// route them to Settings instead of scanning into silence. @@ -53,6 +70,113 @@ final class DeviceScannerViewModel: ObservableObject { @Published var hasScanningError: Bool = false @Published var isConnecting: Bool = false @Published var hasConnectionError: Bool = false + // MARK: - Multicam director: select-then-connect + + /// The multicam scanner is a two-phase edit-mode selector: pick cameras + /// (pure, zero network), THEN connect the chosen set. Selecting must never + /// invite — that was the device-test regression this model fixes. + enum MulticamScanPhase { case selecting, connecting } + @Published var multicamPhase: MulticamScanPhase = .selecting + + /// Cameras the user has picked (client-side only, no transport activity). + @Published var multicamSelectedPeers: Set = [] + /// Selected cameras whose invite is in flight (connecting phase only). + @Published var multicamConnectingPeers: Set = [] + /// Cameras whose invite has established. + @Published var multicamConnectedPeers: Set = [] + /// Cameras whose invite failed (timed out after the retry). + @Published var multicamFailedPeers: Set = [] + + /// Pure selection toggle — no invite, no coordinator message. + func toggleMulticamSelection(_ peer: MCPeerID) { + guard multicamPhase == .selecting else { return } + if multicamSelectedPeers.contains(peer) { + multicamSelectedPeers.remove(peer) + } else { + multicamSelectedPeers.insert(peer) + } + } + + /// Select every discovered, not-yet-selected camera up to the cap (pure). + func selectAllMulticam(maxCameras: Int) { + guard multicamPhase == .selecting else { return } + for peer in connectedPeers where !multicamSelectedPeers.contains(peer) { + guard multicamSelectedPeers.count < maxCameras else { break } + multicamSelectedPeers.insert(peer) + } + } + + /// The "Connect (N)" CTA: enabled only while selecting with a non-empty set. + var canConnectMulticam: Bool { + multicamPhase == .selecting && !multicamSelectedPeers.isEmpty + } + var multicamSelectionCount: Int { multicamSelectedPeers.count } + + /// Enter the connecting phase; returns the peers to invite (the caller + /// fires the invites — the view model stays network-free). + func beginMulticamConnecting() -> [MCPeerID] { + guard multicamPhase == .selecting, !multicamSelectedPeers.isEmpty else { return [] } + multicamPhase = .connecting + multicamConnectingPeers = multicamSelectedPeers + multicamConnectedPeers = [] + multicamFailedPeers = [] + return connectedPeers.filter { multicamSelectedPeers.contains($0) } + } + + /// Reconcile the set of established cameras (reported by the coordinator). + func reconcileMulticamConnected(_ peers: [MCPeerID]) { + guard multicamPhase == .connecting else { return } + let connected = Set(peers).intersection(multicamSelectedPeers) + multicamConnectedPeers = connected + multicamConnectingPeers.subtract(connected) + } + + /// One camera's invite failed for good. + func markMulticamFailed(_ peer: MCPeerID) { + guard multicamPhase == .connecting else { return } + multicamConnectingPeers.remove(peer) + if !multicamConnectedPeers.contains(peer) { multicamFailedPeers.insert(peer) } + } + + /// The connect phase is over once no invite is still outstanding. + var multicamConnectSettled: Bool { + multicamPhase == .connecting && multicamConnectingPeers.isEmpty + } + + /// Back to a clean selecting phase (e.g. after all invites failed). + func resetMulticamToSelecting() { + multicamPhase = .selecting + multicamSelectedPeers = [] + multicamConnectingPeers = [] + multicamConnectedPeers = [] + multicamFailedPeers = [] + } + + /// Row state for the edit-mode circle. Selecting shows only empty/filled; + /// spinner/check/x appear only during the connect phase. + enum MulticamRowState { case unselected, selected, connecting, connected, failed } + func multicamRowState(_ peer: MCPeerID) -> MulticamRowState { + if multicamFailedPeers.contains(peer) { return .failed } + if multicamConnectedPeers.contains(peer) { return .connected } + if multicamConnectingPeers.contains(peer) { return .connecting } + if multicamSelectedPeers.contains(peer) { return .selected } + return .unselected + } + + /// An unselected row is locked (→ paywall) when the cap is already met + /// while selecting. + func multicamRowLocked(_ peer: MCPeerID, maxCameras: Int) -> Bool { + multicamPhase == .selecting + && multicamRowState(peer) == .unselected + && multicamSelectedPeers.count >= maxCameras + } + + /// Whether "Select All" should be offered — some discovered camera is still + /// unselected, and we are still selecting. + var showsMulticamSelectAll: Bool { + multicamPhase == .selecting + && connectedPeers.contains { !multicamSelectedPeers.contains($0) } + } /// When the current scan began. Not @Published: the view samples it on a /// TimelineView clock, so publishing would only cause redundant redraws. diff --git a/RemoteCam/FeatureFlags.swift b/RemoteCam/FeatureFlags.swift index b9541695..de704651 100644 --- a/RemoteCam/FeatureFlags.swift +++ b/RemoteCam/FeatureFlags.swift @@ -31,6 +31,12 @@ struct FeatureFlags { /// one-time buy, and the entitlement code stays in place for when it flips on. static let ENABLE_PRO_SUBSCRIPTION = false + /// Multicam director mode: one monitor controlling several cameras with + /// synced capture. Off until the feature ships (target 9.1.0); while off, + /// cameras advertise `supports_multicam=false` and the scanner keeps its + /// single-camera flow. + static let ENABLE_MULTICAM = false + /// Show the local camera-device picker on the camera screen. On for Mac /// Catalyst only (a Mac has N cameras — built-in, Continuity, USB); /// iPhone keeps its flip button. diff --git a/RemoteCam/FlatBufferSchemas.fbs b/RemoteCam/FlatBufferSchemas.fbs index 303bf1ea..cc350bb6 100644 --- a/RemoteCam/FlatBufferSchemas.fbs +++ b/RemoteCam/FlatBufferSchemas.fbs @@ -38,8 +38,28 @@ enum CommandAction : byte { RequestKeyframe = 21, // monitor -> camera: force a VP9 keyframe FocusAtPoint = 22, // monitor -> camera: focus/exposure point EndSession = 23, // either side: "I am leaving on purpose" - SetCameraPreviewMode = 24 // monitor -> camera: local preview on/standby; + SetCameraPreviewMode = 24, // monitor -> camera: local preview on/standby; // only sent to peers advertising supports_preview_mode + ClockSyncPing = 25, // director -> camera: clock-offset probe carrying the + // director's send time; the camera answers with a + // ClockSyncPing response echoing it plus its own clock. + // Only sent to peers advertising supports_multicam. + ScheduledCapture = 26, // director -> camera: take a photo at a wall-clock + // instant expressed in the camera's own clock domain + // (the director already applied the per-camera offset), + // so N cameras fire together. Only sent to peers + // advertising supports_multicam; the camera acks with a + // CameraStateResponse echoing the capture id. + ScheduledStartRecording = 27, // director -> camera: start recording at the fire + // instant. Reuses the ScheduledCapture params; acks by + // echoing the capture id. Only sent to multicam peers. + ScheduledStopRecording = 28, // director -> camera: stop recording at the fire + // instant, so clip lengths match across the rig. + SetStreamProfile = 29, // director -> camera: reconfigure the live preview + // encoder (resolution/bitrate/fps) for tiered + // multicam previews. Only sent to multicam peers. + RequestVideoResend = 30 // director -> camera: re-send the last collected + // clip (auto-collect retry). Reuses capture_id. } // Whether the camera device drives its own on-screen live preview. On is the @@ -158,6 +178,22 @@ table CommandParameters { focus_point_x: float; // payload for FocusAtPoint (normalized 0..1, focus_point_y: float; // upright-display space; origin top-left) camera_preview_mode: CameraPreviewModeEnum; // payload for SetCameraPreviewMode + clock_sync_t0_ms: uint64; // payload for ClockSyncPing: director clock at send + // ScheduledCapture payload. `fire_at_camera_clock_ms` is the shutter instant + // in the CAMERA's SyncClock domain (director already added the offset), used + // to schedule. `anchor_ms` is the same instant in the DIRECTOR's clock — the + // value is identical across every camera in the shot, so it is the alignment + // key stamped into each clip. `capture_id`/`capture_session_id` group the + // shot; `capture_camera_index` is this camera's 1-based slot for filenames. + capture_fire_at_camera_clock_ms: uint64; + capture_anchor_ms: uint64; + capture_id: string; + capture_session_id: string; + capture_camera_index: int; + // SetStreamProfile payload: live preview-encoder reconfiguration. + stream_max_long_edge: int; + stream_bitrate_kbps: int; + stream_fps: int; } // MARK: - Command Structure @@ -257,6 +293,11 @@ table CameraCapabilities { // not send SetCameraPreviewMode to such a peer (old decoders read the // unknown action as its enum default). supports_preview_mode: bool; + // False/absent = peer cannot join a multicam director session; a director + // must not send scheduled-capture or stream-profile commands to such a + // peer (they would be decoded as Unknown and dropped, silently desyncing + // the rig). + supports_multicam: bool; } // MARK: - Response Structure @@ -272,6 +313,10 @@ table CameraStateResponse { available_lenses: [CameraLensType]; zoom_range: ZoomRange; current_zoom: double; + // Appended fields only below this line (FlatBuffers schema evolution). + clock_sync_echo_t0_ms: uint64; // ClockSyncPing response: echoed director t0 + clock_sync_camera_clock_ms: uint64; // ClockSyncPing response: camera clock at receipt + capture_id_echo: string; // ScheduledCapture ack: the accepted capture id } // MARK: - Frame Data diff --git a/RemoteCam/FlatBufferSchemas_generated.swift b/RemoteCam/FlatBufferSchemas_generated.swift index 951f0efd..86270daf 100644 --- a/RemoteCam/FlatBufferSchemas_generated.swift +++ b/RemoteCam/FlatBufferSchemas_generated.swift @@ -33,8 +33,14 @@ public enum RemoteShutter_CommandAction: Int8, Enum, Verifiable { case focusatpoint = 22 case endsession = 23 case setcamerapreviewmode = 24 - - public static var max: RemoteShutter_CommandAction { return .setcamerapreviewmode } + case clocksyncping = 25 + case scheduledcapture = 26 + case scheduledstartrecording = 27 + case scheduledstoprecording = 28 + case setstreamprofile = 29 + case requestvideoresend = 30 + + public static var max: RemoteShutter_CommandAction { return .requestvideoresend } public static var min: RemoteShutter_CommandAction { return .unknown } } @@ -331,6 +337,15 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { case focusPointX = 36 case focusPointY = 38 case cameraPreviewMode = 40 + case clockSyncT0Ms = 42 + case captureFireAtCameraClockMs = 44 + case captureAnchorMs = 46 + case captureId = 48 + case captureSessionId = 50 + case captureCameraIndex = 52 + case streamMaxLongEdge = 54 + case streamBitrateKbps = 56 + case streamFps = 58 var v: Int32 { Int32(self.rawValue) } var p: VOffset { self.rawValue } } @@ -357,7 +372,18 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { public var focusPointX: Float32 { let o = _accessor.offset(VTOFFSET.focusPointX.v); return o == 0 ? 0.0 : _accessor.readBuffer(of: Float32.self, at: o) } public var focusPointY: Float32 { let o = _accessor.offset(VTOFFSET.focusPointY.v); return o == 0 ? 0.0 : _accessor.readBuffer(of: Float32.self, at: o) } public var cameraPreviewMode: RemoteShutter_CameraPreviewModeEnum { let o = _accessor.offset(VTOFFSET.cameraPreviewMode.v); return o == 0 ? .unknown : RemoteShutter_CameraPreviewModeEnum(rawValue: _accessor.readBuffer(of: Int8.self, at: o)) ?? .unknown } - public static func startCommandParameters(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 19) } + public var clockSyncT0Ms: UInt64 { let o = _accessor.offset(VTOFFSET.clockSyncT0Ms.v); return o == 0 ? 0 : _accessor.readBuffer(of: UInt64.self, at: o) } + public var captureFireAtCameraClockMs: UInt64 { let o = _accessor.offset(VTOFFSET.captureFireAtCameraClockMs.v); return o == 0 ? 0 : _accessor.readBuffer(of: UInt64.self, at: o) } + public var captureAnchorMs: UInt64 { let o = _accessor.offset(VTOFFSET.captureAnchorMs.v); return o == 0 ? 0 : _accessor.readBuffer(of: UInt64.self, at: o) } + public var captureId: String? { let o = _accessor.offset(VTOFFSET.captureId.v); return o == 0 ? nil : _accessor.string(at: o) } + public var captureIdSegmentArray: [UInt8]? { return _accessor.getVector(at: VTOFFSET.captureId.v) } + public var captureSessionId: String? { let o = _accessor.offset(VTOFFSET.captureSessionId.v); return o == 0 ? nil : _accessor.string(at: o) } + public var captureSessionIdSegmentArray: [UInt8]? { return _accessor.getVector(at: VTOFFSET.captureSessionId.v) } + public var captureCameraIndex: Int32 { let o = _accessor.offset(VTOFFSET.captureCameraIndex.v); return o == 0 ? 0 : _accessor.readBuffer(of: Int32.self, at: o) } + public var streamMaxLongEdge: Int32 { let o = _accessor.offset(VTOFFSET.streamMaxLongEdge.v); return o == 0 ? 0 : _accessor.readBuffer(of: Int32.self, at: o) } + public var streamBitrateKbps: Int32 { let o = _accessor.offset(VTOFFSET.streamBitrateKbps.v); return o == 0 ? 0 : _accessor.readBuffer(of: Int32.self, at: o) } + public var streamFps: Int32 { let o = _accessor.offset(VTOFFSET.streamFps.v); return o == 0 ? 0 : _accessor.readBuffer(of: Int32.self, at: o) } + public static func startCommandParameters(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 28) } public static func add(sendToRemote: Bool, _ fbb: inout FlatBufferBuilder) { fbb.add(element: sendToRemote, def: false, at: VTOFFSET.sendToRemote.p) } public static func add(zoomFactor: Double, _ fbb: inout FlatBufferBuilder) { fbb.add(element: zoomFactor, def: 0.0, at: VTOFFSET.zoomFactor.p) } @@ -378,6 +404,15 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { public static func add(focusPointX: Float32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: focusPointX, def: 0.0, at: VTOFFSET.focusPointX.p) } public static func add(focusPointY: Float32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: focusPointY, def: 0.0, at: VTOFFSET.focusPointY.p) } public static func add(cameraPreviewMode: RemoteShutter_CameraPreviewModeEnum, _ fbb: inout FlatBufferBuilder) { fbb.add(element: cameraPreviewMode.rawValue, def: 0, at: VTOFFSET.cameraPreviewMode.p) } + public static func add(clockSyncT0Ms: UInt64, _ fbb: inout FlatBufferBuilder) { fbb.add(element: clockSyncT0Ms, def: 0, at: VTOFFSET.clockSyncT0Ms.p) } + public static func add(captureFireAtCameraClockMs: UInt64, _ fbb: inout FlatBufferBuilder) { fbb.add(element: captureFireAtCameraClockMs, def: 0, at: VTOFFSET.captureFireAtCameraClockMs.p) } + public static func add(captureAnchorMs: UInt64, _ fbb: inout FlatBufferBuilder) { fbb.add(element: captureAnchorMs, def: 0, at: VTOFFSET.captureAnchorMs.p) } + public static func add(captureId: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: captureId, at: VTOFFSET.captureId.p) } + public static func add(captureSessionId: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: captureSessionId, at: VTOFFSET.captureSessionId.p) } + public static func add(captureCameraIndex: Int32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: captureCameraIndex, def: 0, at: VTOFFSET.captureCameraIndex.p) } + public static func add(streamMaxLongEdge: Int32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: streamMaxLongEdge, def: 0, at: VTOFFSET.streamMaxLongEdge.p) } + public static func add(streamBitrateKbps: Int32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: streamBitrateKbps, def: 0, at: VTOFFSET.streamBitrateKbps.p) } + public static func add(streamFps: Int32, _ fbb: inout FlatBufferBuilder) { fbb.add(element: streamFps, def: 0, at: VTOFFSET.streamFps.p) } public static func endCommandParameters(_ fbb: inout FlatBufferBuilder, start: UOffset) -> Offset { let end = Offset(offset: fbb.endTable(at: start)); return end } public static func createCommandParameters( _ fbb: inout FlatBufferBuilder, @@ -399,7 +434,16 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { deviceUniqueIdOffset deviceUniqueId: Offset = Offset(), focusPointX: Float32 = 0.0, focusPointY: Float32 = 0.0, - cameraPreviewMode: RemoteShutter_CameraPreviewModeEnum = .unknown + cameraPreviewMode: RemoteShutter_CameraPreviewModeEnum = .unknown, + clockSyncT0Ms: UInt64 = 0, + captureFireAtCameraClockMs: UInt64 = 0, + captureAnchorMs: UInt64 = 0, + captureIdOffset captureId: Offset = Offset(), + captureSessionIdOffset captureSessionId: Offset = Offset(), + captureCameraIndex: Int32 = 0, + streamMaxLongEdge: Int32 = 0, + streamBitrateKbps: Int32 = 0, + streamFps: Int32 = 0 ) -> Offset { let __start = RemoteShutter_CommandParameters.startCommandParameters(&fbb) RemoteShutter_CommandParameters.add(sendToRemote: sendToRemote, &fbb) @@ -421,6 +465,15 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { RemoteShutter_CommandParameters.add(focusPointX: focusPointX, &fbb) RemoteShutter_CommandParameters.add(focusPointY: focusPointY, &fbb) RemoteShutter_CommandParameters.add(cameraPreviewMode: cameraPreviewMode, &fbb) + RemoteShutter_CommandParameters.add(clockSyncT0Ms: clockSyncT0Ms, &fbb) + RemoteShutter_CommandParameters.add(captureFireAtCameraClockMs: captureFireAtCameraClockMs, &fbb) + RemoteShutter_CommandParameters.add(captureAnchorMs: captureAnchorMs, &fbb) + RemoteShutter_CommandParameters.add(captureId: captureId, &fbb) + RemoteShutter_CommandParameters.add(captureSessionId: captureSessionId, &fbb) + RemoteShutter_CommandParameters.add(captureCameraIndex: captureCameraIndex, &fbb) + RemoteShutter_CommandParameters.add(streamMaxLongEdge: streamMaxLongEdge, &fbb) + RemoteShutter_CommandParameters.add(streamBitrateKbps: streamBitrateKbps, &fbb) + RemoteShutter_CommandParameters.add(streamFps: streamFps, &fbb) return RemoteShutter_CommandParameters.endCommandParameters(&fbb, start: __start) } @@ -445,6 +498,15 @@ public struct RemoteShutter_CommandParameters: FlatBufferObject, Verifiable { try _v.visit(field: VTOFFSET.focusPointX.p, fieldName: "focusPointX", required: false, type: Float32.self) try _v.visit(field: VTOFFSET.focusPointY.p, fieldName: "focusPointY", required: false, type: Float32.self) try _v.visit(field: VTOFFSET.cameraPreviewMode.p, fieldName: "cameraPreviewMode", required: false, type: RemoteShutter_CameraPreviewModeEnum.self) + try _v.visit(field: VTOFFSET.clockSyncT0Ms.p, fieldName: "clockSyncT0Ms", required: false, type: UInt64.self) + try _v.visit(field: VTOFFSET.captureFireAtCameraClockMs.p, fieldName: "captureFireAtCameraClockMs", required: false, type: UInt64.self) + try _v.visit(field: VTOFFSET.captureAnchorMs.p, fieldName: "captureAnchorMs", required: false, type: UInt64.self) + try _v.visit(field: VTOFFSET.captureId.p, fieldName: "captureId", required: false, type: ForwardOffset.self) + try _v.visit(field: VTOFFSET.captureSessionId.p, fieldName: "captureSessionId", required: false, type: ForwardOffset.self) + try _v.visit(field: VTOFFSET.captureCameraIndex.p, fieldName: "captureCameraIndex", required: false, type: Int32.self) + try _v.visit(field: VTOFFSET.streamMaxLongEdge.p, fieldName: "streamMaxLongEdge", required: false, type: Int32.self) + try _v.visit(field: VTOFFSET.streamBitrateKbps.p, fieldName: "streamBitrateKbps", required: false, type: Int32.self) + try _v.visit(field: VTOFFSET.streamFps.p, fieldName: "streamFps", required: false, type: Int32.self) _v.finish() } } @@ -1011,6 +1073,7 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { case activeDeviceId = 10 case supportsFocusPoint = 12 case supportsPreviewMode = 14 + case supportsMulticam = 16 var v: Int32 { Int32(self.rawValue) } var p: VOffset { self.rawValue } } @@ -1024,7 +1087,8 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { public var activeDeviceIdSegmentArray: [UInt8]? { return _accessor.getVector(at: VTOFFSET.activeDeviceId.v) } public var supportsFocusPoint: Bool { let o = _accessor.offset(VTOFFSET.supportsFocusPoint.v); return o == 0 ? false : _accessor.readBuffer(of: Bool.self, at: o) } public var supportsPreviewMode: Bool { let o = _accessor.offset(VTOFFSET.supportsPreviewMode.v); return o == 0 ? false : _accessor.readBuffer(of: Bool.self, at: o) } - public static func startCameraCapabilities(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 6) } + public var supportsMulticam: Bool { let o = _accessor.offset(VTOFFSET.supportsMulticam.v); return o == 0 ? false : _accessor.readBuffer(of: Bool.self, at: o) } + public static func startCameraCapabilities(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 7) } public static func add(frontCamera: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: frontCamera, at: VTOFFSET.frontCamera.p) } public static func add(backCamera: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: backCamera, at: VTOFFSET.backCamera.p) } public static func addVectorOf(cameraDevices: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: cameraDevices, at: VTOFFSET.cameraDevices.p) } @@ -1033,6 +1097,8 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { at: VTOFFSET.supportsFocusPoint.p) } public static func add(supportsPreviewMode: Bool, _ fbb: inout FlatBufferBuilder) { fbb.add(element: supportsPreviewMode, def: false, at: VTOFFSET.supportsPreviewMode.p) } + public static func add(supportsMulticam: Bool, _ fbb: inout FlatBufferBuilder) { fbb.add(element: supportsMulticam, def: false, + at: VTOFFSET.supportsMulticam.p) } public static func endCameraCapabilities(_ fbb: inout FlatBufferBuilder, start: UOffset) -> Offset { let end = Offset(offset: fbb.endTable(at: start)); return end } public static func createCameraCapabilities( _ fbb: inout FlatBufferBuilder, @@ -1041,7 +1107,8 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { cameraDevicesVectorOffset cameraDevices: Offset = Offset(), activeDeviceIdOffset activeDeviceId: Offset = Offset(), supportsFocusPoint: Bool = false, - supportsPreviewMode: Bool = false + supportsPreviewMode: Bool = false, + supportsMulticam: Bool = false ) -> Offset { let __start = RemoteShutter_CameraCapabilities.startCameraCapabilities(&fbb) RemoteShutter_CameraCapabilities.add(frontCamera: frontCamera, &fbb) @@ -1050,6 +1117,7 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { RemoteShutter_CameraCapabilities.add(activeDeviceId: activeDeviceId, &fbb) RemoteShutter_CameraCapabilities.add(supportsFocusPoint: supportsFocusPoint, &fbb) RemoteShutter_CameraCapabilities.add(supportsPreviewMode: supportsPreviewMode, &fbb) + RemoteShutter_CameraCapabilities.add(supportsMulticam: supportsMulticam, &fbb) return RemoteShutter_CameraCapabilities.endCameraCapabilities(&fbb, start: __start) } @@ -1061,6 +1129,7 @@ public struct RemoteShutter_CameraCapabilities: FlatBufferObject, Verifiable { try _v.visit(field: VTOFFSET.activeDeviceId.p, fieldName: "activeDeviceId", required: false, type: ForwardOffset.self) try _v.visit(field: VTOFFSET.supportsFocusPoint.p, fieldName: "supportsFocusPoint", required: false, type: Bool.self) try _v.visit(field: VTOFFSET.supportsPreviewMode.p, fieldName: "supportsPreviewMode", required: false, type: Bool.self) + try _v.visit(field: VTOFFSET.supportsMulticam.p, fieldName: "supportsMulticam", required: false, type: Bool.self) _v.finish() } } @@ -1087,6 +1156,9 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { case availableLenses = 18 case zoomRange = 20 case currentZoom = 22 + case clockSyncEchoT0Ms = 24 + case clockSyncCameraClockMs = 26 + case captureIdEcho = 28 var v: Int32 { Int32(self.rawValue) } var p: VOffset { self.rawValue } } @@ -1107,7 +1179,11 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { public func availableLenses(at index: Int32) -> RemoteShutter_CameraLensType? { let o = _accessor.offset(VTOFFSET.availableLenses.v); return o == 0 ? RemoteShutter_CameraLensType.wideangle : RemoteShutter_CameraLensType(rawValue: _accessor.directRead(of: Int8.self, offset: _accessor.vector(at: o) + index * 1)) } public var zoomRange: RemoteShutter_ZoomRange? { let o = _accessor.offset(VTOFFSET.zoomRange.v); return o == 0 ? nil : RemoteShutter_ZoomRange(_accessor.bb, o: _accessor.indirect(o + _accessor.position)) } public var currentZoom: Double { let o = _accessor.offset(VTOFFSET.currentZoom.v); return o == 0 ? 0.0 : _accessor.readBuffer(of: Double.self, at: o) } - public static func startCameraStateResponse(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 10) } + public var clockSyncEchoT0Ms: UInt64 { let o = _accessor.offset(VTOFFSET.clockSyncEchoT0Ms.v); return o == 0 ? 0 : _accessor.readBuffer(of: UInt64.self, at: o) } + public var clockSyncCameraClockMs: UInt64 { let o = _accessor.offset(VTOFFSET.clockSyncCameraClockMs.v); return o == 0 ? 0 : _accessor.readBuffer(of: UInt64.self, at: o) } + public var captureIdEcho: String? { let o = _accessor.offset(VTOFFSET.captureIdEcho.v); return o == 0 ? nil : _accessor.string(at: o) } + public var captureIdEchoSegmentArray: [UInt8]? { return _accessor.getVector(at: VTOFFSET.captureIdEcho.v) } + public static func startCameraStateResponse(_ fbb: inout FlatBufferBuilder) -> UOffset { fbb.startTable(with: 13) } public static func add(action: RemoteShutter_CommandAction, _ fbb: inout FlatBufferBuilder) { fbb.add(element: action.rawValue, def: 0, at: VTOFFSET.action.p) } public static func add(success: Bool, _ fbb: inout FlatBufferBuilder) { fbb.add(element: success, def: false, at: VTOFFSET.success.p) } @@ -1119,6 +1195,9 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { public static func addVectorOf(availableLenses: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: availableLenses, at: VTOFFSET.availableLenses.p) } public static func add(zoomRange: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: zoomRange, at: VTOFFSET.zoomRange.p) } public static func add(currentZoom: Double, _ fbb: inout FlatBufferBuilder) { fbb.add(element: currentZoom, def: 0.0, at: VTOFFSET.currentZoom.p) } + public static func add(clockSyncEchoT0Ms: UInt64, _ fbb: inout FlatBufferBuilder) { fbb.add(element: clockSyncEchoT0Ms, def: 0, at: VTOFFSET.clockSyncEchoT0Ms.p) } + public static func add(clockSyncCameraClockMs: UInt64, _ fbb: inout FlatBufferBuilder) { fbb.add(element: clockSyncCameraClockMs, def: 0, at: VTOFFSET.clockSyncCameraClockMs.p) } + public static func add(captureIdEcho: Offset, _ fbb: inout FlatBufferBuilder) { fbb.add(offset: captureIdEcho, at: VTOFFSET.captureIdEcho.p) } public static func endCameraStateResponse(_ fbb: inout FlatBufferBuilder, start: UOffset) -> Offset { let end = Offset(offset: fbb.endTable(at: start)); return end } public static func createCameraStateResponse( _ fbb: inout FlatBufferBuilder, @@ -1131,7 +1210,10 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { recordingStartTime: UInt64 = 0, availableLensesVectorOffset availableLenses: Offset = Offset(), zoomRangeOffset zoomRange: Offset = Offset(), - currentZoom: Double = 0.0 + currentZoom: Double = 0.0, + clockSyncEchoT0Ms: UInt64 = 0, + clockSyncCameraClockMs: UInt64 = 0, + captureIdEchoOffset captureIdEcho: Offset = Offset() ) -> Offset { let __start = RemoteShutter_CameraStateResponse.startCameraStateResponse(&fbb) RemoteShutter_CameraStateResponse.add(action: action, &fbb) @@ -1144,6 +1226,9 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { RemoteShutter_CameraStateResponse.addVectorOf(availableLenses: availableLenses, &fbb) RemoteShutter_CameraStateResponse.add(zoomRange: zoomRange, &fbb) RemoteShutter_CameraStateResponse.add(currentZoom: currentZoom, &fbb) + RemoteShutter_CameraStateResponse.add(clockSyncEchoT0Ms: clockSyncEchoT0Ms, &fbb) + RemoteShutter_CameraStateResponse.add(clockSyncCameraClockMs: clockSyncCameraClockMs, &fbb) + RemoteShutter_CameraStateResponse.add(captureIdEcho: captureIdEcho, &fbb) return RemoteShutter_CameraStateResponse.endCameraStateResponse(&fbb, start: __start) } @@ -1159,6 +1244,9 @@ public struct RemoteShutter_CameraStateResponse: FlatBufferObject, Verifiable { try _v.visit(field: VTOFFSET.availableLenses.p, fieldName: "availableLenses", required: false, type: ForwardOffset>.self) try _v.visit(field: VTOFFSET.zoomRange.p, fieldName: "zoomRange", required: false, type: ForwardOffset.self) try _v.visit(field: VTOFFSET.currentZoom.p, fieldName: "currentZoom", required: false, type: Double.self) + try _v.visit(field: VTOFFSET.clockSyncEchoT0Ms.p, fieldName: "clockSyncEchoT0Ms", required: false, type: UInt64.self) + try _v.visit(field: VTOFFSET.clockSyncCameraClockMs.p, fieldName: "clockSyncCameraClockMs", required: false, type: UInt64.self) + try _v.visit(field: VTOFFSET.captureIdEcho.p, fieldName: "captureIdEcho", required: false, type: ForwardOffset.self) _v.finish() } } diff --git a/RemoteCam/FrameStreamReceiver.swift b/RemoteCam/FrameStreamReceiver.swift index e3d9adce..bdd7ec07 100644 --- a/RemoteCam/FrameStreamReceiver.swift +++ b/RemoteCam/FrameStreamReceiver.swift @@ -20,7 +20,12 @@ import CoreImage import VideoToolbox import VideocallCodecs -final class FrameStreamReceiver { +/// `@unchecked Sendable`: `receive(_:)` hops straight to `decodeQueue`, and all +/// mutable decode state is confined there; the `onImage`/`onStall`/ +/// `onKeyframeNeeded` callbacks are wired once before `start()`. This lets the +/// multicam controller hold a per-lane frame sink that calls `receive` from the +/// actor without reaching into a `UIViewController`. +final class FrameStreamReceiver: @unchecked Sendable { /// Decoded frame ready for display. Called on the decode queue — hop to /// main before touching UI. diff --git a/RemoteCam/FrameStreamer.swift b/RemoteCam/FrameStreamer.swift index 000b1cfc..daee4161 100644 --- a/RemoteCam/FrameStreamer.swift +++ b/RemoteCam/FrameStreamer.swift @@ -38,6 +38,12 @@ final class FrameStreamer { private let config: StreamingConfig private let send: Send + /// The live preview profile (resolution/bitrate/fps). Starts at the full + /// peer profile — byte-identical to the 1:1 stream — and only a multicam + /// director changes it (via `applyProfile`) to tier previews. Capture-queue + /// confined, like every other field here. + private var activeProfile: StreamProfile + /// Remaining still-codec chain; first entry is the active still encoder. private var encoders: [FrameEncoding] private var frameCount = 0 @@ -55,7 +61,7 @@ final class FrameStreamer { /// Builds the stateful video encoder (HEVC preferred, VP9 fallback) on first /// use; nil when no video codec is available at runtime or in tests that /// don't exercise video. - private let makeVideoEncoder: () -> StreamVideoEncoding? + private let makeVideoEncoder: (StreamProfile) -> StreamVideoEncoding? private var videoEncoder: StreamVideoEncoding? /// Latched when the video encoder fails on this hardware: never retried, the /// stream stays on stills for the rest of the connection. @@ -74,16 +80,36 @@ final class FrameStreamer { encoders: [FrameEncoding]? = nil, creditAvailable: @escaping () -> Bool = { true }, takeKeyframeRequest: @escaping () -> Bool = { false }, - makeVideoEncoder: @escaping () -> StreamVideoEncoding? = { nil }, + makeVideoEncoder: @escaping (StreamProfile) -> StreamVideoEncoding? = { _ in nil }, send: @escaping Send) { self.config = config self.send = send - self.encoders = encoders ?? Self.makeEncoders(config: config) + // The default profile reproduces today's peer stream exactly, so a + // streamer that is never re-profiled behaves identically to before. + self.activeProfile = StreamProfile( + maxLongEdge: config.maxLongEdge, + bitrateKbps: config.peerHEVC.bitrateKbps, + fps: config.peerHEVC.fps) + self.encoders = encoders ?? Self.makeEncoders(config: config, maxLongEdge: config.maxLongEdge) self.creditAvailable = creditAvailable self.takeKeyframeRequest = takeKeyframeRequest self.makeVideoEncoder = makeVideoEncoder } + /// Multicam: switch this stream to a new preview profile. Capture-queue + /// confined (called from `FrameSender`). Rebuilds the still chain and drops + /// the video encoder so it is rebuilt at the new resolution on the next + /// frame — the same rebuild path the failure fallback already uses, so the + /// new encoder's first frame is a keyframe the monitor re-syncs on. + func applyProfile(_ profile: StreamProfile) { + guard profile != activeProfile else { return } + activeProfile = profile + encoders = Self.makeEncoders(config: config, maxLongEdge: profile.maxLongEdge) + // Only rebuild a video encoder that is actually running; a permanently + // failed one stays failed (stills). + if videoEncoder != nil { videoEncoder = nil } + } + /// The active still-codec head (used by tests). The video stream, when /// enabled, is a separate path and not reflected here. var activeCodec: RemoteCmd.StreamCodec? { encoders.first?.codec } @@ -110,7 +136,7 @@ final class FrameStreamer { /// is available and the dev-only still fallback runs instead. private var useVideo: Bool { guard !videoFailed else { return false } - if videoEncoder == nil { videoEncoder = makeVideoEncoder() } + if videoEncoder == nil { videoEncoder = makeVideoEncoder(activeProfile) } return videoEncoder != nil } @@ -170,13 +196,13 @@ final class FrameStreamer { )) } - private static func makeEncoders(config: StreamingConfig) -> [FrameEncoding] { + private static func makeEncoders(config: StreamingConfig, maxLongEdge: CGFloat) -> [FrameEncoding] { config.preferredCodecs.compactMap { codec in switch codec { case .heic: - return HEICFrameEncoder(maxLongEdge: config.maxLongEdge, quality: config.heicQuality) + return HEICFrameEncoder(maxLongEdge: maxLongEdge, quality: config.heicQuality) case .jpeg: - return JPEGFrameEncoder(maxLongEdge: config.maxLongEdge, quality: config.jpegQuality) + return JPEGFrameEncoder(maxLongEdge: maxLongEdge, quality: config.jpegQuality) case .hevc: return nil // video codec: follow-up PR (HEVCFrameEncoder) case .vp9: diff --git a/RemoteCam/FrameStreamingCoordinator.swift b/RemoteCam/FrameStreamingCoordinator.swift index 718d254a..2a94594b 100644 --- a/RemoteCam/FrameStreamingCoordinator.swift +++ b/RemoteCam/FrameStreamingCoordinator.swift @@ -52,15 +52,22 @@ final class FrameStreamingCoordinator: NSObject { private lazy var frameStreamer = FrameStreamer( creditAvailable: frameCreditAvailable, takeKeyframeRequest: takePeerKeyframeRequest, - makeVideoEncoder: { + makeVideoEncoder: { profile in // Prefer hardware HEVC; fall back to software VP9; nil only on an // arch with neither (dev-only), where the stream drops to stills. + // The multicam profile overrides resolution/bitrate/fps; the rest of + // the encoder tuning (keyframe interval, quantizers) stays put. let config = StreamingConfig.default if HEVCSupport.canEncode { - return HEVCFrameEncoder(maxLongEdge: config.maxLongEdge, settings: config.peerHEVC) + let hevc = HEVCSettings(bitrateKbps: profile.bitrateKbps, fps: profile.fps, + keyframeInterval: config.peerHEVC.keyframeInterval) + return HEVCFrameEncoder(maxLongEdge: profile.maxLongEdge, settings: hevc) } if VP9Support.isAvailable { - return VP9FrameEncoder(maxLongEdge: config.maxLongEdge, settings: config.peerVP9) + var vp9 = config.peerVP9 + vp9.bitrateKbps = profile.bitrateKbps + vp9.fps = profile.fps + return VP9FrameEncoder(maxLongEdge: profile.maxLongEdge, settings: vp9) } return nil }, @@ -111,6 +118,15 @@ final class FrameStreamingCoordinator: NSObject { watchPreviewStreamer.acknowledge() } + /// Multicam: retune the phone-monitor preview stream. Hops onto the capture + /// queue the streamer lives on, so the reconfigure is serialized with frame + /// encoding. + func applyStreamProfile(_ profile: StreamProfile) { + engine.dataOutputQueue.async { [weak self] in + self?.frameStreamer.applyProfile(profile) + } + } + /// Wall-clock of the most recent video sample buffer, for the rig's /// first-frame watchdog (a suspended or stalled camera delivers nothing, /// forever). Written per-frame on the data queue, read from the watchdog. diff --git a/RemoteCam/MultiCamChrome.swift b/RemoteCam/MultiCamChrome.swift new file mode 100644 index 00000000..c141d548 --- /dev/null +++ b/RemoteCam/MultiCamChrome.swift @@ -0,0 +1,36 @@ +// +// MultiCamChrome.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import CoreGraphics +import Foundation + +/// How the director screen lays out the cameras. +enum MulticamDisplayMode: Equatable { + /// One focused viewfinder + a thumbnail strip of the others (the default). + case focus + /// An equal grid of every camera — the "monitor wall" for checking angles. + case grid +} + +/// Pure layout policy for the multicam director screen, kept out of the views +/// so it is unit-testable without a host (mirrors `MonitorChrome`). +enum MultiCamChrome { + + /// Columns for the grid: a near-square arrangement — 2 cameras sit 2-up, + /// 3–4 form a 2×2, and it keeps growing by √n for any future larger rig. + /// One camera has no grid (the focus view fills the screen). + static func gridColumnCount(cameraCount: Int) -> Int { + guard cameraCount > 1 else { return 1 } + return max(1, Int(ceil(Double(cameraCount).squareRoot()))) + } + + /// The grid toggle is offered only when there is more than one camera — + /// with a single camera the focus view already is the whole screen. + static func showsGridToggle(cameraCount: Int) -> Bool { + cameraCount > 1 + } +} diff --git a/RemoteCam/MulticamController.swift b/RemoteCam/MulticamController.swift new file mode 100644 index 00000000..7ac62197 --- /dev/null +++ b/RemoteCam/MulticamController.swift @@ -0,0 +1,1305 @@ +// +// MulticamController.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +// swiftlint:disable cyclomatic_complexity function_body_length + +import Foundation +import MPCCompat +import Photos +import Stormo +import UIKit + +/// The director's aggregate posture across all cameras. One shutter drives +/// them all, so the capture states are aggregate, not per-camera. +enum MulticamState: Equatable { + case monitoring(mode: MonitorMode) + /// A synced photo is in flight: `acksRemaining` cameras have yet to accept + /// or refuse `captureId`. Returns to `.monitoring` when it reaches zero. + case capturingPhoto(captureId: String, acksRemaining: Int) + /// The rig is recording (or still collecting start acks). `acksRemaining` + /// counts cameras that have yet to confirm the scheduled start. + case recording(captureId: String, acksRemaining: Int) + /// A synced stop is in flight; returns to `.monitoring` when every camera + /// has confirmed (or timed out). + case stoppingRecording(captureId: String, acksRemaining: Int) +} + +/// How a camera answered the last synced capture, for the tile badge. +enum CaptureOutcome: Equatable { + case captured + case failed +} + +/// A UI-facing snapshot of one lane. Deliberately a value type carrying only +/// what the chrome/tile needs, so the actor never hands the UI a live +/// reference into its own state. +struct MulticamLaneInfo: Equatable { + let peerID: MCPeerID + let displayName: String + let status: CameraLink.Status + let isFocused: Bool + /// Clock-offset estimate in ms (nil until the first pong), for a future + /// sync-quality indicator; unused by PR3's UI beyond diagnostics. + let clockOffsetMillis: Int64? + /// How this camera answered the last synced capture, for the tile badge. + let captureOutcome: CaptureOutcome? + /// This camera is rolling as part of a synced recording (REC badge). + let isRecording: Bool + /// This camera can't match the running rig quality — tile badge + re-match. + let needsQualityRematch: Bool + /// Where this lane's footage is in the post-take auto-collect. + let collection: CameraLink.LaneCollectionState +} + +/// A Sendable pipe that carries one lane's decoded-preview frames from the +/// controller (actor domain) to exactly that lane's UI-side decoder. The view +/// controller hands one to the controller per lane at creation +/// (`setFrameSink(for:_:)`), so the ~20fps stream routes actor → closure — +/// never reaching into a `UIViewController` from the actor. +typealias MulticamFrameSink = @Sendable (RemoteCmd.OnFrame) -> Void + +/// The main-actor bridge from the controller to the multicam screen — the +/// multicam analog of `MonitorDisplay`. Low-frequency lane changes go through +/// `applyLanes`; the ~20fps preview stream goes through per-lane frame sinks +/// (see `MulticamFrameSink`), each routing to exactly one lane's decoder so a +/// frame from camera B never re-renders camera A. +protocol MulticamDisplay: AnyObject { + func applyLanes(_ lanes: [MulticamLaneInfo]) + /// Aggregate shutter state: `capturing` = a synced photo is in flight + /// (activity ring); `recording` = the rig is rolling (record/stop button). + func applyShutterState(capturing: Bool, recording: Bool) + /// Cameras the browser has found that aren't in the rig — the add-camera + /// sheet's list. + func applyAvailablePeers(_ peers: [MCPeerID]) + /// The rig-wide settings (timer + quality intersection) for the tray. + func applyRigSettings(_ settings: RigSettingsSnapshot) + func exitMulticam() +} + +/// Director side of a multicam session: one controller, several cameras. +/// +/// A sibling of `SessionCoordinator`, not a replacement — `SessionCoordinator` +/// stays the sole brain for the camera role and for 1:1 monitoring, both +/// byte-identical to before. This actor is reached only when the director +/// starts a multicam session (≥2 cameras, behind `ENABLE_MULTICAM`). The +/// camera side is unchanged: a camera cannot tell a multicam director from a +/// single monitor. +/// +/// Mirrors `SessionCoordinator`'s concurrency shape: a FIFO inbox fed by +/// `tell(_:)`, nonisolated transport-delegate callbacks that enqueue, and a +/// lock-boxed transport mirror for the sends that must not queue behind +/// state-machine work (frame acks, clock-sync answers). +public actor MulticamController { + + // MARK: Inbox (mirrors SessionCoordinator) + + private nonisolated let inboxContinuation: Locked.Continuation?> = Locked(nil) + private nonisolated let pendingCount = Locked(0) + + public init() { + var continuation: AsyncStream.Continuation! + let stream = AsyncStream(bufferingPolicy: .unbounded) { continuation = $0 } + self.inboxContinuation.value = continuation + let pending = pendingCount + Task { [weak self] in + for await msg in stream { + guard let self else { break } + await self.handle(msg) + pending.mutate { $0 -= 1 } + } + } + } + + public nonisolated func tell(_ msg: Message) { + pendingCount.mutate { $0 += 1 } + inboxContinuation.value?.yield(msg) + } + + /// Test support: suspend until every enqueued message is processed. + public nonisolated func waitForIdle() async { + while pendingCount.value > 0 { + await Task.yield() + } + } + + public nonisolated func stop() { + clockSyncTask.value?.cancel() + timerTask.value?.cancel() + transportShared.value?.stopSession() + inboxContinuation.value?.finish() + } + + // MARK: Transport + + private var multipeerService: (any MultipeerServiceProtocol)? + private let transportShared = Locked<(any MultipeerServiceProtocol)?>(nil) + private nonisolated let clockSyncTask = Locked?>(nil) + + // MARK: State + + private var state: MulticamState = .monitoring(mode: .photo) + /// Insertion-ordered peer ids, so the strip/grid order is stable as lanes + /// come and go. + private var order: [MCPeerID] = [] + private var links: [MCPeerID: CameraLink] = [:] + private var focusedPeer: MCPeerID? + + /// Cameras the browser has found that are not in the rig yet — the source + /// list for the in-session "add camera" sheet. Insertion-ordered. + private var availableOrder: [MCPeerID] = [] + private var available: Set = [] + + /// One id per director session, part of every capture's filename group. + private let sessionID = UUID().uuidString + /// Cameras that have yet to answer the in-flight synced capture. Empty + /// unless `state` is `.capturingPhoto`. + private var capturingLanes: Set = [] + private var currentCaptureID: String? + /// The most recent photo/video capture id, used to name auto-collected + /// footage on the director under the shared `RS___cam` group. + private var lastCaptureID: String? + + /// Lead time before a synced shutter fires — long enough to cover the + /// worst-case one-way latency plus a retransmit and scheduling slop, so + /// every camera has the command in hand before the instant arrives. + private let captureLeadMillis: UInt64 = 150 + /// How long a camera has to answer before it is counted as failed. + private var captureAckTimeout: TimeInterval = 3 + + // MARK: Rig-wide settings ("the shot belongs to the rig") + + /// The rig's active video quality — applied to every lane. Nil until the + /// first pick (or Automatic). Manual selection within the intersection is + /// first-class; Automatic just recomputes best-in-intersection. + private var activeVideoQuality: (resolution: VideoResolution, frameRate: VideoFrameRate)? + private var activePhotoQuality: (format: PhotoFormat, hdr: HDRMode)? + /// One rig self-timer (seconds); 0 = off. Fans out to every camera so + /// subjects see the countdown, and its expiry triggers the synced capture. + private var rigTimerSeconds: Int = 0 + /// The live countdown, or nil when not counting down. + private var countdown: (remaining: Int, action: MulticamTimedAction)? + /// The countdown tick interval — injectable so tests don't wait real seconds. + private var timerTickInterval: TimeInterval = 1 + private nonisolated let timerTask = Locked?>(nil) + + private weak var display: MulticamDisplay? + + /// One preview-frame sink per lane, keyed by peer. The view controller + /// registers one when it creates a lane; `handleFrame` routes each frame to + /// its source's sink. Cleared when the link is dropped. + private var frameSinks: [MCPeerID: MulticamFrameSink] = [:] + + /// The interval between clock-offset refreshes per camera. + private let clockSyncInterval: TimeInterval = 30 + /// How far ahead a re-invite waits before re-browsing a dropped camera. + private var reconnectRetryDelay: TimeInterval = 3 + private let reconnectInviteTimeout: TimeInterval = 10 + + // MARK: Test / wiring seams + + func needsRematchForTesting(_ peer: MCPeerID) -> Bool { links[peer]?.needsQualityRematch ?? false } + + func setDisplay(_ display: MulticamDisplay) { + self.display = display + // Pre-pump setup: replay the current state directly (no pump epilogue + // to flush a dirty flag). The display is wired after `install`, so the + // first snapshot emitted during install would otherwise reach no one. + publishRigSettingsNow() + publishLanesNow() + } + func setReconnectRetryDelay(_ delay: TimeInterval) { reconnectRetryDelay = delay } + + /// Register (or replace) the preview-frame sink for a lane. Called by the + /// view controller when it creates the lane; the sink is dropped when the + /// link goes away (`handleRemoveCamera`). + func setFrameSink(for peer: MCPeerID, _ sink: @escaping MulticamFrameSink) { + frameSinks[peer] = sink + } + + /// Test support. + func lanesForTesting() -> [MulticamLaneInfo] { laneSnapshot() } + func focusedPeerForTesting() -> MCPeerID? { focusedPeer } + func statusForTesting(_ peer: MCPeerID) -> CameraLink.Status? { links[peer]?.status } + func offsetForTesting(_ peer: MCPeerID) -> Int64? { links[peer]?.latestOffset?.offsetMillis } + + // MARK: - Handoff + + /// Take over a transport the scanner already connected to `initialPeers`. + /// Becomes the transport delegate (the scanner's `SessionCoordinator` + /// stops receiving callbacks from here on), seeds a lane per peer, kicks + /// the capability handshake + clock sync, and keeps browsing so more + /// cameras can be invited later. + func install(transport: any MultipeerServiceProtocol, + initialPeers: [MCPeerID], + mode: MonitorMode) { + multipeerService = transport + transportShared.value = transport + transport.delegate = self + state = .monitoring(mode: mode) + + for peer in initialPeers where links[peer] == nil { + order.append(peer) + links[peer] = CameraLink(peerID: peer) + } + focusedPeer = focusedPeer ?? order.first + + // Keep discovering so the in-session "add camera" flow (PR7) has a + // live peer list; a director that stopped browsing on connect could + // never grow the rig. + transport.startBrowsingOnly() + + for peer in order { beginHandshake(with: peer) } + startClockSyncLoop() + publishLanesNow() // pre-pump setup + } + + /// The initial per-camera handshake: announce the director role (carries + /// our version so the camera can gate us) and ask for capabilities. The + /// camera answers with `CameraCapabilitiesResp`, at which point the lane + /// goes live and its frame pump starts. + private func beginHandshake(with peer: MCPeerID) { + sendTo(peer, RemoteCmd.PeerBecameMonitor.createWithDefaults()) + sendTo(peer, RemoteCmd.RequestCameraCapabilities()) + // Prime the stream: the camera streams once it holds a frame credit. + sendTo(peer, RemoteCmd.RequestFrame(sender: nil)) + } + + // MARK: - Message handling + + /// Derived publishing: handlers mark what changed instead of pushing to the + /// UI themselves, and the pump flushes at most one lanes-publish and one + /// rig-publish per message — so multiple mutations coalesce into one main hop. + private var lanesDirty = false + private var rigDirty = false + private func markLanesDirty() { lanesDirty = true } + private func markRigDirty() { rigDirty = true } + + func handle(_ msg: Message) async { + await route(msg) + if lanesDirty { lanesDirty = false; publishLanesNow() } + if rigDirty { rigDirty = false; publishRigSettingsNow() } + } + + private func route(_ msg: Message) async { + switch msg { + case let connected as OnConnectToDevice: + handlePeerConnected(connected.peer) + + case let disconnected as DisconnectPeer: + if let peer = disconnected.peer { handlePeerDisconnected(peer) } + + case let found as UICmd.BrowserFoundPeer: + handleBrowserFound(found.peer) + + case let lost as UICmd.BrowserLostPeer: + if available.remove(lost.peer) != nil { + availableOrder.removeAll { $0 == lost.peer } + publishAvailable() + } + + case let routed as RoutedMessage: + await handleRouted(routed.message, from: routed.peer) + + case let frame as RemoteCmd.OnFrame: + handleFrame(frame) + + case let started as ResourceTransferStarted: + handleResourceStarted(started) + + case let finished as ResourceTransferFinished: + handleResourceFinished(finished) + + case let measured as ClockPongMeasured: + storePong(measured.pong, t3: measured.t3, from: measured.peer) + + // --- UI commands (single-entry inbox) --- + case is MCCapturePhoto: handleCapturePhoto() + case is MCStartRecording: handleStartRecording() + case is MCStopRecording: handleStopRecording() + case is MCAutomaticVideoQuality: handleAutomaticVideoQuality() + case is MCAutomaticPhotoQuality: handleAutomaticPhotoQuality() + case is MCTimerAdvance: advanceCountdown() + case let q as MCSetVideoQuality: handleSetVideoQuality(resolution: q.resolution, frameRate: q.frameRate) + case let q as MCSetPhotoQuality: handleSetPhotoQuality(format: q.format, hdr: q.hdr) + case let t as MCSetRigTimer: handleSetRigTimer(t.seconds) + case let c as MCPeerCommand: + switch c.kind { + case .focus: handleSetFocusedPeer(c.peer) + case .invite: handleInviteCamera(c.peer) + case .remove: handleRemoveCamera(c.peer) + case .retryCollection: handleRetryCollection(c.peer) + case .nudgeFrame: handleNudgeFrame(c.peer) + case .requestKeyframe: handleRequestKeyframe(c.peer) + case .reconnectTick: reBrowseIfStillMissing(c.peer) + } + + case is UICmd.AppForegrounded: + // Clocks freeze while backgrounded; the estimates are stale. Drop + // them and re-measure, mirroring the frame path's foreground rearm. + for link in links.values { link.clockEstimator.reset() } + pingClocks() + + default: + break + } + } + + /// A camera-addressed message that arrived with its source peer (Seam A). + private func handleRouted(_ message: Message, from peer: MCPeerID) async { + guard let link = links[peer] else { return } + + switch message { + case let became as RemoteCmd.PeerBecameCamera: + // Same-major-or-refuse, per camera. A refused camera is dropped + // from the rig rather than silently streaming a peer we can't + // fully drive. + if !isPeerCompatible(became) { + link.status = .failed + markLanesDirty() + } + + case let caps as RemoteCmd.CameraCapabilitiesResp: + link.capabilities = caps + if link.status != .failed { link.status = .linked } + // A late joiner may not match the running rig quality: flag it (its + // tile badges + the tray offers re-match) rather than silently + // changing the rig. Also refreshes the intersection menu. + refreshRematchFlags() + markLanesDirty() + markRigDirty() + // A multicam-capable camera gets an immediate clock probe so its + // offset is ready well before the first synced capture (PR4), and + // its preview tier (full if focused, else thumbnail). + if link.supportsMulticam { + sendTo(peer, RemoteCmd.ClockSyncPing(t0Millis: SyncClock.nowMillis())) + pushProfile(to: peer) + } + + case let ack as RemoteCmd.ScheduledCaptureAck: + resolvePhotoAck(from: peer, success: ack.error == nil) + + case is RemoteCmd.TakePicAck: + // The fallback (plain TakePic) path's positive ack. + resolvePhotoAck(from: peer, success: true) + + case let resp as RemoteCmd.TakePicResp: + if resp.error != nil { + resolvePhotoAck(from: peer, success: false) + } else if let pic = resp.pic { + // Auto-collect: the camera returned its (EXIF-stamped) still. + // Save it to the director's library under the shared RS_ name. + collectPhoto(pic, from: peer) + } + + case let ack as RemoteCmd.ScheduledRecordingAck: + resolveRecordingAck(from: peer, isStop: ack.isStop, success: ack.error == nil) + + case let ack as RemoteCmd.StartRecordingVideoAck: + // The fallback (plain StartRecordingVideo) path's positive ack. + if ack.error == nil { resolveRecordingAck(from: peer, isStop: false, success: true) } + + default: + // Per-camera command responses (zoom/lens/flash/torch acks) update + // only the focused lane's controls, wired to the UI in a later PR; + // PR3 surfaces frames + status, so these are accepted and ignored. + break + } + } + + private func handlePeerConnected(_ peer: MCPeerID) { + if let existing = links[peer] { + // A reconnecting lane came back — rehandshake and relight it. + existing.status = .linked + } else { + order.append(peer) + links[peer] = CameraLink(peerID: peer) + } + // It's in the rig now, so it's no longer an "available" candidate. + if available.remove(peer) != nil { + availableOrder.removeAll { $0 == peer } + publishAvailable() + } + focusedPeer = focusedPeer ?? peer + syncFocusFlags() + beginHandshake(with: peer) + markLanesDirty() + } + + /// Mirror `focusedPeer` onto each lane's `isFocused` so `CameraLink.snapshot` + /// stays the single source of truth for the tile. + private func syncFocusFlags() { + for (peer, link) in links { link.isFocused = (peer == focusedPeer) } + } + + private func publishAvailable() { + let peers = availableOrder + let display = display + OperationQueue.main.addOperation { display?.applyAvailablePeers(peers) } + } + + private func handlePeerDisconnected(_ peer: MCPeerID) { + guard let link = links[peer] else { return } + // Degrade the tile, keep the rest of the rig recording/monitoring. The + // controller stays browsing, so `browserDidFindPeer` re-invites. + link.status = .reconnecting + markLanesDirty() + armReconnect(peer) + } + + private func handleBrowserFound(_ peer: MCPeerID) { + // A camera we are actively missing is auto-re-invited. + if let link = links[peer], link.status == .reconnecting { + multipeerService?.invitePeer(peer, timeout: reconnectInviteTimeout) + return + } + // An unrelated fresh peer becomes a candidate for the "add camera" + // sheet — it never auto-joins. + guard links[peer] == nil, !available.contains(peer) else { return } + available.insert(peer) + availableOrder.append(peer) + publishAvailable() + } + + /// Invite a discovered camera into the rig (the "add camera" flow). The + /// tier cap is enforced by the UI before this is called. + public nonisolated func inviteCamera(_ peer: MCPeerID) { tell(MCPeerCommand(.invite, peer)) } + + private func handleInviteCamera(_ peer: MCPeerID) { + guard available.contains(peer) else { return } + multipeerService?.invitePeer(peer, timeout: reconnectInviteTimeout) + } + + /// The current number of cameras in the rig — a pure query the UI reads + /// (directly, not through the inbox) to gate the add-camera paywall. + func cameraCount() -> Int { order.count } + + /// Schedule a reconnect tick; the tick itself runs in the pump (ordered), + /// so the only thing off-actor is the one-shot sleep. + private func armReconnect(_ peer: MCPeerID) { + let delay = reconnectRetryDelay + Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000)) + self?.tell(MCPeerCommand(.reconnectTick, peer)) + } + } + + private func reBrowseIfStillMissing(_ peer: MCPeerID) { + guard links[peer]?.status == .reconnecting else { return } + // Rebuild the browse so the lost peer is re-reported at a live address. + multipeerService?.startBrowsingOnly() + } + + private func handleFrame(_ frame: RemoteCmd.OnFrame) { + guard let link = links[frame.peerId] else { return } + if frame.codec == .vp9 { link.sawVP9 = true } + // Route to exactly this lane's decoder (rendering isolation), then ack + // only this camera so its credit window advances and no other camera + // sends on a frame it didn't produce (Seam B). + frameSinks[frame.peerId]?(frame) + sendTo(frame.peerId, RemoteCmd.RequestFrame(sender: nil)) + } + + // MARK: - Per-camera controls (focused peer only) + + /// PR3 wires these to the focused lane; capture (all-camera) lands in PR4. + func setZoom(_ factor: CGFloat) { focusedSend(RemoteCmd.SetZoom(zoomFactor: factor)) } + func toggleTorch() { focusedSend(RemoteCmd.ToggleTorch()) } + + func focusAtPoint(x: Float, y: Float) { + guard let peer = focusedPeer, links[peer]?.capabilities?.supportsFocusPoint == true else { return } + sendTo(peer, RemoteCmd.FocusAtPoint(x: x, y: y)) + } + + func switchLens(_ lens: CameraLensType) { focusedSend(RemoteCmd.SwitchLens(lensType: lens)) } + + public nonisolated func setFocusedPeer(_ peer: MCPeerID) { tell(MCPeerCommand(.focus, peer)) } + + private func handleSetFocusedPeer(_ peer: MCPeerID) { + guard links[peer] != nil else { return } + focusedPeer = peer + syncFocusFlags() + // Retier previews: the newly focused camera goes full-size, the rest + // (including the one that just lost focus) drop to thumbnail. + for p in order { pushProfile(to: p) } + markLanesDirty() + } + + /// The preview tier a camera should be on: full for the focused lane, + /// thumbnail for the rest. + private func desiredProfile(for peer: MCPeerID) -> StreamProfile { + peer == focusedPeer ? .focused : .thumbnail + } + + /// Push a camera its preview profile, but only when it actually changes — + /// and only to a multicam-capable peer (an old peer would misread the + /// unknown action). + private func pushProfile(to peer: MCPeerID) { + guard let link = links[peer], link.supportsMulticam else { return } + let profile = desiredProfile(for: peer) + guard link.lastSentProfile != profile else { return } + link.lastSentProfile = profile + sendTo(peer, RemoteCmd.SetStreamProfile( + maxLongEdge: Int(profile.maxLongEdge), + bitrateKbps: Int(profile.bitrateKbps), + fps: Int(profile.fps))) + } + + /// Logically remove a camera from the rig. The QUIC session has no + /// per-peer teardown, so this stops the lane (no more acks/handshakes) and + /// drops it from the UI; the peer times out on its side. A true per-peer + /// disconnect needs a transport API and is out of scope here. + public nonisolated func removeCamera(_ peer: MCPeerID) { tell(MCPeerCommand(.remove, peer)) } + + private func handleRemoveCamera(_ peer: MCPeerID) { + links[peer] = nil + frameSinks[peer] = nil + order.removeAll { $0 == peer } + if focusedPeer == peer { focusedPeer = order.first; syncFocusFlags() } + markLanesDirty() + } + + private func focusedSend(_ msg: Message) { + guard let peer = focusedPeer else { return } + sendTo(peer, msg) + } + + // MARK: - Synced photo capture (all cameras) + + /// Test seams. + func setCaptureAckTimeout(_ t: TimeInterval) { captureAckTimeout = t } + func captureStateForTesting() -> (id: String, remaining: Int)? { + if case .capturingPhoto(let id, let remaining) = state { return (id, remaining) } + return nil + } + func recordingStateForTesting() -> (id: String, remaining: Int)? { + if case .recording(let id, let remaining) = state { return (id, remaining) } + return nil + } + func stoppingStateForTesting() -> (id: String, remaining: Int)? { + if case .stoppingRecording(let id, let remaining) = state { return (id, remaining) } + return nil + } + func captureOutcomeForTesting(_ peer: MCPeerID) -> CaptureOutcome? { links[peer]?.captureOutcome } + func isRecordingForTesting(_ peer: MCPeerID) -> Bool { links[peer]?.isRecording ?? false } + func availablePeersForTesting() -> [MCPeerID] { availableOrder } + func cameraCountForTesting() -> Int { order.count } + + /// Test seam: put a lane in the state a real handshake would — linked, with + /// known multicam capability and (optionally) a known clock offset — so + /// capture behavior can be asserted deterministically. + func seedLaneForTesting(_ peer: MCPeerID, supportsMulticam: Bool, offsetMillis: Int64?) { + guard let link = links[peer] else { return } + link.status = .linked + link.capabilities = RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: nil, currentCamera: .back, + currentLens: .wideAngle, currentZoom: 1.0, + supportsMulticam: supportsMulticam, error: nil) + if let offsetMillis { + // t0 == t3 == 0 → rtt 0, midpoint 0, so offset == cameraClock. + link.clockEstimator.recordExchange( + t0Millis: 0, cameraClockMillis: UInt64(bitPattern: offsetMillis), t3Millis: 0) + } + } + + /// The ready multicam cameras, in strip order. + private func readyMulticamLanes() -> [CameraLink] { + order.compactMap { links[$0] }.filter { $0.status == .linked && $0.supportsMulticam } + } + + /// Send a scheduled command to each ready camera at a shared instant, each + /// translated into that camera's own clock by its offset. Returns the + /// captureID and the lanes addressed. `build` makes the per-lane message. + private func fanOutScheduled( + _ build: (_ fireAtCameraClock: UInt64, _ anchor: UInt64, _ captureID: String, + _ index: Int) -> Message, + fallback: (CameraLink) -> Message, + lanes: [CameraLink]) -> String { + let captureID = UUID().uuidString + capturingLanes = Set(lanes.map(\.peerID)) + currentCaptureID = captureID + lastCaptureID = captureID + for link in lanes { link.collection = .idle } + + if lanes.allSatisfy({ $0.latestOffset != nil }) { + let fireAt = SyncClock.nowMillis() + captureLeadMillis + for (index, link) in lanes.enumerated() { + let offset = link.latestOffset?.offsetMillis ?? 0 + let fireAtCameraClock = UInt64(Int64(fireAt) + offset) + sendTo(link.peerID, build(fireAtCameraClock, fireAt, captureID, index + 1)) + } + } else { + // A missing offset means we can't promise a sub-frame instant — fire + // now on each camera, under the same shot id. + for link in lanes { sendTo(link.peerID, fallback(link)) } + } + return captureID + } + + /// Fire a synced photo on every ready multicam camera. If the rig timer is + /// set, run one director-side countdown first (fanned out so subjects see + /// it), then fire. Scheduled at a shared instant when every clock offset is + /// known; else a plain `TakePic` fan-out. No-op unless idle with a camera. + public nonisolated func capturePhoto() { tell(MCCapturePhoto()) } + + private func handleCapturePhoto() { + guard rigTimerSeconds > 0 else { performCapturePhoto(); return } + beginCountdown(.photo) + } + + private func performCapturePhoto() { + guard case .monitoring = state else { return } + let ready = readyMulticamLanes() + guard !ready.isEmpty else { return } + for link in ready { link.captureOutcome = nil } + + let captureID = fanOutScheduled( + { fire, anchor, id, index in + RemoteCmd.ScheduledCapture(fireAtCameraClockMillis: fire, anchorMillis: anchor, + captureId: id, sessionId: self.sessionID, cameraIndex: index) + }, + fallback: { _ in RemoteCmd.TakePic(sender: nil, sendMediaToPeer: false) }, + lanes: ready) + + state = .capturingPhoto(captureId: captureID, acksRemaining: capturingLanes.count) + markLanesDirty() + armAckTimeout(captureID) + } + + /// Start a synced recording on every ready multicam camera (timer-gated). + public nonisolated func startRecording() { tell(MCStartRecording()) } + + private func handleStartRecording() { + guard rigTimerSeconds > 0 else { performStartRecording(); return } + beginCountdown(.record) + } + + private func performStartRecording() { + guard case .monitoring = state else { return } + let ready = readyMulticamLanes() + guard !ready.isEmpty else { return } + for link in ready { link.captureOutcome = nil } + + let captureID = fanOutScheduled( + { fire, anchor, id, index in + RemoteCmd.ScheduledStartRecording(fireAtCameraClockMillis: fire, anchorMillis: anchor, + captureId: id, sessionId: self.sessionID, cameraIndex: index) + }, + fallback: { _ in RemoteCmd.StartRecordingVideo(sender: nil) }, + lanes: ready) + + state = .recording(captureId: captureID, acksRemaining: capturingLanes.count) + markLanesDirty() + armAckTimeout(captureID) + } + + /// Stop the synced recording on every rolling camera, anchored so the clips + /// end together. + public nonisolated func stopRecording() { tell(MCStopRecording()) } + + private func handleStopRecording() { + guard case .recording(_, _) = state else { return } + let rolling = order.compactMap { links[$0] }.filter { $0.isRecording } + guard !rolling.isEmpty else { return } + + let captureID = fanOutScheduled( + { fire, anchor, id, index in + RemoteCmd.ScheduledStopRecording(fireAtCameraClockMillis: fire, anchorMillis: anchor, + captureId: id, sessionId: self.sessionID, cameraIndex: index) + }, + fallback: { _ in RemoteCmd.StopRecordingVideo(sender: nil, sendMediaToPeer: false) }, + lanes: rolling) + + state = .stoppingRecording(captureId: captureID, acksRemaining: capturingLanes.count) + markLanesDirty() + armAckTimeout(captureID) + } + + // MARK: - Rig-wide quality ("the shot belongs to the rig") + + /// Test seams. + func setTimerTickInterval(_ t: TimeInterval) { timerTickInterval = t } + /// Advance the rig countdown one tick deterministically (production uses the + /// one-shot Task; tests set a large interval and drive with this instead). + func advanceTimerForTesting() { advanceCountdown() } + func countdownRemainingForTesting() -> Int? { countdown?.remaining } + func activeVideoQualityForTesting() -> (VideoResolution, VideoFrameRate)? { + activeVideoQuality.map { ($0.resolution, $0.frameRate) } + } + func rigTimerForTesting() -> Int { rigTimerSeconds } + + /// The current rig quality menu, computed from every connected lane's + /// current-camera capabilities (the intersection model). + func rigQualityMenu() -> RigQualityMenu { + let menuLanes: [RigQualityMenu.Lane] = order.enumerated().compactMap { index, peer in + guard let link = links[peer], link.status != .failed, + let info = link.capabilities?.getCurrentCameraInfo() else { return nil } + return RigQualityMenu.Lane(name: link.displayName.isEmpty ? "Camera \(index + 1)" : link.displayName, + info: info) + } + return RigQualityMenu(lanes: menuLanes) + } + + /// Manual video pick — fans the chosen quality to every lane and records it + /// as the active rig setting (first-class; not a mode you leave). + public nonisolated func setVideoQuality(resolution: VideoResolution, frameRate: VideoFrameRate) { + tell(MCSetVideoQuality(resolution, frameRate)) + } + + private func handleSetVideoQuality(resolution: VideoResolution, frameRate: VideoFrameRate) { + activeVideoQuality = (resolution, frameRate) + for peer in order { + sendTo(peer, RemoteCmd.SetVideoQuality(resolution: resolution, frameRate: frameRate)) + } + refreshRematchFlags() + markLanesDirty() + } + + public nonisolated func setPhotoQuality(format: PhotoFormat, hdr: HDRMode) { + tell(MCSetPhotoQuality(format, hdr)) + } + + private func handleSetPhotoQuality(format: PhotoFormat, hdr: HDRMode) { + activePhotoQuality = (format, hdr) + for peer in order { + sendTo(peer, RemoteCmd.SetPhotoQuality(format: format, hdrMode: hdr)) + } + markLanesDirty() + } + + /// "Automatic" / re-match: recompute best-in-intersection and apply it. + public nonisolated func applyAutomaticVideoQuality() { tell(MCAutomaticVideoQuality()) } + + private func handleAutomaticVideoQuality() { + let auto = rigQualityMenu().automaticVideo() + handleSetVideoQuality(resolution: auto.resolution, frameRate: auto.frameRate) + } + + public nonisolated func applyAutomaticPhotoQuality() { tell(MCAutomaticPhotoQuality()) } + + private func handleAutomaticPhotoQuality() { + let auto = rigQualityMenu().automaticPhoto() + handleSetPhotoQuality(format: auto.format, hdr: auto.hdr) + } + + /// After caps change (new lane, device switch), flag any lane that can't + /// honor the running rig video setting — its tile is badged and the tray + /// offers a re-match. Never silently changes the rig. + private func refreshRematchFlags() { + guard let active = activeVideoQuality else { return } + let menu = rigQualityMenu() + for peer in order { + guard let link = links[peer], let info = link.capabilities?.getCurrentCameraInfo() else { continue } + let lane = RigQualityMenu.Lane(name: link.displayName, info: info) + link.needsQualityRematch = !menu.laneCanMatch( + lane, resolution: active.resolution, frameRate: active.frameRate) + } + } + + // MARK: - Rig self-timer (inbox-driven countdown) + + public nonisolated func setRigTimer(_ seconds: Int) { tell(MCSetRigTimer(max(0, seconds))) } + + private func handleSetRigTimer(_ seconds: Int) { + rigTimerSeconds = seconds + markRigDirty() + } + + /// Begin a director-side countdown, fanned out to every camera so subjects + /// see it, then fire the synced capture at zero. Each tick is a `tell` on + /// the inbox (so it is ordered with everything else and visible to + /// `waitForIdle`); the only Task is a one-shot sleep between ticks. + private func beginCountdown(_ action: MulticamTimedAction) { + countdown = (remaining: rigTimerSeconds, action: action) + fanOutTimerTick(rigTimerSeconds) + scheduleNextTick() + } + + /// Advance one tick (pump-driven). Fires the capture at zero. + private func advanceCountdown() { + guard let c = countdown else { return } + let remaining = c.remaining - 1 + fanOutTimerTick(remaining) + if remaining > 0 { + countdown = (remaining: remaining, action: c.action) + scheduleNextTick() + } else { + countdown = nil + switch c.action { + case .photo: performCapturePhoto() + case .record: performStartRecording() + } + } + } + + /// One-shot: after `timerTickInterval`, enqueue the next tick. Production + /// only — tests drive `advanceCountdown` directly via `advanceTimerForTesting`. + private func scheduleNextTick() { + let interval = timerTickInterval + timerTask.value?.cancel() + timerTask.value = Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(interval * 1_000_000_000)) + self?.tell(MCTimerAdvance()) + } + } + + private func fanOutTimerTick(_ remaining: Int) { + for peer in order { sendTo(peer, RemoteCmd.TimerCountdown(value: remaining)) } + markRigDirty() // the countdown value is read from `countdown` state at publish + } + + private func publishRigSettingsNow() { + let menu = rigQualityMenu() + let snapshot = RigSettingsSnapshot( + timerSeconds: rigTimerSeconds, + countdown: countdown?.remaining, + activeVideo: activeVideoQuality.map { + RigVideoSelection(resolution: $0.resolution, frameRate: $0.frameRate) + }, + videoOptions: menu.videoPickerOptions(), + heifAvailable: menu.supportsHEIF(), + hdrAvailable: menu.supportsHDR(), + heifBlockedBy: menu.lanesBlockingHEIF(), + hdrBlockedBy: menu.lanesBlockingHDR(), + activePhotoFormat: activePhotoQuality?.format, + activeHDR: activePhotoQuality?.hdr) + let display = display + OperationQueue.main.addOperation { display?.applyRigSettings(snapshot) } + } + + // MARK: Ack aggregation (shared across photo / start / stop) + + private func resolvePhotoAck(from peer: MCPeerID, success: Bool) { + guard case .capturingPhoto = state, capturingLanes.contains(peer) else { return } + links[peer]?.captureOutcome = success ? .captured : .failed + finishLane(peer) + } + + private func resolveRecordingAck(from peer: MCPeerID, isStop: Bool, success: Bool) { + switch state { + case .recording where !isStop: + guard capturingLanes.contains(peer) else { return } + if success { links[peer]?.isRecording = true } else { links[peer]?.captureOutcome = .failed } + finishLane(peer) + case .stoppingRecording where isStop: + guard capturingLanes.contains(peer) else { return } + links[peer]?.isRecording = false + finishLane(peer) + default: + break + } + } + + /// Count one lane's answer and advance the aggregate. A photo or a stop + /// that empties the set returns to monitoring; a start that empties it + /// stays `.recording` (the rig is now rolling). + private func finishLane(_ peer: MCPeerID) { + capturingLanes.remove(peer) + let remaining = capturingLanes.count + switch state { + case .capturingPhoto(let id, _): + state = remaining == 0 ? .monitoring(mode: .photo) : .capturingPhoto(captureId: id, acksRemaining: remaining) + case .recording(let id, _): + state = .recording(captureId: id, acksRemaining: remaining) + case .stoppingRecording(let id, _): + state = remaining == 0 ? .monitoring(mode: .photo) : .stoppingRecording(captureId: id, acksRemaining: remaining) + case .monitoring: + break + } + markLanesDirty() + } + + private func armAckTimeout(_ captureID: String) { + let timeout = captureAckTimeout + Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(timeout * 1_000_000_000)) + guard let self else { return } + await self.expireAcks(captureID) + } + } + + /// Any camera silent past the deadline is settled so the aggregate always + /// resolves: a silent photo lane failed; a silent start lane failed (not + /// rolling); a silent stop lane is forced stopped. + private func expireAcks(_ captureID: String) { + let matches: Bool + switch state { + case .capturingPhoto(let id, _), .recording(let id, _), .stoppingRecording(let id, _): + matches = id == captureID + case .monitoring: + matches = false + } + guard matches else { return } + + for peer in capturingLanes { + switch state { + case .capturingPhoto: links[peer]?.captureOutcome = .failed + case .recording: links[peer]?.captureOutcome = .failed + case .stoppingRecording: links[peer]?.isRecording = false + case .monitoring: break + } + } + capturingLanes.removeAll() + currentCaptureID = nil + switch state { + case .recording(let id, _): + state = .recording(captureId: id, acksRemaining: 0) // rolling, start acks settled + default: + state = .monitoring(mode: .photo) + } + markLanesDirty() + } + + // MARK: - Auto-collect (footage back to the director) + + /// Test seams. + func collectionStateForTesting(_ peer: MCPeerID) -> CameraLink.LaneCollectionState? { links[peer]?.collection } + + /// The lane's 1-based index, for the shared RS_ filename group. + private func cameraIndex(of peer: MCPeerID) -> Int { + (order.firstIndex(of: peer) ?? 0) + 1 + } + + private func rigMetadata(for peer: MCPeerID) -> CaptureSyncMetadata { + CaptureSyncMetadata( + sessionID: sessionID, captureID: lastCaptureID ?? UUID().uuidString, + cameraIndex: cameraIndex(of: peer), anchorMillis: 0, + clockOffsetMillis: 0, roundTripMillis: 0) + } + + /// Save a camera's returned still (already EXIF-stamped on the camera) to + /// the director's library under the shared RS_ name, and mark the lane + /// collected. + private func collectPhoto(_ data: Data, from peer: MCPeerID) { + guard let link = links[peer] else { return } + let name = rigMetadata(for: peer).photoFilename(isHEIC: Self.isHEIC(data)) + link.collection = .collected + markLanesDirty() + Self.savePhotoToLibrary(data, originalFilename: name) + } + + private func handleResourceStarted(_ started: ResourceTransferStarted) { + guard let link = links[started.peer] else { return } + link.collection = .transferring(0) + markLanesDirty() + } + + private func handleResourceFinished(_ finished: ResourceTransferFinished) { + guard let link = links[finished.peer] else { + // No lane owns this transfer any more (camera removed mid-flight); + // still delete the temp file the transport handed us. + finished.localURL.map(Self.discardTempFile) + return + } + guard finished.error == nil, let localURL = finished.localURL else { + // Footage is still safe on the camera; the tile offers a retry. Any + // partial temp file is ours to clean up. + finished.localURL.map(Self.discardTempFile) + link.collection = .failed + markLanesDirty() + return + } + link.collection = .collected + markLanesDirty() + // The resource is named with the RS_ filename by the camera, so save it + // under that; QuickTime sync metadata rides inside the .mov itself. The + // controller owns `localURL` until `saveVideoToLibrary` either moves it + // into the library or deletes it. + Self.saveVideoToLibrary(at: localURL, originalFilename: finished.name) + } + + /// The transport hands the director a temp file per received clip; the + /// controller owns it and must not leak it on any path that doesn't move it + /// into the photo library. + private static func discardTempFile(_ url: URL) { + try? FileManager.default.removeItem(at: url) + } + + /// Re-request a failed lane's footage (the camera still holds it). + public nonisolated func retryCollection(for peer: MCPeerID) { tell(MCPeerCommand(.retryCollection, peer)) } + + private func handleRetryCollection(_ peer: MCPeerID) { + guard let link = links[peer], link.collection == .failed else { return } + link.collection = .transferring(0) + markLanesDirty() + sendTo(peer, RemoteCmd.RequestVideoResend(captureId: lastCaptureID ?? "")) + } + + private static func isHEIC(_ data: Data) -> Bool { + data.count > 12 && data[4] == 0x66 && data[5] == 0x74 && data[6] == 0x79 && data[7] == 0x70 + } + + private static func savePhotoToLibrary(_ data: Data, originalFilename: String) { + PHPhotoLibrary.requestAuthorization { status in + guard status == .authorized else { return } + PHPhotoLibrary.shared().performChanges({ + let options = PHAssetResourceCreationOptions() + options.originalFilename = originalFilename + PHAssetCreationRequest.forAsset().addResource(with: .photo, data: data, options: options) + }, completionHandler: { ok, _ in + print(ok ? "Director collected photo \(originalFilename)" : "collect photo failed") + }) + } + } + + private static func saveVideoToLibrary(at url: URL, originalFilename: String) { + PHPhotoLibrary.requestAuthorization { status in + guard status == .authorized else { + discardTempFile(url) // never authorized to import — don't leak it + return + } + PHPhotoLibrary.shared().performChanges({ + let options = PHAssetResourceCreationOptions() + options.shouldMoveFile = true + options.originalFilename = originalFilename + PHAssetCreationRequest.forAsset().addResource(with: .video, fileURL: url, options: options) + }, completionHandler: { ok, _ in + // `shouldMoveFile` consumes the file only on success; on failure + // it stays behind, so the owner removes it. + if !ok { discardTempFile(url) } + print(ok ? "Director collected clip \(originalFilename)" : "collect clip failed") + }) + } + } + + /// A lane's stall watchdog fired — re-request a frame to unstick just that + /// camera's pump (the others are unaffected). + public nonisolated func nudgeFrame(for peer: MCPeerID) { tell(MCPeerCommand(.nudgeFrame, peer)) } + + private func handleNudgeFrame(_ peer: MCPeerID) { + guard links[peer] != nil else { return } + sendTo(peer, RemoteCmd.RequestFrame(sender: nil)) + } + + /// A lane's decoder desynced — force a keyframe, but only from a camera + /// that has proven it speaks VP9 (else an old peer reads the unknown + /// action as TakePicture). Mirrors the 1:1 `requestKeyframeIfVP9` gate. + public nonisolated func requestKeyframe(for peer: MCPeerID) { tell(MCPeerCommand(.requestKeyframe, peer)) } + + private func handleRequestKeyframe(_ peer: MCPeerID) { + guard links[peer]?.sawVP9 == true else { return } + sendTo(peer, RemoteCmd.RequestKeyframe(sender: nil), mode: .reliable) + } + + // MARK: - Clock sync + + private func startClockSyncLoop() { + clockSyncTask.value?.cancel() + let interval = clockSyncInterval + clockSyncTask.value = Task { [weak self] in + while !Task.isCancelled { + try? await Task.sleep(nanoseconds: UInt64(interval * 1_000_000_000)) + guard let self, !Task.isCancelled else { return } + await self.pingClocks() + } + } + } + + private func pingClocks() { + for peer in order where links[peer]?.supportsMulticam == true { + sendTo(peer, RemoteCmd.ClockSyncPing(t0Millis: SyncClock.nowMillis())) + } + } + + // MARK: - Sending + + @discardableResult + private func sendTo(_ peer: MCPeerID, _ msg: Message, + mode: MCSessionSendDataMode = .reliable) -> Bool { + transportShared.value?.send(msg, to: [peer], mode: mode) ?? false + } + + // MARK: - Snapshots + + // Each lane declares its own snapshot (see `CameraLink.snapshot`); the + // controller just gathers them in order. + private func laneSnapshot() -> [MulticamLaneInfo] { + order.compactMap { links[$0]?.snapshot } + } + + private func publishLanesNow() { + let snapshot = laneSnapshot() + let capturing: Bool + if case .capturingPhoto = state { capturing = true } else { capturing = false } + let recording: Bool + switch state { + case .recording, .stoppingRecording: recording = true + default: recording = false + } + let display = display + OperationQueue.main.addOperation { + display?.applyLanes(snapshot) + display?.applyShutterState(capturing: capturing, recording: recording) + } + } + + private func isPeerCompatible(_ became: RemoteCmd.RoleAnnouncement) -> Bool { + guard let local = PeerAppCompatibility.localVersion else { return true } + return PeerAppCompatibility.decide(local: local, + remoteShortVersion: became.shortVersion) == .compatible + } +} + +/// Wraps a camera-addressed inbound message with its source peer, so the +/// nonisolated delegate can enqueue routing work onto the FIFO inbox without +/// losing the `from` that Seam A preserved. +final class RoutedMessage: Message, @unchecked Sendable { + let message: Message + let peer: MCPeerID + init(message: Message, peer: MCPeerID) { + self.message = message + self.peer = peer + super.init(sender: nil) + } +} + +/// A clock-sync pong with its arrival time already stamped at receipt, so the +/// RTT is measured before inbox queuing but the record is still ordered. +final class ClockPongMeasured: Message, @unchecked Sendable { + let pong: RemoteCmd.ClockSyncPong + let t3: UInt64 + let peer: MCPeerID + init(pong: RemoteCmd.ClockSyncPong, t3: UInt64, peer: MCPeerID) { + self.pong = pong + self.t3 = t3 + self.peer = peer + super.init(sender: nil) + } +} + +// MARK: - MultipeerServiceDelegate + +extension MulticamController: MultipeerServiceDelegate { + + public nonisolated func didReceiveMessage(_ message: Message, from peer: MCPeerID) { + // Stamp the pong's arrival time (`t3`) here, at receipt, so the RTT is + // measured before any queuing — then route it through the inbox like + // everything else, so ordering and `waitForIdle` hold. Everything else + // carries its source through `RoutedMessage`. + if let pong = message as? RemoteCmd.ClockSyncPong { + tell(ClockPongMeasured(pong: pong, t3: SyncClock.nowMillis(), peer: peer)) + return + } + tell(RoutedMessage(message: message, peer: peer)) + } + + private func storePong(_ pong: RemoteCmd.ClockSyncPong, t3: UInt64, from peer: MCPeerID) { + guard let link = links[peer] else { return } + link.clockEstimator.recordExchange( + t0Millis: pong.echoT0Millis, + cameraClockMillis: pong.cameraClockMillis, + t3Millis: t3) + markLanesDirty() + } + + public nonisolated func didReceiveFrameRequest(_ request: RemoteCmd.RequestFrame) { + // The director never sends frames, so a frame-credit ack is a no-op. + } + + public nonisolated func didReceiveFrame(_ frame: RemoteCmd.SendFrame, from peer: MCPeerID) { + tell(RemoteCmd.OnFrame(data: frame.data, + sender: nil, + peerId: peer, + fps: frame.fps, + camPosition: frame.camPosition, + camOrientation: frame.camOrientation, + codec: frame.codec, + sequenceNumber: frame.sequenceNumber)) + } + + public nonisolated func peerDidConnect(_ peer: MCPeerID) { + tell(OnConnectToDevice(peer: peer, sender: nil)) + } + + public nonisolated func peerDidDisconnect(_ peer: MCPeerID) { + tell(DisconnectPeer(peer: peer, sender: nil)) + } + + public nonisolated func didDetectIncompatibility() {} + + public nonisolated func browserDidFindPeer(_ peer: MCPeerID) { + tell(UICmd.BrowserFoundPeer(peer: peer)) + } + + public nonisolated func browserDidLosePeer(_ peer: MCPeerID) { + tell(UICmd.BrowserLostPeer(peer: peer)) + } + + public nonisolated func browserDidFail(_ error: Error) {} + public nonisolated func advertiserDidFail(_ error: Error) {} + + public nonisolated func didStartReceivingResource(name: String, from peer: MCPeerID, progress: Progress) { + tell(ResourceTransferStarted(peer: peer, name: name, progress: progress)) + } + + public nonisolated func didFinishReceivingResource(name: String, from peer: MCPeerID, at localURL: URL?, error: Error?) { + tell(ResourceTransferFinished(peer: peer, name: name, localURL: localURL, error: error)) + } +} + +/// A footage transfer from one camera started (carries a `Progress` to observe). +final class ResourceTransferStarted: Message, @unchecked Sendable { + let peer: MCPeerID + let name: String + let progress: Progress + init(peer: MCPeerID, name: String, progress: Progress) { + self.peer = peer; self.name = name; self.progress = progress + super.init(sender: nil) + } +} + +/// A footage transfer from one camera finished (or failed). +final class ResourceTransferFinished: Message, @unchecked Sendable { + let peer: MCPeerID + let name: String + let localURL: URL? + let error: Error? + init(peer: MCPeerID, name: String, localURL: URL?, error: Error?) { + self.peer = peer; self.name = name; self.localURL = localURL; self.error = error + super.init(sender: nil) + } +} + +// MARK: - Director UI-command messages (single-entry inbox) +// +// Every UI intent is a message on the same FIFO inbox as transport events, so +// there is one arrival-ordered stream and one place state changes — the pump. +// (`install`/`setDisplay` stay direct: they run before the pump matters.) + +enum MulticamTimedAction { case photo, record } + +final class MCCapturePhoto: Message, @unchecked Sendable {} +final class MCStartRecording: Message, @unchecked Sendable {} +final class MCStopRecording: Message, @unchecked Sendable {} +final class MCAutomaticVideoQuality: Message, @unchecked Sendable {} +final class MCAutomaticPhotoQuality: Message, @unchecked Sendable {} +final class MCTimerAdvance: Message, @unchecked Sendable {} + +final class MCPeerCommand: Message, @unchecked Sendable { + enum Kind { case focus, invite, remove, retryCollection, nudgeFrame, requestKeyframe, reconnectTick } + let kind: Kind + let peer: MCPeerID + init(_ kind: Kind, _ peer: MCPeerID) { self.kind = kind; self.peer = peer; super.init(sender: nil) } +} + +final class MCSetVideoQuality: Message, @unchecked Sendable { + let resolution: VideoResolution + let frameRate: VideoFrameRate + init(_ resolution: VideoResolution, _ frameRate: VideoFrameRate) { + self.resolution = resolution; self.frameRate = frameRate; super.init(sender: nil) + } +} + +final class MCSetPhotoQuality: Message, @unchecked Sendable { + let format: PhotoFormat + let hdr: HDRMode + init(_ format: PhotoFormat, _ hdr: HDRMode) { self.format = format; self.hdr = hdr; super.init(sender: nil) } +} + +final class MCSetRigTimer: Message, @unchecked Sendable { + let seconds: Int + init(_ seconds: Int) { self.seconds = seconds; super.init(sender: nil) } +} diff --git a/RemoteCam/MulticamView.swift b/RemoteCam/MulticamView.swift new file mode 100644 index 00000000..4cad84ac --- /dev/null +++ b/RemoteCam/MulticamView.swift @@ -0,0 +1,506 @@ +// +// MulticamView.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import MPCCompat +import SwiftUI + +/// The director screen in focus mode: the selected camera fills the viewfinder +/// (reusing the 1:1 monitor's `LiveFrameView` so it looks and behaves the same) +/// with a floating strip of the other cameras' live thumbnails. Grid mode is a +/// later PR; this is the default surface. +struct MulticamView: View { + @ObservedObject var viewModel: MulticamViewModel + + /// Tap a thumbnail to make that camera the focused one. + let onFocusLane: (CameraLane) -> Void + /// The synced shutter — photo, or record start/stop depending on mode. + let onShutter: () -> Void + /// Toggle the shutter between photo and video mode. + let onToggleMode: () -> Void + /// "Add camera" tapped — the host decides paywall vs. the sheet. + let onAddCamera: () -> Void + /// Invite a discovered camera into the rig. + let onInviteCamera: (MCPeerID) -> Void + /// Rig self-timer changed (seconds; 0 = off). + let onSetTimer: (Int) -> Void + /// Rig video quality picked (fans out to every lane). + let onSelectVideoQuality: (VideoResolution, VideoFrameRate) -> Void + /// "Automatic" — recompute best-in-intersection and apply. + let onAutomaticVideoQuality: () -> Void + /// Rig photo format / HDR picked. + let onSetPhotoFormat: (PhotoFormat) -> Void + let onSetHDR: (Bool) -> Void + /// Retry a lane's failed footage collection. + let onRetryCollection: (CameraLane) -> Void + + var body: some View { + GeometryReader { geo in + let dock = MonitorChromeLayout.dock( + viewSize: geo.size, + interfaceOrientation: viewModel.interfaceOrientation, + input: chromeInput) + + ZStack { + Color.black.ignoresSafeArea() + + if viewModel.displayMode == .grid { + gridWall + } else { + focusedViewfinder + stripOverlay(dock: dock) + } + + // The capture cluster stays in both modes; per-camera controls + // (the strip) are hidden in grid. + shutterOverlay(dock: dock) + topControls + countdownOverlay + } + .sheet(isPresented: $viewModel.showingAddCamera) { + AddCameraSheet(peers: viewModel.availablePeers, onInvite: onInviteCamera) + } + .sheet(isPresented: $viewModel.showingRigTray) { + RigTrayView(settings: viewModel.rigSettings, + onSetTimer: onSetTimer, + onSelectVideoQuality: onSelectVideoQuality, + onAutomaticVideoQuality: onAutomaticVideoQuality, + onSetPhotoFormat: onSetPhotoFormat, + onSetHDR: onSetHDR) + } + } + } + + /// Top-left cluster: the rig settings (tray) button, and the focus/grid + /// toggle when there's more than one camera. Reachable in both modes. + private var topControls: some View { + VStack { + HStack(spacing: 12) { + Button { viewModel.showingRigTray = true } label: { + Image(systemName: "slider.horizontal.3") + .font(.title3) + .foregroundColor(.white) + .frame(width: 44, height: 44) + .background(Color.black.opacity(0.4)) + .clipShape(Circle()) + } + if MultiCamChrome.showsGridToggle(cameraCount: viewModel.lanes.count) { + Button { + viewModel.displayMode = viewModel.displayMode == .grid ? .focus : .grid + } label: { + Image(systemName: viewModel.displayMode == .grid + ? "rectangle.inset.filled" : "square.grid.2x2.fill") + .font(.title3) + .foregroundColor(.white) + .frame(width: 44, height: 44) + .background(Color.black.opacity(0.4)) + .clipShape(Circle()) + } + } + Spacer() + } + .padding(.leading, 12) + .padding(.top, 12) + Spacer() + } + } + + /// The rig self-timer countdown, big and centered so subjects see it. + @ViewBuilder + private var countdownOverlay: some View { + if let n = viewModel.rigSettings.countdown, n > 0 { + Text("\(n)") + .font(.system(size: 96, weight: .bold, design: .rounded)) + .foregroundColor(.white) + .shadow(radius: 8) + } + } + + /// An equal grid of every camera — the monitor wall. Tap a tile to focus it + /// (and return to focus mode). + private var gridWall: some View { + let count = viewModel.lanes.count + let columns = Array( + repeating: GridItem(.flexible(), spacing: 4), + count: MultiCamChrome.gridColumnCount(cameraCount: count)) + return LazyVGrid(columns: columns, spacing: 4) { + ForEach(viewModel.lanes) { lane in + CameraTileView(lane: lane, isThumbnail: true, onRetry: { onRetryCollection(lane) }) + .aspectRatio(9.0 / 16.0, contentMode: .fit) + .onTapGesture { + onFocusLane(lane) + viewModel.displayMode = .focus + } + } + } + .padding(8) + } + + /// The all-camera shutter + a photo/video mode toggle, docked on the same + /// edge the 1:1 monitor uses. Reuses the monitor's `ShutterButton` (and its + /// activity ring) so the two screens feel of a piece. + @ViewBuilder + private func shutterOverlay(dock: MonitorChromeDock) -> some View { + let shutter = ShutterButton( + uiState: viewModel.mode == .video ? .videoMode : .photoMode, + isRecording: viewModel.isRecording, + activity: viewModel.isCapturing ? .capturing : nil, + isEnabled: !viewModel.isCapturing && viewModel.focusedLane != nil, + action: onShutter) + + // The mode toggle is hidden while recording (you can't switch mid-clip). + let modeToggle = Button(action: onToggleMode) { + Image(systemName: viewModel.mode == .video ? "video.fill" : "camera.fill") + .font(.title3) + .foregroundColor(.white) + .frame(width: 44, height: 44) + .background(Color.black.opacity(0.4)) + .clipShape(Circle()) + } + .opacity(viewModel.isRecording ? 0 : 1) + .disabled(viewModel.isRecording) + + switch dock { + case .bottom: + VStack { + Spacer() + ZStack { + shutter + HStack { Spacer(); modeToggle.padding(.trailing, 40) } + } + .padding(.bottom, 24) + } + case .leading: + HStack { + VStack { Spacer(); modeToggle; shutter; Spacer() }.padding(.leading, 24) + Spacer() + } + case .trailing: + HStack { + Spacer() + VStack { Spacer(); modeToggle; shutter; Spacer() }.padding(.trailing, 24) + } + } + } + + private var chromeInput: MonitorChromeInput { + #if targetEnvironment(macCatalyst) + return .pointer + #else + return .touch + #endif + } + + @ViewBuilder + private var focusedViewfinder: some View { + if let focused = viewModel.focusedLane { + LiveFrameView(frames: focused.frames, aspectRatio: .sixteenNine) + .ignoresSafeArea() + } else { + // No camera focused yet (all reconnecting, or none linked). + Rectangle() + .fill(Color.gray.opacity(0.25)) + .overlay( + Image(systemName: "video.slash") + .font(.system(size: 44)) + .foregroundColor(.white.opacity(0.5))) + .ignoresSafeArea() + } + } + + /// The thumbnail rail, docked opposite the (future) action cluster on the + /// same axis the 1:1 chrome uses, so the two screens feel of a piece. + @ViewBuilder + private func stripOverlay(dock: MonitorChromeDock) -> some View { + let others = viewModel.otherLanes + switch dock { + case .bottom: + VStack { + Spacer() + HStack(spacing: 8) { + ForEach(others) { lane in + CameraTileView(lane: lane, isThumbnail: true, onRetry: { onRetryCollection(lane) }) + .frame(width: 96, height: 128) + .onTapGesture { onFocusLane(lane) } + } + addCameraTile.frame(width: 96, height: 128) + } + .padding(.bottom, 96) + } + case .leading, .trailing: + HStack { + if dock == .trailing { Spacer() } + VStack(spacing: 8) { + ForEach(others) { lane in + CameraTileView(lane: lane, isThumbnail: true, onRetry: { onRetryCollection(lane) }) + .frame(width: 128, height: 96) + .onTapGesture { onFocusLane(lane) } + } + addCameraTile.frame(width: 128, height: 96) + } + .padding(dock == .leading ? .leading : .trailing, 12) + if dock == .leading { Spacer() } + } + } + } + + /// The "add camera" affordance at the end of the strip. The host decides + /// whether tapping opens the sheet or the paywall (at the tier cap). + private var addCameraTile: some View { + Button(action: onAddCamera) { + VStack(spacing: 6) { + Image(systemName: "plus.circle.fill").font(.title) + Text(NSLocalizedString("Add camera", comment: "add a camera to the multicam rig")) + .font(.caption2) + } + .foregroundColor(.white) + .frame(maxWidth: .infinity, maxHeight: .infinity) + .background(Color.white.opacity(0.12)) + .clipShape(RoundedRectangle(cornerRadius: 10)) + } + } +} + +/// One camera's tile: its isolated live frame, a name chip, a focus ring, and +/// a reconnecting scrim. `Equatable` on the value inputs so a frame delivered +/// to another lane can't invalidate this tile's chrome — only its own +/// `LiveFrameView` (observing its own `FrameDisplayModel`) re-renders. +struct CameraTileView: View { + @ObservedObject var lane: CameraLane + var isThumbnail: Bool = false + /// Retry a failed footage collection for this lane (nil = not offered). + var onRetry: (() -> Void)? = nil + + var body: some View { + ZStack { + LiveFrameView(frames: lane.frames, aspectRatio: .sixteenNine) + .clipShape(RoundedRectangle(cornerRadius: isThumbnail ? 10 : 0)) + .saturation(lane.status == .linked ? 1 : 0) + + collectionBadge + + if lane.status == .reconnecting { + RoundedRectangle(cornerRadius: isThumbnail ? 10 : 0) + .fill(Color.black.opacity(0.45)) + .overlay( + Text(NSLocalizedString("RECONNECTING", comment: "peer link dropped")) + .font(.caption2.weight(.semibold)) + .foregroundColor(.white)) + } + + if lane.needsQualityRematch { + VStack { + HStack { + Spacer() + Image(systemName: "exclamationmark.triangle.fill") + .font(.caption) + .foregroundColor(.yellow) + .padding(6) + } + Spacer() + } + } + + if lane.isRecording { + VStack { + HStack { + Image(systemName: "record.circle.fill") + .font(.body) + .foregroundColor(.red) + Spacer() + } + Spacer() + } + .padding(6) + } else if let outcome = lane.captureOutcome { + VStack { + HStack { + Spacer() + Image(systemName: outcome == .captured + ? "checkmark.circle.fill" : "exclamationmark.triangle.fill") + .font(.body) + .foregroundColor(outcome == .captured ? .green : .yellow) + .padding(6) + } + Spacer() + } + } + + VStack { + Spacer() + Text(lane.displayName) + .font(.caption2) + .lineLimit(1) + .foregroundColor(.white) + .padding(.horizontal, 6) + .padding(.vertical, 2) + .background(Color.black.opacity(0.5)) + .clipShape(Capsule()) + .padding(4) + } + } + .overlay( + RoundedRectangle(cornerRadius: isThumbnail ? 10 : 0) + .stroke(lane.isFocused ? AppTheme.accent : .clear, lineWidth: 3)) + } + + /// Post-take footage collection: a spinner while transferring, a check when + /// collected, and a tappable retry when the transfer failed. + @ViewBuilder + private var collectionBadge: some View { + switch lane.collection { + case .idle: + EmptyView() + case .transferring: + VStack { + Spacer() + HStack { + ProgressView().tint(.white) + Image(systemName: "square.and.arrow.down").foregroundColor(.white) + } + Spacer() + } + case .collected: + VStack { + HStack { + Spacer() + Image(systemName: "checkmark.icloud.fill").foregroundColor(.green).padding(6) + } + Spacer() + } + case .failed: + Button { onRetry?() } label: { + VStack(spacing: 4) { + Image(systemName: "arrow.clockwise.icloud").font(.title3) + Text(NSLocalizedString("Retry", comment: "retry footage collection")).font(.caption2) + } + .foregroundColor(.white) + .padding(8) + .background(Color.black.opacity(0.55)) + .clipShape(RoundedRectangle(cornerRadius: 8)) + } + } + } +} + +/// Lists the cameras the director's browser has discovered but not yet added, +/// so the user can invite them into the rig mid-session. Shown only below the +/// tier cap (the host routes to the paywall at the cap). +struct AddCameraSheet: View { + let peers: [MCPeerID] + let onInvite: (MCPeerID) -> Void + + var body: some View { + NavigationView { + Group { + if peers.isEmpty { + VStack(spacing: 12) { + ProgressView() + Text(NSLocalizedString("Searching for cameras…", + comment: "add-camera sheet, no peers yet")) + .foregroundColor(.secondary) + } + } else { + List(peers, id: \.self) { peer in + Button { onInvite(peer) } label: { + HStack { + Image(systemName: "iphone.radiowaves.left.and.right") + .foregroundColor(AppTheme.accent) + Text(peer.displayName) + Spacer() + Text(NSLocalizedString("Add", comment: "invite this camera")) + .foregroundColor(AppTheme.accent) + } + } + } + } + } + .navigationTitle(NSLocalizedString("Add camera", + comment: "add a camera to the multicam rig")) + } + } +} + +/// The rig settings tray: one self-timer + rig-wide quality (the intersection +/// model). Manual quality selection is first-class; "Automatic" is the reset at +/// the top. Reachable from both focus and grid modes. +struct RigTrayView: View { + let settings: RigSettingsSnapshot + let onSetTimer: (Int) -> Void + let onSelectVideoQuality: (VideoResolution, VideoFrameRate) -> Void + let onAutomaticVideoQuality: () -> Void + let onSetPhotoFormat: (PhotoFormat) -> Void + let onSetHDR: (Bool) -> Void + + private let timerStops = [0, 3, 5, 10, 20] + + var body: some View { + NavigationView { + Form { + Section(NSLocalizedString("Timer", comment: "rig self-timer")) { + Picker(NSLocalizedString("Timer", comment: ""), + selection: Binding(get: { settings.timerSeconds }, + set: { onSetTimer($0) })) { + ForEach(timerStops, id: \.self) { s in + Text(s == 0 ? NSLocalizedString("Off", comment: "timer off") : "\(s)s").tag(s) + } + } + .pickerStyle(.segmented) + } + + Section(NSLocalizedString("Video Quality", comment: "rig video quality")) { + Button { + onAutomaticVideoQuality() + } label: { + HStack { + Text(NSLocalizedString("Automatic", comment: "best-in-intersection")) + Spacer() + Text(settings.activeVideo?.label + ?? NSLocalizedString("Auto", comment: "automatic rig quality")) + .foregroundColor(.secondary) + } + } + ForEach(settings.videoOptions) { opt in + Button { + if opt.enabled { onSelectVideoQuality(opt.resolution, opt.frameRate) } + } label: { + HStack { + Text(opt.label) + .foregroundColor(opt.enabled ? .primary : .secondary) + Spacer() + if !opt.enabled, let blocker = opt.blockedBy.first { + Text(String(format: NSLocalizedString("%@ can't", comment: "camera blocks a quality"), blocker)) + .font(.caption) + .foregroundColor(.secondary) + } + if settings.activeVideo?.matches(opt) == true { + Image(systemName: "checkmark").foregroundColor(AppTheme.accent) + } + } + } + .disabled(!opt.enabled) + } + } + + Section(NSLocalizedString("Photo", comment: "rig photo quality")) { + Toggle(NSLocalizedString("HEIF", comment: "photo format"), + isOn: Binding(get: { settings.activePhotoFormat == .heif }, + set: { onSetPhotoFormat($0 ? .heif : .jpeg) })) + .disabled(!settings.heifAvailable) + Toggle(NSLocalizedString("HDR", comment: "photo HDR"), + isOn: Binding(get: { settings.activeHDR == .on }, + set: { onSetHDR($0) })) + .disabled(!settings.hdrAvailable) + if !settings.hdrAvailable, let blocker = settings.hdrBlockedBy.first { + Text(String(format: NSLocalizedString("%@ can't do HDR", comment: "camera blocks HDR"), blocker)) + .font(.caption).foregroundColor(.secondary) + } + } + } + .navigationTitle(NSLocalizedString("Rig Settings", comment: "multicam settings tray")) + } + } +} diff --git a/RemoteCam/MulticamViewController.swift b/RemoteCam/MulticamViewController.swift new file mode 100644 index 00000000..2474170f --- /dev/null +++ b/RemoteCam/MulticamViewController.swift @@ -0,0 +1,190 @@ +// +// MulticamViewController.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import MPCCompat +import StoreKit +import SwiftUI +import UIKit + +/// Hosts the multicam director screen. Owns the `MulticamController` actor and +/// the `MulticamViewModel`, and bridges the two: controller snapshots become +/// lane updates, and per-lane preview frames are routed to exactly one lane's +/// decoder (the rendering-isolation contract). +/// +/// The 1:1 `MonitorViewController` is untouched; this is a parallel screen +/// reached only from a multicam session. +public final class MulticamViewController: UIViewController { + + private let controller: MulticamController + private let viewModel = MulticamViewModel() + private var hosting: UIHostingController? + + /// `controller` must already be `install`-ed with its transport + peers by + /// the caller (the scanner handoff), so lanes light up immediately. + init(controller: MulticamController) { + self.controller = controller + super.init(nibName: nil, bundle: nil) + } + + @available(*, unavailable) + required init?(coder: NSCoder) { fatalError("init(coder:) has not been implemented") } + + public override var supportedInterfaceOrientations: UIInterfaceOrientationMask { .allButUpsideDown } + public override var shouldAutorotate: Bool { true } + + public override func viewDidLoad() { + super.viewDidLoad() + view.backgroundColor = .black + + // The controller's UI commands are `nonisolated` (they `tell` the + // single inbox), so the call sites are plain synchronous sends — no + // `Task { await }` wrappers. + let multicamView = MulticamView( + viewModel: viewModel, + onFocusLane: { [weak self] lane in self?.controller.setFocusedPeer(lane.peerID) }, + onShutter: { [weak self] in self?.triggerShutter() }, + onToggleMode: { [weak self] in + guard let self, !self.viewModel.isRecording else { return } + self.viewModel.mode = self.viewModel.mode == .photo ? .video : .photo + }, + onAddCamera: { [weak self] in self?.handleAddCameraTapped() }, + onInviteCamera: { [weak self] peer in + guard let self else { return } + self.viewModel.showingAddCamera = false + self.controller.inviteCamera(peer) + }, + onSetTimer: { [weak self] seconds in self?.controller.setRigTimer(seconds) }, + onSelectVideoQuality: { [weak self] res, fps in + self?.controller.setVideoQuality(resolution: res, frameRate: fps) + }, + onAutomaticVideoQuality: { [weak self] in self?.controller.applyAutomaticVideoQuality() }, + onSetPhotoFormat: { [weak self] format in + self?.controller.setPhotoQuality(format: format, + hdr: self?.viewModel.rigSettings.activeHDR ?? .off) + }, + onSetHDR: { [weak self] on in + self?.controller.setPhotoQuality( + format: self?.viewModel.rigSettings.activePhotoFormat ?? .jpeg, + hdr: on ? .on : .off) + }, + onRetryCollection: { [weak self] lane in self?.controller.retryCollection(for: lane.peerID) }) + hosting = embedSwiftUIView(multicamView) + + Task { await controller.setDisplay(self) } + } + + public override func viewWillAppear(_ animated: Bool) { + super.viewWillAppear(animated) + navigationController?.setNavigationBarHidden(true, animated: animated) + syncInterfaceOrientation() + } + + public override func viewWillTransition(to size: CGSize, + with coordinator: UIViewControllerTransitionCoordinator) { + super.viewWillTransition(to: size, with: coordinator) + coordinator.animate(alongsideTransition: { _ in self.syncInterfaceOrientation() }) + } + + public override func viewWillDisappear(_ animated: Bool) { + super.viewWillDisappear(animated) + navigationController?.setNavigationBarHidden(false, animated: animated) + } + + /// "Add camera" tapped: at the tier cap, route free users to the paywall + /// (the same Settings sheet every other gate uses); otherwise open the + /// discovered-cameras sheet. + private func handleAddCameraTapped() { + Task { @MainActor in + let count = await controller.cameraCount() + if count >= StoreManager.shared.maxCameras() { + showPaywall() + } else { + viewModel.showingAddCamera = true + } + } + } + + /// Reuse the existing Settings/paywall sheet — no bespoke multicam paywall. + func showPaywall() { + let ctrl = UIHostingController(rootView: SettingsView()) + ctrl.modalPresentationStyle = .pageSheet + present(ctrl, animated: true) + } + + /// Route the shutter: a photo, or record start/stop, per the current mode. + private func triggerShutter() { + switch (viewModel.mode, viewModel.isRecording) { + case (.photo, _): controller.capturePhoto() + case (.video, false): controller.startRecording() + case (.video, true): controller.stopRecording() + } + } + + private func syncInterfaceOrientation() { + let orientation = view.window?.windowScene?.interfaceOrientation ?? .portrait + if viewModel.interfaceOrientation != orientation { + viewModel.interfaceOrientation = orientation + } + } + + deinit { + for lane in viewModel.lanes { lane.receiver.invalidate() } + controller.stop() + } + + /// Wire a freshly created lane's decoder: its frames drive only its own + /// `FrameDisplayModel`, and its stall/keyframe recovery targets only its + /// own peer on the controller. + private func wire(_ lane: CameraLane) { + let peer = lane.peerID + lane.receiver.onImage = { [weak lane] image in + OperationQueue.main.addOperation { lane?.frames.cameraImage = image } + } + lane.receiver.onStall = { [weak self] in + self?.controller.nudgeFrame(for: peer) + } + lane.receiver.onKeyframeNeeded = { [weak self] in + self?.controller.requestKeyframe(for: peer) + } + lane.receiver.start() + // Hand the controller a sink that feeds only this lane's decoder, so it + // routes frames actor → closure without touching this view controller. + // `receive` is queue-hopping and thread-safe; the weak capture lets the + // lane deallocate freely. + let receiver = lane.receiver + Task { await controller.setFrameSink(for: peer) { [weak receiver] frame in + receiver?.receive(frame) + } } + } +} + +// MARK: - MulticamDisplay + +extension MulticamViewController: MulticamDisplay { + + func applyLanes(_ lanes: [MulticamLaneInfo]) { + let created = viewModel.apply(lanes) + for lane in created { wire(lane) } + } + + func applyShutterState(capturing: Bool, recording: Bool) { + viewModel.isCapturing = capturing + viewModel.isRecording = recording + } + + func applyAvailablePeers(_ peers: [MCPeerID]) { + viewModel.availablePeers = peers + } + + func applyRigSettings(_ settings: RigSettingsSnapshot) { + viewModel.rigSettings = settings + } + + func exitMulticam() { + navigationController?.popViewController(animated: true) + } +} diff --git a/RemoteCam/MulticamViewModel.swift b/RemoteCam/MulticamViewModel.swift new file mode 100644 index 00000000..ef969b0d --- /dev/null +++ b/RemoteCam/MulticamViewModel.swift @@ -0,0 +1,111 @@ +// +// MulticamViewModel.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import Combine +import Foundation +import MPCCompat +import SwiftUI + +/// One camera's UI state in the director screen. Its `frames` model is +/// isolated exactly like the 1:1 monitor's `FrameDisplayModel`: only this +/// lane's tile observes it, so a 20fps stream from camera B never re-renders +/// camera A's tile or the surrounding chrome. +final class CameraLane: ObservableObject, Identifiable { + let peerID: MCPeerID + var id: MCPeerID { peerID } + let displayName: String + + /// Live preview frames for this lane only (not `@Published` on the parent + /// view model — see `FrameDisplayModel`). + let frames = FrameDisplayModel() + + /// The whole lane snapshot, published as one value — the fields below are + /// pure reads, so a status change and a badge change coalesce into a single + /// SwiftUI invalidation (frames are separate, in `FrameDisplayModel`). + @Published private(set) var info: MulticamLaneInfo + + var status: CameraLink.Status { info.status } + var isFocused: Bool { info.isFocused } + var captureOutcome: CaptureOutcome? { info.captureOutcome } + var isRecording: Bool { info.isRecording } + var needsQualityRematch: Bool { info.needsQualityRematch } + var collection: CameraLink.LaneCollectionState { info.collection } + + /// This lane's own decoder + stall watchdog. The view controller wires its + /// `onImage` to set `frames.cameraImage`, and its stall/keyframe callbacks + /// back to the controller for this peer. + let receiver = FrameStreamReceiver() + + init(info: MulticamLaneInfo) { + self.peerID = info.peerID + self.displayName = info.displayName + self.info = info + } + + /// Mechanical reconcile: one Equatable compare, one assignment. + func update(_ info: MulticamLaneInfo) { + if self.info != info { self.info = info } + } +} + +/// Top-level state for the multicam director screen. Holds the ordered lanes +/// and which one is focused; per-lane frame churn lives in each `CameraLane`. +final class MulticamViewModel: ObservableObject { + @Published private(set) var lanes: [CameraLane] = [] + @Published var interfaceOrientation: UIInterfaceOrientation = .portrait + /// A synced photo is in flight — drives the shutter's activity ring. + @Published var isCapturing: Bool = false + /// The rig is recording — the shutter becomes a stop button. + @Published var isRecording: Bool = false + /// Photo vs video shutter mode. + @Published var mode: MonitorMode = .photo + /// Focus (viewfinder + strip) vs grid (monitor wall) layout. + @Published var displayMode: MulticamDisplayMode = .focus + /// Cameras discovered but not yet in the rig — the add-camera sheet's list. + @Published var availablePeers: [MCPeerID] = [] + /// Whether the add-camera sheet is showing. + @Published var showingAddCamera: Bool = false + /// Rig-wide settings (timer + quality intersection) for the tray. + @Published var rigSettings = RigSettingsSnapshot() + /// Whether the rig settings tray is showing. + @Published var showingRigTray: Bool = false + + var focusedLane: CameraLane? { lanes.first { $0.isFocused } } + var otherLanes: [CameraLane] { lanes.filter { !$0.isFocused } } + + /// Reconcile against a controller snapshot: add new lanes, drop gone ones, + /// preserve existing `CameraLane` instances (and their receivers/frames) + /// so streams are never interrupted by a status change elsewhere. + /// + /// Returns the lanes that were newly created, so the view controller can + /// wire their receivers. + @discardableResult + func apply(_ infos: [MulticamLaneInfo]) -> [CameraLane] { + var existing = Dictionary(uniqueKeysWithValues: lanes.map { ($0.peerID, $0) }) + var created: [CameraLane] = [] + + let next: [CameraLane] = infos.map { info in + if let lane = existing.removeValue(forKey: info.peerID) { + lane.update(info) // one Equatable compare + assignment + return lane + } + let lane = CameraLane(info: info) + created.append(lane) + return lane + } + + // Tear down receivers for lanes that went away. + for gone in existing.values { gone.receiver.invalidate() } + + lanes = next + return created + } + + func lane(for peer: MCPeerID) -> CameraLane? { + lanes.first { $0.peerID == peer } + } +} diff --git a/RemoteCam/MultipeerService.swift b/RemoteCam/MultipeerService.swift index 98200ee8..b197ec36 100644 --- a/RemoteCam/MultipeerService.swift +++ b/RemoteCam/MultipeerService.swift @@ -12,14 +12,17 @@ import Combine import UIKit protocol MultipeerServiceDelegate: AnyObject { - func didReceiveMessage(_ message: Message) + /// `peer` is the message's source. The 1:1 SessionCoordinator can ignore + /// it (there is only one peer a message can come from); a multicam + /// director needs it to route responses to the right camera. + func didReceiveMessage(_ message: Message, from peer: MCPeerID) func didReceiveFrameRequest(_ request: RemoteCmd.RequestFrame) func didReceiveFrame(_ frame: RemoteCmd.SendFrame, from peer: MCPeerID) func peerDidConnect(_ peer: MCPeerID) func peerDidDisconnect(_ peer: MCPeerID) func didDetectIncompatibility() - func didStartReceivingResource(name: String, progress: Progress) - func didFinishReceivingResource(name: String, at localURL: URL?, error: Error?) + func didStartReceivingResource(name: String, from peer: MCPeerID, progress: Progress) + func didFinishReceivingResource(name: String, from peer: MCPeerID, at localURL: URL?, error: Error?) func browserDidFindPeer(_ peer: MCPeerID) func browserDidLosePeer(_ peer: MCPeerID) func browserDidFail(_ error: Error) @@ -205,7 +208,7 @@ class MultipeerService: NSObject, MCSessionDelegate, case let frame as RemoteCmd.SendFrame: delegate?.didReceiveFrame(frame, from: peerID) default: - delegate?.didReceiveMessage(inboundMessage) + delegate?.didReceiveMessage(inboundMessage, from: peerID) } } @@ -215,11 +218,11 @@ class MultipeerService: NSObject, MCSessionDelegate, public func session(_ session: MCSession, didStartReceivingResourceWithName resourceName: String, fromPeer peerID: MCPeerID, with progress: Progress) { - delegate?.didStartReceivingResource(name: resourceName, progress: progress) + delegate?.didStartReceivingResource(name: resourceName, from: peerID, progress: progress) } public func session(_ session: MCSession, didFinishReceivingResourceWithName resourceName: String, fromPeer peerID: MCPeerID, at localURL: URL?, withError error: Error?) { - delegate?.didFinishReceivingResource(name: resourceName, at: localURL, error: error) + delegate?.didFinishReceivingResource(name: resourceName, from: peerID, at: localURL, error: error) } } diff --git a/RemoteCam/RecordingPipeline.swift b/RemoteCam/RecordingPipeline.swift index 99f51cda..87bba920 100644 --- a/RemoteCam/RecordingPipeline.swift +++ b/RemoteCam/RecordingPipeline.swift @@ -71,6 +71,13 @@ class RecordingPipeline { /// writer geometry must not chase a mid-recording aspect change. var recordingAspectRatio: AspectRatio = .sixteenNine + /// Set for a synced multicam recording only: embeds the shot's alignment + /// fields as QuickTime metadata in the .mov and names the saved clip + /// `RS___cam.mov`. Nil for ordinary single-camera recording, + /// which writes and saves exactly as before. Consumed (cleared) when the + /// clip is saved. + var pendingSyncMetadata: CaptureSyncMetadata? + private let videoCropContext = CIContext(options: [.useSoftwareRenderer: false]) private var videoInput: AVAssetWriterInput! @@ -132,6 +139,12 @@ class RecordingPipeline { // Create an asset writer do { self.assetWriter = try AVAssetWriter(outputURL: outputFilePath, fileType: .mov) + // Synced multicam: embed the shot's alignment fields in the file so + // an editor can group and time-align the angles. The anchor rides + // as an opaque numeric key, not a wall-clock date. + if let metadata = pendingSyncMetadata { + self.assetWriter?.metadata = metadata.quickTimeMetadataItems() + } } catch { onError?(NSLocalizedString("Unable to start recording", comment: "")) } @@ -193,18 +206,25 @@ class RecordingPipeline { PHPhotoLibrary.requestAuthorization { [weak self] status in if status == .authorized { // Save the movie file to the photo library and cleanup. + let syncMetadata = self?.pendingSyncMetadata PHPhotoLibrary.shared().performChanges({ let options = PHAssetResourceCreationOptions() - options.shouldMoveFile = true + // A multicam clip is COPIED (not moved) so the temp file + // survives the staggered/retried auto-collect transfer to + // the director; the next recording cleans it up. + options.shouldMoveFile = (syncMetadata == nil) + // Synced clip: name it under the shared RS_ group. + if let syncMetadata { options.originalFilename = syncMetadata.videoFilename() } let creationRequest = PHAssetCreationRequest.forAsset() creationRequest.addResource(with: .video, fileURL: outputFileURL, options: options) }, completionHandler: { success, error in if !success { print("AVCam couldn't save the movie to your photo library: \(String(describing: error))") } - cleanupFileAt(outputFileURL) + if syncMetadata == nil { cleanupFileAt(outputFileURL) } } ) + self?.pendingSyncMetadata = nil } else { DispatchQueue.main.async { self?.onPhotosAccessDenied?() diff --git a/RemoteCam/RemoteCmdFlatBuffers.swift b/RemoteCam/RemoteCmdFlatBuffers.swift index 206e718a..070e7296 100644 --- a/RemoteCam/RemoteCmdFlatBuffers.swift +++ b/RemoteCam/RemoteCmdFlatBuffers.swift @@ -29,6 +29,15 @@ func serializeToFlatBuffer(_ msg: Message) -> Data? { case let m as RemoteCmd.SendFrame: return m.toFlatBuffer() case let m as RemoteCmd.RequestFrame: return m.toFlatBuffer() case let m as RemoteCmd.RequestKeyframe: return m.toFlatBuffer() + case let m as RemoteCmd.ClockSyncPing: return m.toFlatBuffer() + case let m as RemoteCmd.ClockSyncPong: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledCapture: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledCaptureAck: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledStartRecording: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledStopRecording: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledRecordingAck: return m.toFlatBuffer() + case let m as RemoteCmd.SetStreamProfile: return m.toFlatBuffer() + case let m as RemoteCmd.RequestVideoResend: return m.toFlatBuffer() case let m as RemoteCmd.SetZoom: return m.toFlatBuffer() case let m as RemoteCmd.SetZoomResp: return m.toFlatBuffer() case let m as RemoteCmd.FocusAtPoint: return m.toFlatBuffer() @@ -418,7 +427,8 @@ private func encodeCapabilitiesEnvelope( cameraDevicesVectorOffset: devicesVector, activeDeviceIdOffset: activeIDOffset, supportsFocusPoint: c.supportsFocusPoint, - supportsPreviewMode: c.supportsPreviewMode) + supportsPreviewMode: c.supportsPreviewMode, + supportsMulticam: c.supportsMulticam) let stateOffset = RemoteShutter_CameraState.createCameraState( &fbb, @@ -740,6 +750,128 @@ extension RemoteCmd.RequestKeyframe { } } +extension RemoteCmd.ClockSyncPing { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let params = RemoteShutter_CommandParameters.createCommandParameters( + &fbb, clockSyncT0Ms: t0Millis) + return buildCommand(&fbb, action: .clocksyncping, parameters: params) + } +} + +extension RemoteCmd.ClockSyncPong { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let resp = RemoteShutter_CameraStateResponse.createCameraStateResponse( + &fbb, + action: .clocksyncping, + success: true, + clockSyncEchoT0Ms: echoT0Millis, + clockSyncCameraClockMs: cameraClockMillis) + return buildResponse(&fbb, action: .clocksyncping, response: resp) + } +} + +extension RemoteCmd.ScheduledCapture { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let captureIdOffset = fbb.create(string: captureId) + let sessionIdOffset = fbb.create(string: sessionId) + let params = RemoteShutter_CommandParameters.createCommandParameters( + &fbb, + captureFireAtCameraClockMs: fireAtCameraClockMillis, + captureAnchorMs: anchorMillis, + captureIdOffset: captureIdOffset, + captureSessionIdOffset: sessionIdOffset, + captureCameraIndex: Int32(cameraIndex)) + return buildCommand(&fbb, action: .scheduledcapture, parameters: params) + } +} + +extension RemoteCmd.ScheduledCaptureAck { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let echoOffset = fbb.create(string: captureId) + let errorOffset = (error as NSError?).map { fbb.create(string: $0.localizedDescription) } ?? Offset() + let resp = RemoteShutter_CameraStateResponse.createCameraStateResponse( + &fbb, + action: .scheduledcapture, + success: error == nil, + errorOffset: errorOffset, + captureIdEchoOffset: echoOffset) + return buildResponse(&fbb, action: .scheduledcapture, response: resp) + } +} + +extension RemoteCmd.ScheduledStartRecording { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let captureIdOffset = fbb.create(string: captureId) + let sessionIdOffset = fbb.create(string: sessionId) + let params = RemoteShutter_CommandParameters.createCommandParameters( + &fbb, + captureFireAtCameraClockMs: fireAtCameraClockMillis, + captureAnchorMs: anchorMillis, + captureIdOffset: captureIdOffset, + captureSessionIdOffset: sessionIdOffset, + captureCameraIndex: Int32(cameraIndex)) + return buildCommand(&fbb, action: .scheduledstartrecording, parameters: params) + } +} + +extension RemoteCmd.ScheduledStopRecording { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let captureIdOffset = fbb.create(string: captureId) + let sessionIdOffset = fbb.create(string: sessionId) + let params = RemoteShutter_CommandParameters.createCommandParameters( + &fbb, + captureFireAtCameraClockMs: fireAtCameraClockMillis, + captureAnchorMs: anchorMillis, + captureIdOffset: captureIdOffset, + captureSessionIdOffset: sessionIdOffset, + captureCameraIndex: Int32(cameraIndex)) + return buildCommand(&fbb, action: .scheduledstoprecording, parameters: params) + } +} + +extension RemoteCmd.RequestVideoResend { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let idOffset = fbb.create(string: captureId) + let params = RemoteShutter_CommandParameters.createCommandParameters(&fbb, captureIdOffset: idOffset) + return buildCommand(&fbb, action: .requestvideoresend, parameters: params) + } +} + +extension RemoteCmd.SetStreamProfile { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let params = RemoteShutter_CommandParameters.createCommandParameters( + &fbb, + streamMaxLongEdge: Int32(maxLongEdge), + streamBitrateKbps: Int32(bitrateKbps), + streamFps: Int32(fps)) + return buildCommand(&fbb, action: .setstreamprofile, parameters: params) + } +} + +extension RemoteCmd.ScheduledRecordingAck { + func toFlatBuffer() -> Data { + var fbb = FlatBufferBuilder() + let echoOffset = fbb.create(string: captureId) + let errorOffset = (error as NSError?).map { fbb.create(string: $0.localizedDescription) } ?? Offset() + let action: RemoteShutter_CommandAction = isStop ? .scheduledstoprecording : .scheduledstartrecording + let resp = RemoteShutter_CameraStateResponse.createCameraStateResponse( + &fbb, + action: action, + success: error == nil, + errorOffset: errorOffset, + captureIdEchoOffset: echoOffset) + return buildResponse(&fbb, action: action, response: resp) + } +} + extension RemoteCmd.ToggleFlash { func toFlatBuffer() -> Data { var fbb = FlatBufferBuilder() @@ -1100,6 +1232,42 @@ extension RemoteCmd { case .requestkeyframe: return RequestKeyframe(sender: nil) + case .clocksyncping: + return ClockSyncPing(t0Millis: params?.clockSyncT0Ms ?? 0) + + case .scheduledcapture: + return ScheduledCapture( + fireAtCameraClockMillis: params?.captureFireAtCameraClockMs ?? 0, + anchorMillis: params?.captureAnchorMs ?? 0, + captureId: params?.captureId ?? "", + sessionId: params?.captureSessionId ?? "", + cameraIndex: Int(params?.captureCameraIndex ?? 0)) + + case .scheduledstartrecording: + return ScheduledStartRecording( + fireAtCameraClockMillis: params?.captureFireAtCameraClockMs ?? 0, + anchorMillis: params?.captureAnchorMs ?? 0, + captureId: params?.captureId ?? "", + sessionId: params?.captureSessionId ?? "", + cameraIndex: Int(params?.captureCameraIndex ?? 0)) + + case .scheduledstoprecording: + return ScheduledStopRecording( + fireAtCameraClockMillis: params?.captureFireAtCameraClockMs ?? 0, + anchorMillis: params?.captureAnchorMs ?? 0, + captureId: params?.captureId ?? "", + sessionId: params?.captureSessionId ?? "", + cameraIndex: Int(params?.captureCameraIndex ?? 0)) + + case .setstreamprofile: + return SetStreamProfile( + maxLongEdge: Int(params?.streamMaxLongEdge ?? 0), + bitrateKbps: Int(params?.streamBitrateKbps ?? 0), + fps: Int(params?.streamFps ?? 0)) + + case .requestvideoresend: + return RequestVideoResend(captureId: params?.captureId ?? "") + case .toggleflash: return ToggleFlash() @@ -1160,6 +1328,19 @@ extension RemoteCmd { let nsError: Error? = errorStr.map { NSError(domain: "RemoteCmd", code: -1, userInfo: [NSLocalizedDescriptionKey: $0]) } switch resp.action { + case .clocksyncping: + return ClockSyncPong(echoT0Millis: resp.clockSyncEchoT0Ms, + cameraClockMillis: resp.clockSyncCameraClockMs) + + case .scheduledcapture: + return ScheduledCaptureAck(captureId: resp.captureIdEcho ?? "", error: nsError) + + case .scheduledstartrecording: + return ScheduledRecordingAck(captureId: resp.captureIdEcho ?? "", isStop: false, error: nsError) + + case .scheduledstoprecording: + return ScheduledRecordingAck(captureId: resp.captureIdEcho ?? "", isStop: true, error: nsError) + case .startrecording: let startTime: Date? = resp.recordingStartTime > 0 ? Date(timeIntervalSince1970: Double(resp.recordingStartTime) / 1000.0) : nil return StartRecordingVideoAck(sender: nil, recordingStartTime: startTime, error: nsError) @@ -1303,6 +1484,7 @@ extension RemoteCmd { activeDeviceID: activeDeviceID, supportsFocusPoint: caps?.supportsFocusPoint ?? false, supportsPreviewMode: caps?.supportsPreviewMode ?? false, + supportsMulticam: caps?.supportsMulticam ?? false, previewMode: state.map { fromFBPreviewMode($0.previewMode) } ?? .on, error: error ) diff --git a/RemoteCam/RemoteCmds.swift b/RemoteCam/RemoteCmds.swift index 4071071e..b8996d88 100644 --- a/RemoteCam/RemoteCmds.swift +++ b/RemoteCam/RemoteCmds.swift @@ -194,6 +194,163 @@ public class RemoteCmd: Message, @unchecked Sendable { } } + /// Director → camera: clock-offset probe. `t0Millis` is the director's + /// monotonic clock at send; the camera answers immediately with a + /// `ClockSyncPong` echoing it. Only sent to peers advertising + /// `supportsMulticam` (an old peer would decode it as Unknown and drop it). + public class ClockSyncPing: Message, @unchecked Sendable { + public let t0Millis: UInt64 + init(t0Millis: UInt64, sender: AnyObject? = nil) { + self.t0Millis = t0Millis + super.init(sender: sender) + } + } + + /// Camera → director: answer to `ClockSyncPing`. Echoes the director's + /// `t0Millis` (so the director can compute RTT against its own clock) and + /// carries the camera's monotonic clock at receipt — the pair the + /// director's `ClockOffsetEstimator` turns into an offset sample. + public class ClockSyncPong: Message, @unchecked Sendable { + public let echoT0Millis: UInt64 + public let cameraClockMillis: UInt64 + init(echoT0Millis: UInt64, cameraClockMillis: UInt64, sender: AnyObject? = nil) { + self.echoT0Millis = echoT0Millis + self.cameraClockMillis = cameraClockMillis + super.init(sender: sender) + } + } + + /// Director → camera: fire the shutter at `fireAtCameraClockMillis`, a + /// wall-clock instant already translated into this camera's own + /// `SyncClock` domain (the director applied the per-camera offset), so all + /// cameras expose together. `anchorMillis` is the same instant in the + /// director's clock — identical across every camera in the shot, so it is + /// the alignment key each clip is stamped with. Only sent to peers that + /// advertised `supportsMulticam`. + public class ScheduledCapture: Message, @unchecked Sendable { + public let fireAtCameraClockMillis: UInt64 + public let anchorMillis: UInt64 + public let captureId: String + public let sessionId: String + public let cameraIndex: Int + + public init(fireAtCameraClockMillis: UInt64, + anchorMillis: UInt64, + captureId: String, + sessionId: String, + cameraIndex: Int, + sender: AnyObject? = nil) { + self.fireAtCameraClockMillis = fireAtCameraClockMillis + self.anchorMillis = anchorMillis + self.captureId = captureId + self.sessionId = sessionId + self.cameraIndex = cameraIndex + super.init(sender: sender) + } + } + + /// Camera → director: the scheduled capture was accepted (or refused). Sent + /// immediately on receipt — before the shutter actually fires — so the + /// director can aggregate acks without waiting for N photos. `error` set = + /// the camera could not schedule it (wrong state, fire time long past). + public class ScheduledCaptureAck: Message, @unchecked Sendable { + public let captureId: String + public let error: Error? + + public init(captureId: String, error: Error? = nil, sender: AnyObject? = nil) { + self.captureId = captureId + self.error = error + super.init(sender: sender) + } + } + + /// Director → camera: begin recording at `fireAtCameraClockMillis` (this + /// camera's clock domain), so every camera rolls together. Same fields and + /// meaning as `ScheduledCapture`. Only sent to `supportsMulticam` peers. + public class ScheduledStartRecording: Message, @unchecked Sendable { + public let fireAtCameraClockMillis: UInt64 + public let anchorMillis: UInt64 + public let captureId: String + public let sessionId: String + public let cameraIndex: Int + + public init(fireAtCameraClockMillis: UInt64, anchorMillis: UInt64, + captureId: String, sessionId: String, cameraIndex: Int, + sender: AnyObject? = nil) { + self.fireAtCameraClockMillis = fireAtCameraClockMillis + self.anchorMillis = anchorMillis + self.captureId = captureId + self.sessionId = sessionId + self.cameraIndex = cameraIndex + super.init(sender: sender) + } + } + + /// Director → camera: stop recording at the fire instant, so clip lengths + /// line up across the rig. Reuses the same params (index/anchor unused). + public class ScheduledStopRecording: Message, @unchecked Sendable { + public let fireAtCameraClockMillis: UInt64 + public let anchorMillis: UInt64 + public let captureId: String + public let sessionId: String + public let cameraIndex: Int + + public init(fireAtCameraClockMillis: UInt64, anchorMillis: UInt64, + captureId: String, sessionId: String, cameraIndex: Int, + sender: AnyObject? = nil) { + self.fireAtCameraClockMillis = fireAtCameraClockMillis + self.anchorMillis = anchorMillis + self.captureId = captureId + self.sessionId = sessionId + self.cameraIndex = cameraIndex + super.init(sender: sender) + } + } + + /// Camera → director: the scheduled record start/stop was accepted (or + /// refused). Immediate, like `ScheduledCaptureAck`; `isStop` distinguishes + /// the two so the director's start/stop aggregation stay separate. + public class ScheduledRecordingAck: Message, @unchecked Sendable { + public let captureId: String + public let isStop: Bool + public let error: Error? + + public init(captureId: String, isStop: Bool, error: Error? = nil, + sender: AnyObject? = nil) { + self.captureId = captureId + self.isStop = isStop + self.error = error + super.init(sender: sender) + } + } + + /// Director → camera: reconfigure the live preview encoder for tiered + /// multicam previews (the focused lane full-size, the rest thumbnails). + /// Only sent to `supportsMulticam` peers. + public class SetStreamProfile: Message, @unchecked Sendable { + public let maxLongEdge: Int + public let bitrateKbps: Int + public let fps: Int + + public init(maxLongEdge: Int, bitrateKbps: Int, fps: Int, sender: AnyObject? = nil) { + self.maxLongEdge = maxLongEdge + self.bitrateKbps = bitrateKbps + self.fps = fps + super.init(sender: sender) + } + } + + /// Director → camera: re-send the clip for `captureId` — the auto-collect + /// retry after a failed transfer. The camera keeps its last multicam clip + /// until collected, so it can honor this. + public class RequestVideoResend: Message, @unchecked Sendable { + public let captureId: String + public init(captureId: String, sender: AnyObject? = nil) { + self.captureId = captureId + super.init(sender: sender) + } + } + public class OnFrame: Message, @unchecked Sendable { public let data: Data public let peerId: MCPeerID @@ -410,6 +567,10 @@ public class RemoteCmd: Message, @unchecked Sendable { /// `RemoteCmd.SetCameraPreviewMode`. The monitor's standby gate reads /// this so it never sends the command to a peer that would misread it. public let supportsPreviewMode: Bool + /// True when this peer's build can join a multicam director session + /// (scheduled capture, stream profiles). A director must not send + /// multicam commands to a peer that doesn't advertise this. + public let supportsMulticam: Bool /// The camera's current local-preview mode, so the monitor can reflect /// it from the first capabilities exchange. public let previewMode: CameraPreviewMode @@ -426,6 +587,7 @@ public class RemoteCmd: Message, @unchecked Sendable { activeDeviceID: String? = nil, supportsFocusPoint: Bool = false, supportsPreviewMode: Bool = false, + supportsMulticam: Bool = false, previewMode: CameraPreviewMode = .on, error: Error?) { self.frontCamera = frontCamera @@ -441,6 +603,7 @@ public class RemoteCmd: Message, @unchecked Sendable { self.activeDeviceID = activeDeviceID self.supportsFocusPoint = supportsFocusPoint self.supportsPreviewMode = supportsPreviewMode + self.supportsMulticam = supportsMulticam self.previewMode = previewMode self.error = error super.init(sender: nil) diff --git a/RemoteCam/RigQualityMenu.swift b/RemoteCam/RigQualityMenu.swift new file mode 100644 index 00000000..d42f25e0 --- /dev/null +++ b/RemoteCam/RigQualityMenu.swift @@ -0,0 +1,169 @@ +// +// RigQualityMenu.swift +// RemoteShutter +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import Foundation + +/// The rig-wide quality menu: "the shot belongs to the rig", so video/photo +/// quality is the **intersection** of what every connected camera can do. A +/// pure value type — the picker UI and the fan-out both read it, and it is +/// unit-tested without any transport. +/// +/// Built from each lane's current-camera `CameraInfo` (resolution/fps matrix, +/// HEIF, HDR). An option is offered only when *every* camera supports it; a +/// non-intersection option is still listed, greyed, naming the camera(s) that +/// block it. +struct RigQualityMenu: Equatable { + + /// One lane's contribution: a display name + the camera it is currently on. + struct Lane: Equatable { + let name: String + let info: RemoteCmd.CameraInfo + } + + /// The universal floor every iPhone camera supports — used when the + /// intersection is empty and as the Automatic fallback. + static let floor: (resolution: VideoResolution, frameRate: VideoFrameRate) = (.hd1080p, .fps30) + + private let lanes: [Lane] + + init(lanes: [Lane]) { + self.lanes = lanes + } + + // MARK: - Video + + /// Every (resolution, frame-rate) pair the whole rig supports. An empty rig + /// has no cameras to agree on, so no options (Automatic then uses the floor). + func videoOptions() -> [(resolution: VideoResolution, frameRate: VideoFrameRate)] { + guard !lanes.isEmpty else { return [] } + return allVideoPairs().filter { blockingLanes(resolution: $0.resolution, frameRate: $0.frameRate).isEmpty } + } + + /// Cameras that cannot do this (resolution, fps) — empty means it is in the + /// rig intersection. Used to grey and annotate a picker row. + func blockingLanes(resolution: VideoResolution, frameRate: VideoFrameRate) -> [String] { + lanes.filter { !laneSupports($0, resolution: resolution, frameRate: frameRate) }.map(\.name) + } + + /// Automatic = the best pair every camera can do: highest resolution, then + /// highest frame rate. Falls back to the 1080p30 floor when the rig shares + /// nothing (heterogeneous cameras) or there are no lanes. + func automaticVideo() -> (resolution: VideoResolution, frameRate: VideoFrameRate) { + let best = videoOptions().max { a, b in + if a.resolution.rawValue != b.resolution.rawValue { + return a.resolution.rawValue < b.resolution.rawValue + } + return a.frameRate.rawValue < b.frameRate.rawValue + } + return best ?? Self.floor + } + + // MARK: - Photo + + /// HEIF is a rig option only when every camera can encode it. + func supportsHEIF() -> Bool { + !lanes.isEmpty && lanes.allSatisfy { $0.info.supportsHEIF } + } + + /// HDR is a rig option only when every camera can do it. + func supportsHDR() -> Bool { + !lanes.isEmpty && lanes.allSatisfy { $0.info.supportsHDR } + } + + /// Cameras blocking HEIF / HDR, for greying the photo tiles. + func lanesBlockingHEIF() -> [String] { lanes.filter { !$0.info.supportsHEIF }.map(\.name) } + func lanesBlockingHDR() -> [String] { lanes.filter { !$0.info.supportsHDR }.map(\.name) } + + /// Automatic photo = HEIF if the whole rig can, else JPEG; HDR on if all can. + func automaticPhoto() -> (format: PhotoFormat, hdr: HDRMode) { + (supportsHEIF() ? .heif : .jpeg, supportsHDR() ? .on : .off) + } + + // MARK: - Late joiner + + /// Whether a lane can honor a currently-applied rig video setting. A late + /// joiner that can't is badged and offered a "re-match" (re-run Automatic) + /// rather than silently changing the running rig. + func laneCanMatch(_ lane: Lane, + resolution: VideoResolution, + frameRate: VideoFrameRate) -> Bool { + laneSupports(lane, resolution: resolution, frameRate: frameRate) + } + + // MARK: - Internals + + private func allVideoPairs() -> [(resolution: VideoResolution, frameRate: VideoFrameRate)] { + var pairs: [(VideoResolution, VideoFrameRate)] = [] + for resolution in [VideoResolution.uhd4k, .hd1080p] { + for frameRate in [VideoFrameRate.fps60, .fps30, .fps24] { + pairs.append((resolution, frameRate)) + } + } + return pairs.map { (resolution: $0.0, frameRate: $0.1) } + } + + private func laneSupports(_ lane: Lane, + resolution: VideoResolution, + frameRate: VideoFrameRate) -> Bool { + let matrix = lane.info.getResolutionFrameRates() + if let rates = matrix[resolution] { return rates.contains(frameRate) } + // A camera that never advertised the matrix (older build) is treated as + // the universal floor only. + return resolution == .hd1080p && frameRate == .fps30 + } + + // MARK: - View-facing picker rows + + /// Every listed video option (highest first), each flagged enabled (in the + /// rig intersection) or greyed with the cameras that block it. Manual + /// selection of any enabled row is first-class. + func videoPickerOptions() -> [RigVideoOption] { + allVideoPairs().map { pair in + let blockers = blockingLanes(resolution: pair.resolution, frameRate: pair.frameRate) + return RigVideoOption(resolution: pair.resolution, frameRate: pair.frameRate, + enabled: blockers.isEmpty, blockedBy: blockers) + } + } +} + +/// One row of the rig video-quality picker. +struct RigVideoOption: Equatable, Identifiable { + let resolution: VideoResolution + let frameRate: VideoFrameRate + let enabled: Bool + let blockedBy: [String] + + var id: String { "\(resolution.rawValue):\(frameRate.rawValue)" } + var label: String { "\(resolution.displayName)\(frameRate.displayName)" } +} + +/// The rig's active video quality, or nil for Automatic (best-in-intersection). +/// Typed so the tray matches the running quality by value, not by comparing +/// rendered label strings. +struct RigVideoSelection: Equatable { + let resolution: VideoResolution + let frameRate: VideoFrameRate + + var label: String { "\(resolution.displayName)\(frameRate.displayName)" } + func matches(_ option: RigVideoOption) -> Bool { + resolution == option.resolution && frameRate == option.frameRate + } +} + +/// The rig-settings snapshot the director hands the tray UI. +struct RigSettingsSnapshot: Equatable { + var timerSeconds: Int = 0 + var countdown: Int? + var activeVideo: RigVideoSelection? + var videoOptions: [RigVideoOption] = [] + var heifAvailable: Bool = false + var hdrAvailable: Bool = false + var heifBlockedBy: [String] = [] + var hdrBlockedBy: [String] = [] + var activePhotoFormat: PhotoFormat? + var activeHDR: HDRMode? +} diff --git a/RemoteCam/RolePickerView.swift b/RemoteCam/RolePickerView.swift index 01396218..8e8b3c27 100644 --- a/RemoteCam/RolePickerView.swift +++ b/RemoteCam/RolePickerView.swift @@ -34,8 +34,14 @@ struct RolePickerView: View { Button(action: onRemote) { glassPanel( icon: "antenna.radiowaves.left.and.right", - title: NSLocalizedString("Remote", comment: ""), - subtitle: NSLocalizedString("Control the shutter", comment: ""), + // Multicam reframes this role as the "Director" of one or + // more cameras; single-cam keeps the "Remote" wording. + title: FeatureFlags.ENABLE_MULTICAM + ? NSLocalizedString("Director", comment: "") + : NSLocalizedString("Remote", comment: ""), + subtitle: FeatureFlags.ENABLE_MULTICAM + ? NSLocalizedString("Control one or more iPhone cameras", comment: "") + : NSLocalizedString("Control the shutter", comment: ""), tint: AppTheme.secondary ) } diff --git a/RemoteCam/ScannerLobby.swift b/RemoteCam/ScannerLobby.swift index 4a74c0b3..74b675c8 100644 --- a/RemoteCam/ScannerLobby.swift +++ b/RemoteCam/ScannerLobby.swift @@ -26,6 +26,15 @@ protocol ScannerLobby: AnyObject, Sendable { /// Navigate to the role picker after a peer connects. func goToRole() + /// Multicam "Connect (N)": the set of connected cameras changed (a selected + /// camera established). Default no-op — only the production scanner + /// implements it, and only when `ENABLE_MULTICAM`. + func didCollectMulticamCameras(_ peers: [MCPeerID]) + + /// Multicam "Connect (N)": a selected camera's invite failed for good, so + /// the scanner can mark that row and proceed with the ones that connected. + func didFailMulticamCamera(_ peer: MCPeerID) + /// Pop navigation back to the scanner screen (called when scanning restarts). func returnToLobby() @@ -33,6 +42,11 @@ protocol ScannerLobby: AnyObject, Sendable { func presentScanningError() } +extension ScannerLobby { + func didCollectMulticamCameras(_ peers: [MCPeerID]) {} + func didFailMulticamCamera(_ peer: MCPeerID) {} +} + /// Binds a `ScannerLobby` to `RemoteCamSession` — the protocol-typed /// counterpart of Theater's `SetViewCtrl` (whose generic parameter requires /// a concrete class). diff --git a/RemoteCam/SessionCoordinator.swift b/RemoteCam/SessionCoordinator.swift index 58e8ff0f..ff311a3d 100644 --- a/RemoteCam/SessionCoordinator.swift +++ b/RemoteCam/SessionCoordinator.swift @@ -126,6 +126,27 @@ final class DeferredPopAndScan: Message, @unchecked Sendable { /// Incompatibility detected by the transport (routed through the inbox — the /// old code mutated state directly on the delegate thread). final class IncompatibilityDetected: Message, @unchecked Sendable {} +/// Camera side: a scheduled multicam capture's fire time has arrived. Enqueued +/// by the off-actor delay task so the shutter is pulled by the message pump, +/// in order with every other state transition — never racing it. +final class FireScheduledCapture: Message, @unchecked Sendable { + let metadata: CaptureSyncMetadata + init(metadata: CaptureSyncMetadata) { + self.metadata = metadata + super.init(sender: nil) + } +} +/// Camera side: a scheduled multicam recording's start instant has arrived +/// (same off-actor-delay → pump-message pattern as `FireScheduledCapture`). +final class FireScheduledRecordingStart: Message, @unchecked Sendable { + let metadata: CaptureSyncMetadata + init(metadata: CaptureSyncMetadata) { + self.metadata = metadata + super.init(sender: nil) + } +} +/// Camera side: a scheduled multicam recording's stop instant has arrived. +final class FireScheduledRecordingStop: Message, @unchecked Sendable {} /// Retry tick for the capabilities ladder. final class RetryCapabilities: Message, @unchecked Sendable { let attempt: Int @@ -222,6 +243,61 @@ public actor SessionCoordinator { /// camera peer speaks VP9, which gates sending `RemoteCmd.RequestKeyframe`. private var monitorReceivedVP9Frame = false + /// Camera side: the sync metadata for a scheduled multicam capture that is + /// about to fire. Set when `FireScheduledCapture` triggers the shutter and + /// consumed by the next `OnPicture`, so exactly that photo is stamped and + /// saved under its `RS___cam` filename. Nil for ordinary + /// single-camera captures, which save exactly as before. + private var pendingSyncMetadata: CaptureSyncMetadata? + /// The sync metadata for the multicam clip currently being recorded, used + /// to name its auto-collect transfer (`RS_…cam.mov`) and to stagger the + /// send by camera index. Kept until the next recording so a director retry + /// (`RequestVideoResend`) can re-send from `lastMulticamClipURL`. + private var pendingVideoSyncMetadata: CaptureSyncMetadata? + private var lastMulticamClipURL: URL? + /// Per-camera transfer stagger, so N×4K clips don't all hit the link at once. + private var multicamTransferStaggerSeconds: Double = 2 + /// Test seam. + func setMulticamTransferStaggerForTesting(_ s: Double) { multicamTransferStaggerSeconds = s } + + /// How far in the past a scheduled fire time may be before the camera + /// refuses it (clock error and one retransmit can nudge it slightly late; + /// beyond this it is stale and would fire out of sync). + private let scheduledCaptureMaxLatenessMillis: Int64 = 1000 + + /// Camera side: who is driving this session. `.solo` is an ordinary 1:1 + /// session (or none yet); `.director` latches the first time any scheduled + /// multicam command arrives, and is cleared when the session ends + /// (`popToScanning`). It gates the ONE behavior change on the camera — keep + /// recording through a director drop instead of stopping — so a + /// single-camera session is never affected. A reconnect goes through + /// `.reconnecting`, not teardown, so the latch survives a transient drop. + private enum CameraDriver: Equatable { + case solo + case director + } + private var cameraDriver: CameraDriver = .solo + + /// Test support. + func inMulticamSessionForTesting() -> Bool { cameraDriver == .director } + + /// Multicam director "collecting" mode. Off by default and only ever set + /// by the scanner when `ENABLE_MULTICAM` and the monitor role, so every + /// non-multicam path is byte-identical. While set, a peer connecting in + /// `.scanning` is accumulated (the machine stays scanning, keeps browsing) + /// instead of transitioning to `.connected` and auto-advancing to the 1:1 + /// monitor; the scanner reads `multicamCollectedPeers` on "Start". + private var multicamCollecting = false + private var multicamCollectedPeers: [MCPeerID] = [] + /// Per-peer invite attempts during a multicam "Connect (N)" — the scanner + /// invites the whole selected set at once, so retry/timeout is tracked per + /// camera rather than through the single-peer `link`. + private var multicamInviteAttempts: [MCPeerID: Int] = [:] + + /// Test support. + func multicamCollectingForTesting() -> Bool { multicamCollecting } + func multicamCollectedPeersForTesting() -> [MCPeerID] { multicamCollectedPeers } + /// Test support. func monitorReceivedVP9FrameForTesting() -> Bool { monitorReceivedVP9Frame } @@ -295,6 +371,47 @@ public actor SessionCoordinator { var connectedPeers: [MCPeerID] { multipeerService?.connectedPeers ?? [] } + /// How many cameras the collecting scanner has connected. The scanner + /// reads this on "Start" to choose the single-camera vs director path. + func multicamConnectedCount() -> Int { multicamCollecting ? connectedPeers.count : 0 } + + /// Hand the live transport (and the ≥2 cameras it is connected to) to a + /// `MulticamController`. Detaches this coordinator from the transport — + /// nils its references without stopping the session — so the multicam + /// controller becomes the sole delegate and this coordinator's `stop()` + /// (on scanner teardown) cannot kill a session the director is using. + /// Returns nil unless collecting with two or more cameras (the single + /// camera case stays on the classic monitor — see below). + func detachTransportForMulticam() -> (transport: any MultipeerServiceProtocol, peers: [MCPeerID])? { + guard multicamCollecting, let transport = multipeerService else { return nil } + let peers = transport.connectedPeers + guard peers.count >= 2 else { return nil } + multipeerService = nil + transportShared.value = nil + multicamCollecting = false + multicamCollectedPeers = [] + multicamInviteAttempts = [:] + return (transport, peers) + } + + /// The single-camera exit from collecting: promote the one connected peer + /// to a normal `.connected` session so the classic `MonitorViewController` + /// path runs exactly as it does without the flag. Returns false (leaving + /// the scanner as-is) unless collecting with exactly one camera. + func promoteSingleCollectedToConnected() async -> Bool { + guard multicamCollecting, connectedPeers.count == 1, + let peer = connectedPeers.first, let liveLobby = lobby?.value else { return false } + multicamCollecting = false + multicamCollectedPeers = [] + multicamInviteAttempts = [:] + link = .linked(peer) + OperationQueue.main.addOperation { + liveLobby.scannerViewModel.connectedToPeer() + } + await transition(to: .connected) + return true + } + private func unableToProcessError(_ msg: Message) async -> NSError { let deviceName = await MainActor.run { UIDevice.current.name } return NSError( @@ -302,23 +419,27 @@ public actor SessionCoordinator { } @discardableResult - func sendMessage(_ msg: Message, mode: MCSessionSendDataMode = .reliable) -> Bool { + func sendMessage(_ msg: Message, to peers: [MCPeerID]? = nil, + mode: MCSessionSendDataMode = .reliable) -> Bool { guard let multipeerService else { // Watch Remote mode never starts a multipeer session. return false } - return multipeerService.send(msg, to: connectedPeers, mode: mode) + return multipeerService.send(msg, to: peers ?? connectedPeers, mode: mode) } /// Send, or pop to scanning with a connection-error alert on failure — - /// the old `sendCommandOrGoToScanning`. - func sendOrGoToScanning(_ msg: Message, mode: MCSessionSendDataMode = .reliable) async { + /// the old `sendCommandOrGoToScanning`. `peers` nil = all connected + /// (identical for this 1:1 coordinator; the param exists so per-peer + /// sends like frame acks address only their source). + func sendOrGoToScanning(_ msg: Message, to peers: [MCPeerID]? = nil, + mode: MCSessionSendDataMode = .reliable) async { guard multipeerService != nil else { // Watch Remote mode: there is no peer and no scanning state to fall back to. debugLog("sendOrGoToScanning: no multipeer session, dropping \(type(of: msg))") return } - if !sendMessage(msg, mode: mode) { + if !sendMessage(msg, to: peers, mode: mode) { await popToScanning() let presenter = alertPresenter OperationQueue.main.addOperation { @@ -542,6 +663,14 @@ public actor SessionCoordinator { await sendOrGoToScanning(RemoteCmd.RequestFrame(sender: nil)) } + /// Ack the camera that sent this frame so only its credit window + /// advances. With one connected peer this is identical to the broadcast + /// form; with several cameras a broadcast ack would let every camera + /// send on one camera's consumed frame. + private func requestFrame(acking frame: RemoteCmd.OnFrame) async { + await sendOrGoToScanning(RemoteCmd.RequestFrame(sender: nil), to: [frame.peerId]) + } + /// Pop to scanning (stops at the lobby floor like the old machine) and /// restart discovery via `.scanning`'s entry behavior. func popToScanning() async { @@ -549,6 +678,12 @@ public actor SessionCoordinator { peerSupportsFocusPoint = false peerSupportsPreviewMode = false monitorReceivedVP9Frame = false + // The session is being torn down for good (deliberate leave, EndSession, + // or a dead link) — a fresh session starts single-cam until a director + // says otherwise. A reconnect goes through `.reconnecting`, not here, so + // the latch survives a transient director drop. + cameraDriver = .solo + pendingSyncMetadata = nil switch state { case .scanning: // Already there — re-entering would restart discovery and reset @@ -641,6 +776,14 @@ public actor SessionCoordinator { startScanning(lobby: liveLobby) case let connect as ConnectToDevice: + if multicamCollecting { + // "Connect (N)" fires an invite per selected camera; each is + // tracked independently (retry/timeout per peer) rather than + // through the single-peer `link`. + multicamInviteAttempts[connect.peer] = 1 + multipeerService?.invitePeer(connect.peer, timeout: inviteTimeout) + break + } link = .inviting(connect.peer, attempt: 1) multipeerService?.invitePeer(connect.peer, timeout: inviteTimeout) OperationQueue.main.addOperation { @@ -648,6 +791,22 @@ public actor SessionCoordinator { } case let disconnected as DisconnectPeer: + if multicamCollecting, let peer = disconnected.peer, + let attempt = multicamInviteAttempts[peer] { + // A selected camera's invite dropped/timed out — retry once, + // then report it failed so the scanner can proceed with the + // cameras that did connect. + if attempt < maxConnectAttempts { + multicamInviteAttempts[peer] = attempt + 1 + multipeerService?.invitePeer(peer, timeout: inviteTimeout) + } else { + multicamInviteAttempts[peer] = nil + OperationQueue.main.addOperation { + liveLobby.didFailMulticamCamera(peer) + } + } + break + } guard let invited = link.invited, invited.peer == disconnected.peer else { startScanning(lobby: liveLobby) break @@ -679,6 +838,21 @@ public actor SessionCoordinator { break case let connected as OnConnectToDevice: + if multicamCollecting { + // A selected camera established. Accumulate and stay scanning + // (no transition to `.connected`, which would stop browsing); + // report the full connected set so the scanner advances the + // row to "connected" and, once every invite settles, hands off. + multicamInviteAttempts[connected.peer] = nil + if !multicamCollectedPeers.contains(connected.peer) { + multicamCollectedPeers.append(connected.peer) + } + let peers = connectedPeers + OperationQueue.main.addOperation { + liveLobby.didCollectMulticamCameras(peers) + } + break + } link = .linked(connected.peer) OperationQueue.main.addOperation { liveLobby.scannerViewModel.connectedToPeer() @@ -891,6 +1065,42 @@ public actor SessionCoordinator { await showCameraAlert(NSLocalizedString("Taking picture", comment: "")) await transition(to: .cameraTakingPic(sendMediaToPeer: pic.sendMediaToPeer, generation: generation)) + case let scheduled as RemoteCmd.ScheduledCapture: + await handleScheduledCapture(scheduled) + + case let scheduled as RemoteCmd.ScheduledStartRecording: + await handleScheduledStartRecording(scheduled) + + case let profile as RemoteCmd.SetStreamProfile: + ctrl.applyStreamProfile(StreamProfile( + maxLongEdge: CGFloat(profile.maxLongEdge), + bitrateKbps: UInt32(max(0, profile.bitrateKbps)), + fps: UInt32(max(0, profile.fps)))) + + case let fire as FireScheduledRecordingStart: + // The start instant arrived. Stamp the recording with its sync + // metadata (for the QuickTime keys + the RS_ filename), then roll. + cameraDriver = .director + pendingVideoSyncMetadata = fire.metadata + ctrl.setVideoSyncMetadata(fire.metadata) + ctrl.currentCameraMode = .Video + ctrl.updateCameraStatus() + ctrl.startRecordingVideo() + await transition(to: .cameraRecordingVideo) + + case let fire as FireScheduledCapture: + // The fire instant arrived (enqueued by the off-actor delay task). + // Pull the shutter through the normal photo path: save locally + // stamped, AND return the (stamped) still to the director so it + // auto-collects every angle. + pendingSyncMetadata = fire.metadata + ctrl.currentCameraMode = .Photo + ctrl.updateCameraStatus() + ctrl.takePicture(true) + let generation = scheduleTimeout(.cameraTakingPic) + await showCameraAlert(NSLocalizedString("Taking picture", comment: "")) + await transition(to: .cameraTakingPic(sendMediaToPeer: true, generation: generation)) + case is RemoteCmd.ToggleCamera: do { _ = try await ctrl.toggleCamera() @@ -1093,6 +1303,84 @@ public actor SessionCoordinator { sendMessage(RemoteCmd.RequestKeyframe(sender: nil), mode: .reliable) } + /// Camera side of a synced multicam shot. Acks (or nacks) immediately so + /// the director can aggregate without waiting for the photo, then schedules + /// the shutter for the fire instant. The delay runs off the actor and only + /// enqueues `FireScheduledCapture`, so the capture is pulled by the message + /// pump in order — never racing a state transition. + private func handleScheduledCapture(_ scheduled: RemoteCmd.ScheduledCapture) async { + cameraDriver = .director + let now = SyncClock.nowMillis() + let lateness = Int64(now) - Int64(scheduled.fireAtCameraClockMillis) + guard lateness <= scheduledCaptureMaxLatenessMillis else { + await sendOrGoToScanning(RemoteCmd.ScheduledCaptureAck( + captureId: scheduled.captureId, + error: NSError(domain: "Scheduled capture fire time already passed", + code: 0, userInfo: nil))) + return + } + await sendOrGoToScanning(RemoteCmd.ScheduledCaptureAck(captureId: scheduled.captureId)) + + let metadata = CaptureSyncMetadata( + sessionID: scheduled.sessionId, + captureID: scheduled.captureId, + cameraIndex: scheduled.cameraIndex, + anchorMillis: scheduled.anchorMillis, + // The camera does not compute its own clock offset — the director + // did, when it translated the fire time into this clock. The + // shared `anchorMillis` is the alignment key; these are diagnostics. + clockOffsetMillis: 0, + roundTripMillis: 0) + + let delayMillis = max(0, -lateness) + Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(delayMillis) * 1_000_000) + self?.tell(FireScheduledCapture(metadata: metadata)) + } + } + + /// Camera side of a synced multicam recording start. Same ack-now, + /// schedule-off-the-actor pattern as `handleScheduledCapture`; the fire + /// message rolls the recording through the normal pipeline. + private func handleScheduledStartRecording(_ scheduled: RemoteCmd.ScheduledStartRecording) async { + cameraDriver = .director + let lateness = Int64(SyncClock.nowMillis()) - Int64(scheduled.fireAtCameraClockMillis) + guard lateness <= scheduledCaptureMaxLatenessMillis else { + await sendOrGoToScanning(RemoteCmd.ScheduledRecordingAck( + captureId: scheduled.captureId, isStop: false, + error: NSError(domain: "Scheduled recording start already passed", + code: 0, userInfo: nil))) + return + } + await sendOrGoToScanning(RemoteCmd.ScheduledRecordingAck( + captureId: scheduled.captureId, isStop: false)) + + let metadata = CaptureSyncMetadata( + sessionID: scheduled.sessionId, captureID: scheduled.captureId, + cameraIndex: scheduled.cameraIndex, anchorMillis: scheduled.anchorMillis, + clockOffsetMillis: 0, roundTripMillis: 0) + let delayMillis = max(0, -lateness) + Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(delayMillis) * 1_000_000) + self?.tell(FireScheduledRecordingStart(metadata: metadata)) + } + } + + /// Camera side of a synced recording stop — acks now, fires the stop at the + /// instant so every camera's clip ends together. + private func handleScheduledStopRecording(_ scheduled: RemoteCmd.ScheduledStopRecording) async { + let lateness = Int64(SyncClock.nowMillis()) - Int64(scheduled.fireAtCameraClockMillis) + // A late stop still needs to fire (a clip left rolling is worse than a + // slightly-long clip), so we never nack it — just clamp the delay. + await sendOrGoToScanning(RemoteCmd.ScheduledRecordingAck( + captureId: scheduled.captureId, isStop: true)) + let delayMillis = max(0, -lateness) + Task { [weak self] in + try? await Task.sleep(nanoseconds: UInt64(delayMillis) * 1_000_000) + self?.tell(FireScheduledRecordingStop()) + } + } + private func inCameraTakingPic(_ msg: Message, sendMediaToPeer: Bool, generation: Int) async { switch msg { case is RemoteCmd.RequestKeyframe: @@ -1103,6 +1391,7 @@ public actor SessionCoordinator { case let timeout as UICmd.StateTimeout: guard timeout.stateName == .cameraTakingPic && timeout.generation == generation else { break } + pendingSyncMetadata = nil await dismissCameraAlert() let sent = sendMessage(RemoteCmd.TakePicResp( sender: nil, @@ -1114,9 +1403,17 @@ public actor SessionCoordinator { } case let picture as UICmd.OnPicture: - if let pic = picture.pic { - photoLibrarySaver(pic) - } + // A scheduled multicam capture stamps the still with its sync + // metadata once; that same stamped image is saved locally under the + // shared RS___cam name AND returned to the director so + // it collects every angle. An ordinary capture saves as before. + let metadata = pendingSyncMetadata + let stampedPic = picture.pic.map { pic in metadata.map { $0.stamped(pic) } ?? pic } + if let pic = stampedPic { + if let metadata { saveSyncedPictureToLibrary(pic, metadata: metadata) } + else { photoLibrarySaver(pic) } + } + pendingSyncMetadata = nil await dismissCameraAlert() guard sendMessage(RemoteCmd.TakePicAck(sender: nil)) else { await popToScanning() @@ -1124,7 +1421,7 @@ public actor SessionCoordinator { } let resp = RemoteCmd.TakePicResp( sender: nil, - pic: sendMediaToPeer ? picture.pic : nil, + pic: sendMediaToPeer ? stampedPic : nil, error: picture.error) guard sendMessage(resp) else { await popToScanning() @@ -1190,10 +1487,39 @@ public actor SessionCoordinator { await sendOrGoToScanning(RemoteCmd.StopRecordingVideoAck(sender: nil), mode: .reliable) await transition(to: .cameraTransmittingVideo) + case let scheduled as RemoteCmd.ScheduledStopRecording: + await handleScheduledStopRecording(scheduled) + + case let profile as RemoteCmd.SetStreamProfile: + // The preview keeps streaming while recording, so a focus change + // can retune this lane here too. + ctrl.applyStreamProfile(StreamProfile( + maxLongEdge: CGFloat(profile.maxLongEdge), + bitrateKbps: UInt32(max(0, profile.bitrateKbps)), + fps: UInt32(max(0, profile.fps)))) + + case is FireScheduledRecordingStop: + // The scheduled stop instant arrived. Save locally AND push the clip + // to the director so it auto-collects every angle (the existing + // resource-transfer path; QuickTime sync metadata rides in the file). + ctrl.stopRecordingVideo(true) + await transition(to: .cameraTransmittingVideo) + case let disconnected as DisconnectPeer: if let lost = disconnected.peer, lost == peer, connectedPeers.isEmpty { - ctrl.stopRecordingVideo(false) - await loseSessionPeer(lost) + if cameraDriver == .director { + // Resilient camera: the director dropped mid-recording, but + // the OTHER cameras in the rig keep rolling — so must this + // one. Keep recording (do NOT stop), and enter the normal + // reconnect path; the clip is saved when the scheduled stop + // finally fires, or when the user stops on-device. This is + // the ONE behavior gated on `cameraDriver == .director`; a + // single-camera session falls through to the stop below. + await loseSessionPeer(lost) + } else { + ctrl.stopRecordingVideo(false) + await loseSessionPeer(lost) + } } case is UICmd.ScannerDidAppear: @@ -1390,6 +1716,19 @@ public actor SessionCoordinator { case is UICmd.AppForegrounded: await rearmAfterForeground() + case let collect as UICmd.SetMulticamCollecting: + multicamCollecting = collect.on + + case is RemoteCmd.RequestVideoResend: + // Auto-collect retry: the director's transfer failed. Re-send the + // last multicam clip, which we kept for exactly this. + if let url = lastMulticamClipURL, FileManager.default.fileExists(atPath: url.path), + let name = pendingVideoSyncMetadata?.videoFilename() { + for director in connectedPeers { + await sendVideoResourceNow(url: url, name: name, to: director) + } + } + case is UICmd.PeerTrafficObserved: // Arrives with every inbound message; only a wait cares (see // `inReconnecting`), and it is swallowed here so the rest of the @@ -1415,6 +1754,17 @@ public actor SessionCoordinator { case is RemoteCmd.TakePic: await sendOrGoToScanning(RemoteCmd.TakePicResp(sender: nil, error: unableToProcessError(msg))) + case let scheduled as RemoteCmd.ScheduledCapture: + // Not in a state that can take a picture — nack so the director's + // ack aggregation completes instead of timing out. + await sendOrGoToScanning(RemoteCmd.ScheduledCaptureAck( + captureId: scheduled.captureId, error: unableToProcessError(msg))) + case let scheduled as RemoteCmd.ScheduledStartRecording: + await sendOrGoToScanning(RemoteCmd.ScheduledRecordingAck( + captureId: scheduled.captureId, isStop: false, error: unableToProcessError(msg))) + case let scheduled as RemoteCmd.ScheduledStopRecording: + await sendOrGoToScanning(RemoteCmd.ScheduledRecordingAck( + captureId: scheduled.captureId, isStop: true, error: unableToProcessError(msg))) case is RemoteCmd.ToggleCamera: await sendOrGoToScanning(RemoteCmd.ToggleCameraResp(cameraCapabilities: nil, error: unableToProcessError(msg))) case is RemoteCmd.SelectCameraDevice: @@ -1521,26 +1871,43 @@ public actor SessionCoordinator { return } - let resourceName = "video_\(UUID().uuidString).mov" + // Multicam auto-collect: name the transfer with the shared RS_ filename + // so the director saves it under the group, remember the clip for a + // possible retry, and stagger the send by camera index so N clips don't + // saturate the link at once. + let multicamMeta = pendingVideoSyncMetadata + let resourceName = multicamMeta?.videoFilename() ?? "video_\(UUID().uuidString).mov" + if multicamMeta != nil { lastMulticamClipURL = sendVideo.videoURL } + let staggerDelay = multicamMeta.map { Double($0.cameraIndex - 1) * multicamTransferStaggerSeconds } ?? 0 tell(UICmd.VideoResourceTransferStarted(totalBytes: fileSize, resourceName: resourceName, sender: nil)) + let url = sendVideo.videoURL for peer in peers { - let sendProgress = multipeerService.sendResource( - at: sendVideo.videoURL, - withName: resourceName, - toPeer: peer - ) { [weak self] error in - if let error { - self?.tell(UICmd.VideoResourceTransferFailed(error: error, resourceName: resourceName, sender: nil)) - } else { - self?.tell(UICmd.VideoResourceTransferCompleted(resourceName: resourceName, success: true, sender: nil)) + if staggerDelay > 0 { + let ns = UInt64(staggerDelay * 1_000_000_000) + Task { [weak self] in + try? await Task.sleep(nanoseconds: ns) + await self?.sendVideoResourceNow(url: url, name: resourceName, to: peer) } + } else { + await sendVideoResourceNow(url: url, name: resourceName, to: peer) } + } + } - if let progress = sendProgress { - trackSendProgress(progress, resourceName: resourceName, into: multipeerService) + /// The actual staggered send of one clip to one peer. + private func sendVideoResourceNow(url: URL, name: String, to peer: MCPeerID) async { + guard let multipeerService else { return } + let sendProgress = multipeerService.sendResource(at: url, withName: name, toPeer: peer) { [weak self] error in + if let error { + self?.tell(UICmd.VideoResourceTransferFailed(error: error, resourceName: name, sender: nil)) + } else { + self?.tell(UICmd.VideoResourceTransferCompleted(resourceName: name, success: true, sender: nil)) } } + if let progress = sendProgress { + trackSendProgress(progress, resourceName: name, into: multipeerService) + } } private nonisolated func trackSendProgress(_ progress: Progress, @@ -1590,7 +1957,7 @@ public actor SessionCoordinator { case let frame as RemoteCmd.OnFrame: noteMonitorFrame(frame) monitor?.show(frame: frame) - await requestFrame() + await requestFrame(acking: frame) case is UICmd.StreamStalled: await requestFrame() @@ -1893,7 +2260,7 @@ public actor SessionCoordinator { case let frame as RemoteCmd.OnFrame: noteMonitorFrame(frame) monitor?.show(frame: frame) - await requestFrame() + await requestFrame(acking: frame) case is UICmd.StreamStalled: await requestFrame() @@ -2354,6 +2721,36 @@ public actor SessionCoordinator { } } + /// Save a synced multicam still: the image is stamped with the shot's sync + /// fields (EXIF) and lands under the shared `RS___cam` name, + /// so any editor can group and align the angles. Full-res stays on this + /// camera — the director does not collect stills in v1. + private nonisolated func saveSyncedPictureToLibrary(_ data: Data, + metadata: CaptureSyncMetadata) { + let stamped = metadata.stamped(data) + // HEIC magic: bytes 4..12 are "ftyp" then a heic/heix/mif1 brand. + let isHEIC = stamped.count > 12 + && stamped[4] == 0x66 && stamped[5] == 0x74 + && stamped[6] == 0x79 && stamped[7] == 0x70 + PHPhotoLibrary.requestAuthorization { status in + guard status == .authorized else { + DispatchQueue.main.async { + showPhotosAccessDeniedModal(for: .photo) + } + return + } + PHPhotoLibrary.shared().performChanges({ + let options = PHAssetResourceCreationOptions() + options.originalFilename = metadata.photoFilename(isHEIC: isHEIC) + PHAssetCreationRequest.forAsset() + .addResource(with: .photo, data: stamped, options: options) + }) { (success: Bool, _: Error?) in + print(success ? "Saved synced photo \(metadata.filenamePrefix)" + : "Failed to save synced photo!") + } + } + } + /// Monitor-side picture save (with the App Store review prompt). private nonisolated func savePictureOnMonitor(_ imageData: Data) { PHPhotoLibrary.requestAuthorization { status in @@ -2426,7 +2823,22 @@ public actor SessionCoordinator { extension SessionCoordinator: MultipeerServiceDelegate { - public nonisolated func didReceiveMessage(_ message: Message) { + public nonisolated func didReceiveMessage(_ message: Message, from peer: MCPeerID) { + if let ping = message as? RemoteCmd.ClockSyncPing { + // Answered here, off the actor inbox: the sample's quality is the + // camera clock *at receipt*, and queuing behind state-machine work + // would smear it (same pacing argument as didReceiveFrameRequest). + // Stateless, so no isolation is needed. + _ = transportShared.value?.send( + RemoteCmd.ClockSyncPong(echoT0Millis: ping.t0Millis, + cameraClockMillis: SyncClock.nowMillis()), + to: [peer], mode: .reliable) + tell(UICmd.PeerTrafficObserved()) + return + } + // `peer` is otherwise deliberately unused: this coordinator links + // exactly one peer, so every message's source is unambiguous. The + // multicam director's controller is the consumer that routes by it. tell(UICmd.PeerTrafficObserved()) tell(message) } @@ -2492,7 +2904,9 @@ extension SessionCoordinator: MultipeerServiceDelegate { tell(UICmd.BrowserFailed(error: error)) } - public nonisolated func didStartReceivingResource(name resourceName: String, progress: Progress) { + public nonisolated func didStartReceivingResource(name resourceName: String, from peer: MCPeerID, progress: Progress) { + // `peer` unused: this 1:1 coordinator has one source. The multicam + // director routes by it. debugLog("📥 DEBUG: Started receiving resource: \(resourceName)") guard resourceName.hasPrefix("video_") else { return } @@ -2543,7 +2957,7 @@ extension SessionCoordinator: MultipeerServiceDelegate { .store(in: &service.progressCancellables) } - public nonisolated func didFinishReceivingResource(name resourceName: String, at localURL: URL?, error: Error?) { + public nonisolated func didFinishReceivingResource(name resourceName: String, from peer: MCPeerID, at localURL: URL?, error: Error?) { debugLog("📥 DEBUG: Finished receiving resource: \(resourceName)") if let error { diff --git a/RemoteCam/SettingsView.swift b/RemoteCam/SettingsView.swift index b2c1acd6..090b1735 100644 --- a/RemoteCam/SettingsView.swift +++ b/RemoteCam/SettingsView.swift @@ -57,7 +57,12 @@ struct SettingsView: View { } header: { Text(NSLocalizedString("Pro — Unlock Everything", comment: "")) } footer: { - Text(NSLocalizedString("Pro unlocks every feature, including future ones.", comment: "")) + VStack(alignment: .leading, spacing: 4) { + Text(NSLocalizedString("Pro unlocks every feature, including future ones.", comment: "")) + if FeatureFlags.ENABLE_MULTICAM { + Text(NSLocalizedString("Direct up to 4 cameras at once", comment: "Pro multicam feature")) + } + } } // À la carte: individual features for a one-time purchase. diff --git a/RemoteCam/StoreManager.swift b/RemoteCam/StoreManager.swift index 5fece1fc..1181ae1f 100644 --- a/RemoteCam/StoreManager.swift +++ b/RemoteCam/StoreManager.swift @@ -92,6 +92,13 @@ final class StoreManager: ObservableObject { hasFullAccess() || UserDefaults.standard.bool(forKey: PurchaseKey.tapToFocus) } + /// How many cameras a multicam director may connect: a free 2-camera + /// teaser, or up to 4 with full access. The director's entitlement is what + /// counts (matching every other gate — checked locally on this device). + func maxCameras() -> Int { + hasFullAccess() ? 4 : 2 + } + // MARK: - Init private var updateListenerTask: Task? diff --git a/RemoteCam/StreamingConfig.swift b/RemoteCam/StreamingConfig.swift index be7d333d..be0f655f 100644 --- a/RemoteCam/StreamingConfig.swift +++ b/RemoteCam/StreamingConfig.swift @@ -150,3 +150,20 @@ struct StreamingConfig { static let `default` = StreamingConfig() } + +/// A live-adjustable preview profile for one multicam lane. The director sends +/// the focused camera the full profile and the others a smaller thumbnail +/// profile, so N previews fit the aggregate bandwidth + decode budget. Only the +/// three levers that matter for that trade-off are here; the encoders keep +/// their other tuning (quantizers, keyframe interval) from `StreamingConfig`. +struct StreamProfile: Equatable { + var maxLongEdge: CGFloat + var bitrateKbps: UInt32 + var fps: UInt32 + + /// The focused lane: today's full peer preview (unchanged from 1:1). + static let focused = StreamProfile(maxLongEdge: 1200, bitrateKbps: 1200, fps: 30) + /// An unfocused strip/grid tile: smaller and cheaper to decode, so four of + /// them stay within ~2.7 Mbps aggregate and a sane VideoToolbox load. + static let thumbnail = StreamProfile(maxLongEdge: 640, bitrateKbps: 500, fps: 20) +} diff --git a/RemoteCam/UICmds.swift b/RemoteCam/UICmds.swift index f934eb84..3810a0c3 100644 --- a/RemoteCam/UICmds.swift +++ b/RemoteCam/UICmds.swift @@ -564,4 +564,18 @@ extension UICmd { super.init(sender: nil) } } + + /// Multicam director "collecting" mode: while set, the scanner accumulates + /// several connected cameras instead of auto-advancing to the 1:1 monitor + /// on the first connect. Sent by the scanner only when `ENABLE_MULTICAM` + /// and the monitor role — off, the coordinator's scanning path is + /// byte-identical to before. + public class SetMulticamCollecting: Message, @unchecked Sendable { + let on: Bool + init(on: Bool) { + self.on = on + super.init(sender: nil) + } + } + } diff --git a/RemoteCam/da.lproj/Localizable.strings b/RemoteCam/da.lproj/Localizable.strings index 70827b3e..76b49f1b 100644 --- a/RemoteCam/da.lproj/Localizable.strings +++ b/RemoteCam/da.lproj/Localizable.strings @@ -238,3 +238,28 @@ "IncompatibleBothTitle" = "Appen er ikke opdateret"; "IncompatibleBothBody" = "Opdater Remote Shutter på begge enheder."; "IncompatibleUpdateButton" = "Opdater"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Tilføj kamera"; +"Add" = "Tilføj"; +"Searching for cameras…" = "Søger efter kameraer …"; +"Director" = "Instruktør"; +"Control one or more iPhone cameras" = "Styr et eller flere iPhone-kameraer"; +"Direct up to 4 cameras at once" = "Instruér op til 4 kameraer på én gang"; +"RECONNECTING" = "GENOPRETTER FORBINDELSE"; +"Connect (%d)" = "Forbind (%d)"; +"Select All" = "Vælg alle"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Timer"; +"Off" = "Fra"; +"Video Quality" = "Videokvalitet"; +"Automatic" = "Automatisk"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Rig-indstillinger"; +"%@ can't" = "%@ kan ikke"; +"%@ can't do HDR" = "%@ kan ikke HDR"; +"Retry" = "Prøv igen"; diff --git a/RemoteCam/de-DE.lproj/Localizable.strings b/RemoteCam/de-DE.lproj/Localizable.strings index 4fbae8eb..fbd5ecb8 100644 --- a/RemoteCam/de-DE.lproj/Localizable.strings +++ b/RemoteCam/de-DE.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "Die App ist nicht aktuell"; "IncompatibleBothBody" = "Bitte aktualisieren Sie Remote Shutter auf beiden Geräten."; "IncompatibleUpdateButton" = "Aktualisieren"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Kamera hinzufügen"; +"Add" = "Hinzufügen"; +"Searching for cameras…" = "Suche nach Kameras …"; +"Director" = "Regie"; +"Control one or more iPhone cameras" = "Steuere eine oder mehrere iPhone-Kameras"; +"Direct up to 4 cameras at once" = "Bis zu 4 Kameras gleichzeitig steuern"; +"RECONNECTING" = "VERBINDUNG WIRD WIEDERHERGESTELLT"; +"Connect (%d)" = "Verbinden (%d)"; +"Select All" = "Alle auswählen"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Timer"; +"Off" = "Aus"; +"Video Quality" = "Videoqualität"; +"Automatic" = "Automatisch"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Rig-Einstellungen"; +"%@ can't" = "%@ kann nicht"; +"%@ can't do HDR" = "%@ kann kein HDR"; +"Retry" = "Wiederholen"; diff --git a/RemoteCam/en.lproj/Localizable.strings b/RemoteCam/en.lproj/Localizable.strings index 622cb8ca..093a9097 100644 --- a/RemoteCam/en.lproj/Localizable.strings +++ b/RemoteCam/en.lproj/Localizable.strings @@ -359,3 +359,30 @@ "IncompatibleBothTitle" = "App is out of date"; "IncompatibleBothBody" = "Please update Remote Shutter on both devices."; "IncompatibleUpdateButton" = "Update"; + +// Multicam director (behind ENABLE_MULTICAM) + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Add camera"; +"Add" = "Add"; +"Searching for cameras…" = "Searching for cameras…"; +"Director" = "Director"; +"Control one or more iPhone cameras" = "Control one or more iPhone cameras"; +"Direct up to 4 cameras at once" = "Direct up to 4 cameras at once"; +"RECONNECTING" = "RECONNECTING"; +"Connect (%d)" = "Connect (%d)"; +"Select All" = "Select All"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Timer"; +"Off" = "Off"; +"Video Quality" = "Video Quality"; +"Automatic" = "Automatic"; +"Photo" = "Photo"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Rig Settings"; +"%@ can't" = "%@ can't"; +"%@ can't do HDR" = "%@ can't do HDR"; +"Retry" = "Retry"; diff --git a/RemoteCam/es-MX.lproj/Localizable.strings b/RemoteCam/es-MX.lproj/Localizable.strings index 8367704b..00a0ec2b 100644 --- a/RemoteCam/es-MX.lproj/Localizable.strings +++ b/RemoteCam/es-MX.lproj/Localizable.strings @@ -253,3 +253,28 @@ "IncompatibleBothTitle" = "La app está desactualizada"; "IncompatibleBothBody" = "Actualiza Remote Shutter en ambos dispositivos."; "IncompatibleUpdateButton" = "Actualizar"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Agregar cámara"; +"Add" = "Agregar"; +"Searching for cameras…" = "Buscando cámaras…"; +"Director" = "Director"; +"Control one or more iPhone cameras" = "Controla una o más cámaras de iPhone"; +"Direct up to 4 cameras at once" = "Dirige hasta 4 cámaras a la vez"; +"RECONNECTING" = "RECONECTANDO"; +"Connect (%d)" = "Conectar (%d)"; +"Select All" = "Seleccionar todas"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Temporizador"; +"Off" = "Desactivado"; +"Video Quality" = "Calidad de video"; +"Automatic" = "Automático"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Ajustes del equipo"; +"%@ can't" = "%@ no puede"; +"%@ can't do HDR" = "%@ no admite HDR"; +"Retry" = "Reintentar"; diff --git a/RemoteCam/fr-FR.lproj/Localizable.strings b/RemoteCam/fr-FR.lproj/Localizable.strings index bfdbe936..36f32739 100644 --- a/RemoteCam/fr-FR.lproj/Localizable.strings +++ b/RemoteCam/fr-FR.lproj/Localizable.strings @@ -253,3 +253,28 @@ "IncompatibleBothTitle" = "L'app n'est pas à jour"; "IncompatibleBothBody" = "Mettez Remote Shutter à jour sur les deux appareils."; "IncompatibleUpdateButton" = "Mettre à jour"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Ajouter une caméra"; +"Add" = "Ajouter"; +"Searching for cameras…" = "Recherche de caméras…"; +"Director" = "Réalisateur"; +"Control one or more iPhone cameras" = "Contrôlez une ou plusieurs caméras iPhone"; +"Direct up to 4 cameras at once" = "Dirigez jusqu'à 4 caméras à la fois"; +"RECONNECTING" = "RECONNEXION"; +"Connect (%d)" = "Connecter (%d)"; +"Select All" = "Tout sélectionner"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Minuteur"; +"Off" = "Désactivé"; +"Video Quality" = "Qualité vidéo"; +"Automatic" = "Automatique"; +"Photo" = "Photo"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Réglages du rig"; +"%@ can't" = "%@ ne peut pas"; +"%@ can't do HDR" = "%@ ne gère pas le HDR"; +"Retry" = "Réessayer"; diff --git a/RemoteCam/hi.lproj/Localizable.strings b/RemoteCam/hi.lproj/Localizable.strings index 7202bcc8..a4048da0 100644 --- a/RemoteCam/hi.lproj/Localizable.strings +++ b/RemoteCam/hi.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "ऐप पुराना है"; "IncompatibleBothBody" = "दोनों डिवाइस पर Remote Shutter अपडेट करें।"; "IncompatibleUpdateButton" = "अपडेट करें"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "कैमरा जोड़ें"; +"Add" = "जोड़ें"; +"Searching for cameras…" = "कैमरे खोजे जा रहे हैं…"; +"Director" = "डायरेक्टर"; +"Control one or more iPhone cameras" = "एक या अधिक iPhone कैमरों को नियंत्रित करें"; +"Direct up to 4 cameras at once" = "एक साथ 4 कैमरों तक निर्देशित करें"; +"RECONNECTING" = "फिर से कनेक्ट हो रहा है"; +"Connect (%d)" = "कनेक्ट करें (%d)"; +"Select All" = "सभी चुनें"; + +// Multicam rig settings tray +"Auto" = "ऑटो"; +"Timer" = "टाइमर"; +"Off" = "बंद"; +"Video Quality" = "वीडियो गुणवत्ता"; +"Automatic" = "स्वचालित"; +"Photo" = "फ़ोटो"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "रिग सेटिंग्स"; +"%@ can't" = "%@ नहीं कर सकता"; +"%@ can't do HDR" = "%@ HDR नहीं कर सकता"; +"Retry" = "पुनः प्रयास करें"; diff --git a/RemoteCam/it.lproj/Localizable.strings b/RemoteCam/it.lproj/Localizable.strings index 86d24496..555992a5 100644 --- a/RemoteCam/it.lproj/Localizable.strings +++ b/RemoteCam/it.lproj/Localizable.strings @@ -238,3 +238,28 @@ "IncompatibleBothTitle" = "L'app non è aggiornata"; "IncompatibleBothBody" = "Aggiorna Remote Shutter su entrambi i dispositivi."; "IncompatibleUpdateButton" = "Aggiorna"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Aggiungi fotocamera"; +"Add" = "Aggiungi"; +"Searching for cameras…" = "Ricerca fotocamere…"; +"Director" = "Regista"; +"Control one or more iPhone cameras" = "Controlla una o più fotocamere iPhone"; +"Direct up to 4 cameras at once" = "Dirigi fino a 4 fotocamere contemporaneamente"; +"RECONNECTING" = "RICONNESSIONE"; +"Connect (%d)" = "Connetti (%d)"; +"Select All" = "Seleziona tutte"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Timer"; +"Off" = "Off"; +"Video Quality" = "Qualità video"; +"Automatic" = "Automatico"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Impostazioni rig"; +"%@ can't" = "%@ non può"; +"%@ can't do HDR" = "%@ non supporta HDR"; +"Retry" = "Riprova"; diff --git a/RemoteCam/ja.lproj/Localizable.strings b/RemoteCam/ja.lproj/Localizable.strings index 37f53d83..c6766369 100644 --- a/RemoteCam/ja.lproj/Localizable.strings +++ b/RemoteCam/ja.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "アプリが最新ではありません"; "IncompatibleBothBody" = "両方のデバイスでRemote Shutterを更新してください。"; "IncompatibleUpdateButton" = "更新"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "カメラを追加"; +"Add" = "追加"; +"Searching for cameras…" = "カメラを検索中…"; +"Director" = "ディレクター"; +"Control one or more iPhone cameras" = "1台以上のiPhoneカメラを操作"; +"Direct up to 4 cameras at once" = "最大4台のカメラを同時に操作"; +"RECONNECTING" = "再接続中"; +"Connect (%d)" = "接続 (%d)"; +"Select All" = "すべて選択"; + +// Multicam rig settings tray +"Auto" = "自動"; +"Timer" = "タイマー"; +"Off" = "オフ"; +"Video Quality" = "ビデオ画質"; +"Automatic" = "自動"; +"Photo" = "写真"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "リグ設定"; +"%@ can't" = "%@は非対応"; +"%@ can't do HDR" = "%@はHDR非対応"; +"Retry" = "再試行"; diff --git a/RemoteCam/ko.lproj/Localizable.strings b/RemoteCam/ko.lproj/Localizable.strings index 2be276f6..1a85801e 100644 --- a/RemoteCam/ko.lproj/Localizable.strings +++ b/RemoteCam/ko.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "앱이 최신 버전이 아닙니다"; "IncompatibleBothBody" = "두 기기 모두 Remote Shutter를 업데이트하세요."; "IncompatibleUpdateButton" = "업데이트"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "카메라 추가"; +"Add" = "추가"; +"Searching for cameras…" = "카메라 검색 중…"; +"Director" = "디렉터"; +"Control one or more iPhone cameras" = "하나 이상의 iPhone 카메라 제어"; +"Direct up to 4 cameras at once" = "최대 4대의 카메라를 동시에 제어"; +"RECONNECTING" = "다시 연결 중"; +"Connect (%d)" = "연결 (%d)"; +"Select All" = "모두 선택"; + +// Multicam rig settings tray +"Auto" = "자동"; +"Timer" = "타이머"; +"Off" = "끔"; +"Video Quality" = "비디오 화질"; +"Automatic" = "자동"; +"Photo" = "사진"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "리그 설정"; +"%@ can't" = "%@ 불가"; +"%@ can't do HDR" = "%@ HDR 불가"; +"Retry" = "다시 시도"; diff --git a/RemoteCam/ms.lproj/Localizable.strings b/RemoteCam/ms.lproj/Localizable.strings index 658638a8..f434c870 100644 --- a/RemoteCam/ms.lproj/Localizable.strings +++ b/RemoteCam/ms.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "Apl bukan versi terkini"; "IncompatibleBothBody" = "Sila kemas kini Remote Shutter pada kedua-dua peranti."; "IncompatibleUpdateButton" = "Kemas kini"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Tambah kamera"; +"Add" = "Tambah"; +"Searching for cameras…" = "Mencari kamera…"; +"Director" = "Pengarah"; +"Control one or more iPhone cameras" = "Kawal satu atau lebih kamera iPhone"; +"Direct up to 4 cameras at once" = "Arahkan sehingga 4 kamera serentak"; +"RECONNECTING" = "MENYAMBUNG SEMULA"; +"Connect (%d)" = "Sambung (%d)"; +"Select All" = "Pilih semua"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Pemasa"; +"Off" = "Mati"; +"Video Quality" = "Kualiti video"; +"Automatic" = "Automatik"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Tetapan rig"; +"%@ can't" = "%@ tidak boleh"; +"%@ can't do HDR" = "%@ tidak boleh HDR"; +"Retry" = "Cuba lagi"; diff --git a/RemoteCam/pt-BR.lproj/Localizable.strings b/RemoteCam/pt-BR.lproj/Localizable.strings index c93a9d2f..e1fe7221 100644 --- a/RemoteCam/pt-BR.lproj/Localizable.strings +++ b/RemoteCam/pt-BR.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "O app está desatualizado"; "IncompatibleBothBody" = "Atualize o Remote Shutter nos dois aparelhos."; "IncompatibleUpdateButton" = "Atualizar"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Adicionar câmera"; +"Add" = "Adicionar"; +"Searching for cameras…" = "Procurando câmeras…"; +"Director" = "Diretor"; +"Control one or more iPhone cameras" = "Controle uma ou mais câmeras do iPhone"; +"Direct up to 4 cameras at once" = "Dirija até 4 câmeras ao mesmo tempo"; +"RECONNECTING" = "RECONECTANDO"; +"Connect (%d)" = "Conectar (%d)"; +"Select All" = "Selecionar todas"; + +// Multicam rig settings tray +"Auto" = "Auto"; +"Timer" = "Temporizador"; +"Off" = "Desligado"; +"Video Quality" = "Qualidade de vídeo"; +"Automatic" = "Automático"; +"Photo" = "Foto"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Ajustes do rig"; +"%@ can't" = "%@ não pode"; +"%@ can't do HDR" = "%@ não faz HDR"; +"Retry" = "Tentar novamente"; diff --git a/RemoteCam/ru.lproj/Localizable.strings b/RemoteCam/ru.lproj/Localizable.strings index eef69c8f..f6dc8709 100644 --- a/RemoteCam/ru.lproj/Localizable.strings +++ b/RemoteCam/ru.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "Приложение устарело"; "IncompatibleBothBody" = "Обновите Remote Shutter на обоих устройствах."; "IncompatibleUpdateButton" = "Обновить"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Добавить камеру"; +"Add" = "Добавить"; +"Searching for cameras…" = "Поиск камер…"; +"Director" = "Режиссёр"; +"Control one or more iPhone cameras" = "Управляйте одной или несколькими камерами iPhone"; +"Direct up to 4 cameras at once" = "Управляйте до 4 камерами одновременно"; +"RECONNECTING" = "ПЕРЕПОДКЛЮЧЕНИЕ"; +"Connect (%d)" = "Подключить (%d)"; +"Select All" = "Выбрать все"; + +// Multicam rig settings tray +"Auto" = "Авто"; +"Timer" = "Таймер"; +"Off" = "Выкл"; +"Video Quality" = "Качество видео"; +"Automatic" = "Автоматически"; +"Photo" = "Фото"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Настройки рига"; +"%@ can't" = "%@ не может"; +"%@ can't do HDR" = "%@ не поддерживает HDR"; +"Retry" = "Повторить"; diff --git a/RemoteCam/tr.lproj/Localizable.strings b/RemoteCam/tr.lproj/Localizable.strings index e5648595..5a412c74 100644 --- a/RemoteCam/tr.lproj/Localizable.strings +++ b/RemoteCam/tr.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "Uygulama güncel değil"; "IncompatibleBothBody" = "Remote Shutter'ı iki cihazda da güncelleyin."; "IncompatibleUpdateButton" = "Güncelle"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Kamera ekle"; +"Add" = "Ekle"; +"Searching for cameras…" = "Kameralar aranıyor…"; +"Director" = "Yönetmen"; +"Control one or more iPhone cameras" = "Bir veya daha fazla iPhone kamerasını kontrol edin"; +"Direct up to 4 cameras at once" = "Aynı anda 4 kameraya kadar yönetin"; +"RECONNECTING" = "YENİDEN BAĞLANIYOR"; +"Connect (%d)" = "Bağlan (%d)"; +"Select All" = "Tümünü seç"; + +// Multicam rig settings tray +"Auto" = "Otomatik"; +"Timer" = "Zamanlayıcı"; +"Off" = "Kapalı"; +"Video Quality" = "Video kalitesi"; +"Automatic" = "Otomatik"; +"Photo" = "Fotoğraf"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Rig ayarları"; +"%@ can't" = "%@ yapamaz"; +"%@ can't do HDR" = "%@ HDR yapamaz"; +"Retry" = "Tekrar dene"; diff --git a/RemoteCam/vi.lproj/Localizable.strings b/RemoteCam/vi.lproj/Localizable.strings index 783675f7..5311f1bc 100644 --- a/RemoteCam/vi.lproj/Localizable.strings +++ b/RemoteCam/vi.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "Ứng dụng chưa được cập nhật"; "IncompatibleBothBody" = "Hãy cập nhật Remote Shutter trên cả hai thiết bị."; "IncompatibleUpdateButton" = "Cập nhật"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "Thêm camera"; +"Add" = "Thêm"; +"Searching for cameras…" = "Đang tìm camera…"; +"Director" = "Đạo diễn"; +"Control one or more iPhone cameras" = "Điều khiển một hoặc nhiều camera iPhone"; +"Direct up to 4 cameras at once" = "Điều khiển tối đa 4 camera cùng lúc"; +"RECONNECTING" = "ĐANG KẾT NỐI LẠI"; +"Connect (%d)" = "Kết nối (%d)"; +"Select All" = "Chọn tất cả"; + +// Multicam rig settings tray +"Auto" = "Tự động"; +"Timer" = "Hẹn giờ"; +"Off" = "Tắt"; +"Video Quality" = "Chất lượng video"; +"Automatic" = "Tự động"; +"Photo" = "Ảnh"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "Cài đặt rig"; +"%@ can't" = "%@ không thể"; +"%@ can't do HDR" = "%@ không hỗ trợ HDR"; +"Retry" = "Thử lại"; diff --git a/RemoteCam/zh-Hans.lproj/Localizable.strings b/RemoteCam/zh-Hans.lproj/Localizable.strings index ecad6b64..528ce42a 100644 --- a/RemoteCam/zh-Hans.lproj/Localizable.strings +++ b/RemoteCam/zh-Hans.lproj/Localizable.strings @@ -359,3 +359,28 @@ "IncompatibleBothTitle" = "应用不是最新版本"; "IncompatibleBothBody" = "请在两台设备上都更新 Remote Shutter。"; "IncompatibleUpdateButton" = "更新"; + +// Multicam director (behind ENABLE_MULTICAM) +"Add camera" = "添加相机"; +"Add" = "添加"; +"Searching for cameras…" = "正在搜索相机…"; +"Director" = "导演"; +"Control one or more iPhone cameras" = "控制一台或多台 iPhone 相机"; +"Direct up to 4 cameras at once" = "同时导演最多 4 台相机"; +"RECONNECTING" = "正在重新连接"; +"Connect (%d)" = "连接 (%d)"; +"Select All" = "全选"; + +// Multicam rig settings tray +"Auto" = "自动"; +"Timer" = "定时器"; +"Off" = "关闭"; +"Video Quality" = "视频质量"; +"Automatic" = "自动"; +"Photo" = "照片"; +"HEIF" = "HEIF"; +"HDR" = "HDR"; +"Rig Settings" = "多机位设置"; +"%@ can't" = "%@ 不支持"; +"%@ can't do HDR" = "%@ 不支持 HDR"; +"Retry" = "重试"; diff --git a/RemoteCamTests/CaptureSyncMetadataTests.swift b/RemoteCamTests/CaptureSyncMetadataTests.swift new file mode 100644 index 00000000..fd9f9cc3 --- /dev/null +++ b/RemoteCamTests/CaptureSyncMetadataTests.swift @@ -0,0 +1,118 @@ +// +// CaptureSyncMetadataTests.swift +// RemoteShutterTests +// +// Created by Dario Lencina on 2026. +// Copyright © 2026 Security Union. All rights reserved. +// + +import AVFoundation +import ImageIO +import UniformTypeIdentifiers +import XCTest +@testable import RemoteShutter + +final class CaptureSyncMetadataTests: XCTestCase { + + private let sample = CaptureSyncMetadata( + sessionID: "6BB65B12-30A4-4A5C-9F41-000000000001", + captureID: "D0E1F2A3-1111-2222-3333-000000000002", + cameraIndex: 3, + anchorMillis: 1_754_800_000_123, + clockOffsetMillis: -42, + roundTripMillis: 11 + ) + + func testFilenamePrefixGroupsBySessionCaptureAndCamera() { + XCTAssertEqual(sample.filenamePrefix, "RS_6bb65b12_d0e1f2a3_cam3") + } + + func testJSONRoundTrip() { + guard let json = sample.jsonString() else { + return XCTFail("expected JSON encoding to succeed") + } + XCTAssertEqual(CaptureSyncMetadata.fromJSONString(json), sample) + } + + func testJSONIsDeterministic() { + XCTAssertEqual(sample.jsonString(), sample.jsonString()) + } + + /// The alignment key rides in UserComment as opaque JSON; the EXIF capture + /// date is the camera's WALL clock (a real recent date), never the + /// monotonic-uptime anchor — which would land photos in ~1970. + func testStampedExifDateIsWallClockNotTheMonotonicAnchor() throws { + let png = try makeTinyPNG() + // A 2025 wall-clock instant; the sample's anchorMillis (~1.7e12 ms of + // uptime, i.e. ~55000 years) would parse to a nonsense EXIF date. + let capturedAt = Date(timeIntervalSince1970: 1_754_800_000.250) + let stamped = sample.stamped(png, capturedAt: capturedAt) + + let source = try XCTUnwrap(CGImageSourceCreateWithData(stamped as CFData, nil)) + let props = try XCTUnwrap( + CGImageSourceCopyPropertiesAtIndex(source, 0, nil) as? [CFString: Any]) + let exif = try XCTUnwrap(props[kCGImagePropertyExifDictionary] as? [CFString: Any]) + + // "yyyy:MM:dd HH:mm:ss" — assert the year is the capturedAt year in the + // formatter's own zone (timezone-robust: we don't hard-code HH:mm:ss). + let dateString = try XCTUnwrap(exif[kCGImagePropertyExifDateTimeOriginal] as? String) + let year = Int(dateString.prefix(4)) + XCTAssertEqual(year, 2025, "EXIF DateTimeOriginal must be the wall clock, got \(dateString)") + XCTAssertEqual(exif[kCGImagePropertyExifSubsecTimeOriginal] as? String, "250") + + // The anchor is present, but only inside the opaque UserComment JSON. + let userComment = try XCTUnwrap(exif[kCGImagePropertyExifUserComment] as? String) + let decoded = try XCTUnwrap(CaptureSyncMetadata.fromJSONString(userComment)) + XCTAssertEqual(decoded.anchorMillis, sample.anchorMillis) + } + + /// A 1×1 PNG, enough for CGImageSource to round-trip through the stamper. + private func makeTinyPNG() throws -> Data { + let data = NSMutableData() + let dest = try XCTUnwrap(CGImageDestinationCreateWithData( + data, UTType.png.identifier as CFString, 1, nil)) + let ctx = try XCTUnwrap(CGContext( + data: nil, width: 1, height: 1, bitsPerComponent: 8, bytesPerRow: 4, + space: CGColorSpaceCreateDeviceRGB(), + bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)) + let image = try XCTUnwrap(ctx.makeImage()) + CGImageDestinationAddImage(dest, image, nil) + XCTAssertTrue(CGImageDestinationFinalize(dest)) + return data as Data + } + + func testQuickTimeItemsCarryAnchorAndIDs() { + let items = sample.quickTimeMetadataItems() + XCTAssertEqual(items.count, 4) + + func value(for key: String) -> Any? { + items.first { ($0.key as? String) == key }?.value + } + XCTAssertEqual( + (value(for: CaptureSyncMetadata.QuickTimeKey.anchor) as? NSNumber)?.uint64Value, + sample.anchorMillis) + XCTAssertEqual( + value(for: CaptureSyncMetadata.QuickTimeKey.capture) as? String, + sample.captureID) + XCTAssertEqual( + value(for: CaptureSyncMetadata.QuickTimeKey.session) as? String, + sample.sessionID) + XCTAssertEqual( + (value(for: CaptureSyncMetadata.QuickTimeKey.offset) as? NSNumber)?.int64Value, + sample.clockOffsetMillis) + for item in items { + XCTAssertEqual(item.keySpace, .quickTimeMetadata) + XCTAssertNotNil(item.identifier) + } + } + + /// The wire capability appended for multicam must default to false for + /// legacy peers (absent field) — PR0 ships it inert. + func testCapabilitiesDefaultToNoMulticam() { + let resp = RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: nil, + currentCamera: .back, currentLens: .wideAngle, + currentZoom: 1.0, error: nil) + XCTAssertFalse(resp.supportsMulticam) + } +} diff --git a/RemoteCamTests/ClockOffsetEstimatorTests.swift b/RemoteCamTests/ClockOffsetEstimatorTests.swift new file mode 100644 index 00000000..de19c9f3 --- /dev/null +++ b/RemoteCamTests/ClockOffsetEstimatorTests.swift @@ -0,0 +1,80 @@ +// +// ClockOffsetEstimatorTests.swift +// RemoteShutterTests +// +// Created by Dario Lencina on 2026. +// Copyright © 2026 Security Union. All rights reserved. +// + +import XCTest +@testable import RemoteShutter + +final class ClockOffsetEstimatorTests: XCTestCase { + + func testSymmetricExchangeRecoversExactOffset() { + // Camera runs 500ms ahead. Ping takes 10ms each way: sent at t0=1000, + // received at director-time 1010 = camera clock 1510, pong back at + // t3=1020. Symmetric latency → the rtt/2 midpoint is exact. + var estimator = ClockOffsetEstimator() + let sample = estimator.recordExchange( + t0Millis: 1000, cameraClockMillis: 1510, t3Millis: 1020) + XCTAssertEqual(sample?.offsetMillis, 500) + XCTAssertEqual(sample?.roundTripMillis, 20) + } + + func testAsymmetricLatencyErrorIsBoundedByHalfRoundTrip() { + // Worst case: all 20ms on the outbound leg. Receive happens at + // director-time 1020 (camera 1520) but the midpoint guess is 1010 — + // estimate 510, true offset 500. |error| = rtt/2. + var estimator = ClockOffsetEstimator() + let sample = estimator.recordExchange( + t0Millis: 1000, cameraClockMillis: 1520, t3Millis: 1020) + XCTAssertEqual(sample?.offsetMillis, 510) + } + + func testNegativeOffsetWhenCameraRunsBehind() { + var estimator = ClockOffsetEstimator() + let sample = estimator.recordExchange( + t0Millis: 10_000, cameraClockMillis: 8_010, t3Millis: 10_020) + XCTAssertEqual(sample?.offsetMillis, -2_000) + } + + func testBestPrefersMinimumRoundTrip() { + var estimator = ClockOffsetEstimator() + // A slow, queued exchange with a skewed midpoint... + estimator.recordExchange(t0Millis: 1000, cameraClockMillis: 1900, t3Millis: 1400) + // ...must not outvote a clean 8ms exchange. + estimator.recordExchange(t0Millis: 2000, cameraClockMillis: 2504, t3Millis: 2008) + XCTAssertEqual(estimator.best?.roundTripMillis, 8) + XCTAssertEqual(estimator.best?.offsetMillis, 500) + } + + func testWindowSlidesAndDropsOldest() { + var estimator = ClockOffsetEstimator() + // Fill the window with a fast-but-old sample first. + estimator.recordExchange(t0Millis: 0, cameraClockMillis: 501, t3Millis: 2) + for i in 0.. ([MCPeerID], DeviceScannerViewModel) { + let vm = DeviceScannerViewModel() + vm.role = .monitor + let peers = (0.. 0) + XCTAssertEqual(vm.multicamSelectionCount, k) + + // (c) tap Connect → exactly the k selected peers to invite. + let toInvite = vm.beginMulticamConnecting() + XCTAssertEqual(Set(toInvite), Set(peers.prefix(k))) + XCTAssertEqual(toInvite.count, k) + if k > 0 { XCTAssertEqual(vm.multicamPhase, .connecting) } + } + } + } + } + + /// (d) Rows transition selected → connecting → connected, and the handoff + /// decision matches the connected count. + func testConnectingTransitionsAndHandoffDestination() { + for connectCount in 0...4 { + let (peers, vm) = makeVM(discovered: 4) + let selected = Array(peers.prefix(max(connectCount, 1))) + selected.forEach { vm.toggleMulticamSelection($0) } + let toInvite = vm.beginMulticamConnecting() + + // All selected are "connecting" right after Connect. + for p in toInvite { XCTAssertEqual(vm.multicamRowState(p), .connecting) } + + // The first `connectCount` connect; the rest fail. + let connected = Array(selected.prefix(connectCount)) + vm.reconcileMulticamConnected(connected) + for p in selected where !connected.contains(p) { vm.markMulticamFailed(p) } + + for p in connected { XCTAssertEqual(vm.multicamRowState(p), .connected) } + XCTAssertTrue(vm.multicamConnectSettled) + + switch MulticamHandoff.decide(connected: connected) { + case .none: XCTAssertEqual(connectCount, 0) + case .classicMonitor(let p): XCTAssertEqual(connectCount, 1); XCTAssertEqual(p, connected[0]) + case .director(let ps): XCTAssertGreaterThanOrEqual(connectCount, 2); XCTAssertEqual(ps, connected) + } + } + } + + /// Cap: with 10 discovered you can select up to `cap`; the (cap+1)th row + /// locks, and Select All stops at the cap. + func testCapLocksBeyondMaxCameras() { + for cap in [2, 4] { + let (peers, vm) = makeVM(discovered: 10) + for i in 0.. (MulticamController, FakeMultipeerService, FakeMulticamDisplay) { + let controller = MulticamController() + let transport = FakeMultipeerService() + transport.sendResult = true + transport.connectedPeers = peers + let display = FakeMulticamDisplay() + await controller.setDisplay(display) + await controller.install(transport: transport, initialPeers: peers, mode: .photo) + await controller.waitForIdle() + return (controller, transport, display) + } + + private func sent(_ transport: FakeMultipeerService, _ type: T.Type) + -> [(msg: Message, peers: [MCPeerID], mode: MCSessionSendDataMode)] { + transport.sentMessages.filter { $0.msg is T } + } + + // MARK: - Handshake + + func testInstallSeedsLaneAndHandshakesEveryCamera() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + + let lanes = await controller.lanesForTesting() + XCTAssertEqual(Set(lanes.map(\.peerID)), [camA, camB]) + + // Each camera is asked for capabilities, exactly addressed to itself. + let requests = sent(transport, RemoteCmd.RequestCameraCapabilities.self) + XCTAssertEqual(Set(requests.flatMap(\.peers)), [camA, camB]) + for req in requests { XCTAssertEqual(req.peers.count, 1) } + + // And discovery keeps running so more cameras can be added later. + XCTAssertGreaterThanOrEqual(transport.discoveryStarts, 1) + } + + func testFirstCameraIsFocused() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + let focused = await controller.focusedPeerForTesting() + XCTAssertEqual(focused, camA) + } + + // MARK: - Capabilities → live + + func testCapabilitiesMarkLaneLinkedAndPingClockWhenMulticam() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + transport.sentMessages.removeAll() + + controller.didReceiveMessage(multicamCaps(), from: camA) + await controller.waitForIdle() + + let statusA = await controller.statusForTesting(camA) + XCTAssertEqual(statusA, .linked) + // A multicam-capable camera gets an immediate clock probe, to itself. + let pings = sent(transport, RemoteCmd.ClockSyncPing.self) + XCTAssertEqual(pings.map(\.peers), [[camA]]) + } + + // MARK: - Frame routing (Seam B) + + func testFrameRoutesToItsLaneAndAcksOnlyItsSource() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + // Register a per-lane sink for each camera, mirroring how the view + // controller wires one at lane creation. + let collector = FrameSinkCollector() + let (a, b) = (camA, camB) + await controller.setFrameSink(for: a) { _ in collector.record(a) } + await controller.setFrameSink(for: b) { _ in collector.record(b) } + transport.sentMessages.removeAll() + + controller.didReceiveFrame(sendFrame(), from: camB) + await controller.waitForIdle() + + XCTAssertEqual(collector.peers, [camB], + "the frame reaches only its own lane's decoder, never another's") + let acks = sent(transport, RemoteCmd.RequestFrame.self) + XCTAssertEqual(acks.map(\.peers), [[camB]], + "the frame ack must address only the camera that sent the frame") + } + + // MARK: - Focused-camera commands + + func testPerCameraCommandTargetsOnlyTheFocusedPeer() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.setFocusedPeer(camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + await controller.setZoom(2.0) + await controller.waitForIdle() + + let zooms = sent(transport, RemoteCmd.SetZoom.self) + XCTAssertEqual(zooms.map(\.peers), [[camB]]) + } + + // MARK: - Disconnect / reconnect + + func testDisconnectDegradesOnlyThatLane() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + + controller.peerDidDisconnect(camA) + await controller.waitForIdle() + + let statusA = await controller.statusForTesting(camA) + let statusB = await controller.statusForTesting(camB) + XCTAssertEqual(statusA, .reconnecting) + XCTAssertEqual(statusB, .linked, "one camera dropping must not disturb the others") + } + + func testBrowserReinvitesOnlyAReconnectingCamera() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.peerDidDisconnect(camA) + await controller.waitForIdle() + transport.invitedPeers.removeAll() + + controller.browserDidFindPeer(camA) + await controller.waitForIdle() + XCTAssertEqual(transport.invitedPeers.map(\.peer), [camA]) + + // A camera that is already linked is not re-invited on a browser hit. + transport.invitedPeers.removeAll() + controller.browserDidFindPeer(camB) + await controller.waitForIdle() + XCTAssertTrue(transport.invitedPeers.isEmpty) + } + + // MARK: - Clock sync + + func testPongUpdatesThatLanesOffset() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + + // Camera A is 500ms ahead; symmetric 20ms RTT is faked via the pong's + // camera clock relative to our own — we only assert an offset landed. + controller.didReceiveMessage( + RemoteCmd.ClockSyncPong(echoT0Millis: SyncClock.nowMillis(), + cameraClockMillis: SyncClock.nowMillis() + 500), + from: camA) + await controller.waitForIdle() + + let offsetA = await controller.offsetForTesting(camA) + let offsetB = await controller.offsetForTesting(camB) + XCTAssertNotNil(offsetA) + XCTAssertNil(offsetB) + } + + // MARK: - Synced photo capture + + func testScheduledCaptureAppliesPerLaneOffsets() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 100) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: -50) + transport.sentMessages.removeAll() + + controller.capturePhoto() + await controller.waitForIdle() + + let scheduled = sent(transport, RemoteCmd.ScheduledCapture.self) + XCTAssertEqual(scheduled.count, 2) + for s in scheduled { XCTAssertEqual(s.peers.count, 1) } + + func fireAt(_ peer: MCPeerID) -> UInt64? { + (scheduled.first { $0.peers == [peer] }?.msg as? RemoteCmd.ScheduledCapture)? + .fireAtCameraClockMillis + } + // Same shared anchor + each lane's own offset ⇒ the fire instants differ + // by exactly the offset difference (100 − (−50) = 150), regardless of + // the nondeterministic base. + let a = fireAt(camA), b = fireAt(camB) + XCTAssertNotNil(a); XCTAssertNotNil(b) + XCTAssertEqual(Int64(a!) - Int64(b!), 150) + + // One shared capture id across both cameras. + let ids = Set(scheduled.compactMap { ($0.msg as? RemoteCmd.ScheduledCapture)?.captureId }) + XCTAssertEqual(ids.count, 1) + } + + func testNonMulticamLaneIsExcludedFromCapture() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 10) + await controller.seedLaneForTesting(camB, supportsMulticam: false, offsetMillis: 10) + transport.sentMessages.removeAll() + + controller.capturePhoto() + await controller.waitForIdle() + + let scheduled = sent(transport, RemoteCmd.ScheduledCapture.self) + XCTAssertEqual(scheduled.map(\.peers), [[camA]]) + let capture = await controller.captureStateForTesting() + XCTAssertEqual(capture?.remaining, 1) + } + + func testAckAggregationReturnsToMonitoring() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 0) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: 0) + + controller.capturePhoto() + await controller.waitForIdle() + let captureID = await controller.captureStateForTesting()?.id + XCTAssertNotNil(captureID) + + controller.didReceiveMessage(RemoteCmd.ScheduledCaptureAck(captureId: captureID!), from: camA) + await controller.waitForIdle() + let afterFirst = await controller.captureStateForTesting() + XCTAssertEqual(afterFirst?.remaining, 1) + + controller.didReceiveMessage(RemoteCmd.ScheduledCaptureAck(captureId: captureID!), from: camB) + await controller.waitForIdle() + let afterSecond = await controller.captureStateForTesting() + let outA = await controller.captureOutcomeForTesting(camA) + let outB = await controller.captureOutcomeForTesting(camB) + XCTAssertNil(afterSecond, "back to monitoring") + XCTAssertEqual(outA, .captured) + XCTAssertEqual(outB, .captured) + } + + func testCaptureTimeoutMarksTheSilentLaneAndCompletes() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + await controller.setCaptureAckTimeout(0.1) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 0) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: 0) + + controller.capturePhoto() + await controller.waitForIdle() + let captureID = await controller.captureStateForTesting()?.id + controller.didReceiveMessage(RemoteCmd.ScheduledCaptureAck(captureId: captureID!), from: camA) + await controller.waitForIdle() + + // Wait out the ack timeout; camB never answered. + try? await Task.sleep(nanoseconds: 300_000_000) + let resolved = await controller.captureStateForTesting() + let outA = await controller.captureOutcomeForTesting(camA) + let outB = await controller.captureOutcomeForTesting(camB) + XCTAssertNil(resolved, "aggregate resolves") + XCTAssertEqual(outA, .captured) + XCTAssertEqual(outB, .failed) + } + + func testFallbackToPlainTakePicWhenAnOffsetIsMissing() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 20) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: nil) + transport.sentMessages.removeAll() + + controller.capturePhoto() + await controller.waitForIdle() + + XCTAssertTrue(sent(transport, RemoteCmd.ScheduledCapture.self).isEmpty, + "a missing offset forces the plain fan-out") + let takes = sent(transport, RemoteCmd.TakePic.self) + XCTAssertEqual(Set(takes.flatMap(\.peers)), [camA, camB]) + let fallbackState = await controller.captureStateForTesting() + XCTAssertEqual(fallbackState?.remaining, 2) + } + + // MARK: - Add camera + + func testDiscoveredPeerBecomesAvailableButDoesNotAutoJoin() async { + let camC = MCPeerID(displayName: "CameraC") + let (controller, transport, _) = await makeController(peers: [camA, camB]) + transport.invitedPeers.removeAll() + + controller.browserDidFindPeer(camC) + await controller.waitForIdle() + + let available = await controller.availablePeersForTesting() + XCTAssertEqual(available, [camC]) + // A fresh peer is a candidate only — it is never auto-invited. + XCTAssertTrue(transport.invitedPeers.isEmpty) + // And it is not in the rig. + let count = await controller.cameraCountForTesting() + XCTAssertEqual(count, 2) + } + + func testInviteCameraInvitesADiscoveredPeer() async { + let camC = MCPeerID(displayName: "CameraC") + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.browserDidFindPeer(camC) + await controller.waitForIdle() + transport.invitedPeers.removeAll() + + controller.inviteCamera(camC) + await controller.waitForIdle() + XCTAssertEqual(transport.invitedPeers.map(\.peer), [camC]) + + // Inviting a peer that was never discovered does nothing. + transport.invitedPeers.removeAll() + controller.inviteCamera(MCPeerID(displayName: "Ghost")) + await controller.waitForIdle() + XCTAssertTrue(transport.invitedPeers.isEmpty) + } + + func testAvailablePeerClearsOnceItJoins() async { + let camC = MCPeerID(displayName: "CameraC") + let (controller, _, _) = await makeController(peers: [camA, camB]) + controller.browserDidFindPeer(camC) + await controller.waitForIdle() + let avail = await controller.availablePeersForTesting() + XCTAssertEqual(avail, [camC]) + + controller.peerDidConnect(camC) + await controller.waitForIdle() + let availAfter = await controller.availablePeersForTesting() + let countAfter = await controller.cameraCountForTesting() + XCTAssertTrue(availAfter.isEmpty) + XCTAssertEqual(countAfter, 3) + } + + // MARK: - Stream profile tiering + + func testFocusedLaneGetsFullProfileOthersGetThumbnail() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + transport.sentMessages.removeAll() + + // Both cameras report multicam caps; camA is focused (first-in). + controller.didReceiveMessage(multicamCaps(), from: camA) + controller.didReceiveMessage(multicamCaps(), from: camB) + await controller.waitForIdle() + + let profiles = sent(transport, RemoteCmd.SetStreamProfile.self) + func profile(_ p: MCPeerID) -> RemoteCmd.SetStreamProfile? { + profiles.first { $0.peers == [p] }?.msg as? RemoteCmd.SetStreamProfile + } + XCTAssertEqual(profile(camA)?.maxLongEdge, Int(StreamProfile.focused.maxLongEdge)) + XCTAssertEqual(profile(camB)?.maxLongEdge, Int(StreamProfile.thumbnail.maxLongEdge)) + } + + func testFocusSwitchRetiersBothLanes() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.didReceiveMessage(multicamCaps(), from: camA) + controller.didReceiveMessage(multicamCaps(), from: camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + await controller.setFocusedPeer(camB) + await controller.waitForIdle() + + let profiles = sent(transport, RemoteCmd.SetStreamProfile.self) + func edge(_ p: MCPeerID) -> Int? { + (profiles.first { $0.peers == [p] }?.msg as? RemoteCmd.SetStreamProfile)?.maxLongEdge + } + // camB becomes full, camA drops to thumbnail — no redundant re-sends. + XCTAssertEqual(edge(camB), Int(StreamProfile.focused.maxLongEdge)) + XCTAssertEqual(edge(camA), Int(StreamProfile.thumbnail.maxLongEdge)) + } + + func testProfileNotResentWhenUnchanged() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.didReceiveMessage(multicamCaps(), from: camA) + controller.didReceiveMessage(multicamCaps(), from: camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + // Focusing the already-focused camera changes no tier → no profile sends. + await controller.setFocusedPeer(camA) + await controller.waitForIdle() + XCTAssertTrue(sent(transport, RemoteCmd.SetStreamProfile.self).isEmpty) + } + + // MARK: - Synced video + + func testStartAndStopAnchorsGiveMatchingClipLengths() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 100) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: -50) + transport.sentMessages.removeAll() + + controller.startRecording() + await controller.waitForIdle() + let starts = sent(transport, RemoteCmd.ScheduledStartRecording.self) + func startFire(_ p: MCPeerID) -> UInt64 { + (starts.first { $0.peers == [p] }!.msg as! RemoteCmd.ScheduledStartRecording).fireAtCameraClockMillis + } + + // Mark both rolling so stopRecording targets them. + let recID = await controller.recordingStateForTesting()?.id + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: recID!, isStop: false), from: camA) + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: recID!, isStop: false), from: camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + controller.stopRecording() + await controller.waitForIdle() + let stops = sent(transport, RemoteCmd.ScheduledStopRecording.self) + func stopFire(_ p: MCPeerID) -> UInt64 { + (stops.first { $0.peers == [p] }!.msg as! RemoteCmd.ScheduledStopRecording).fireAtCameraClockMillis + } + + // Per-lane clip length (stop − start) is identical across cameras: each + // lane's offset cancels, leaving the shared (stopBase − startBase). + let lenA = Int64(stopFire(camA)) - Int64(startFire(camA)) + let lenB = Int64(stopFire(camB)) - Int64(startFire(camB)) + XCTAssertEqual(lenA, lenB, "clip lengths must match across the rig") + // And the per-lane fire instants differ by the offset delta (150) for + // both start and stop. + XCTAssertEqual(Int64(startFire(camA)) - Int64(startFire(camB)), 150) + XCTAssertEqual(Int64(stopFire(camA)) - Int64(stopFire(camB)), 150) + } + + func testStartAcksMarkLanesRecording() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 0) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: 0) + + controller.startRecording() + await controller.waitForIdle() + let recID = await controller.recordingStateForTesting()?.id + XCTAssertNotNil(recID) + + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: recID!, isStop: false), from: camA) + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: recID!, isStop: false), from: camB) + await controller.waitForIdle() + + let recA = await controller.isRecordingForTesting(camA) + let recB = await controller.isRecordingForTesting(camB) + XCTAssertTrue(recA) + XCTAssertTrue(recB) + // Still recording (all start acks in, remaining 0). + let stillRecording = await controller.recordingStateForTesting() + XCTAssertEqual(stillRecording?.remaining, 0) + + // Stop resolves back to monitoring. + controller.stopRecording() + await controller.waitForIdle() + let stopID = await controller.stoppingStateForTesting()?.id + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: stopID!, isStop: true), from: camA) + controller.didReceiveMessage(RemoteCmd.ScheduledRecordingAck(captureId: stopID!, isStop: true), from: camB) + await controller.waitForIdle() + let afterStop = await controller.recordingStateForTesting() + let stoppingAfter = await controller.stoppingStateForTesting() + XCTAssertNil(afterStop) + XCTAssertNil(stoppingAfter, "back to monitoring") + let recAafter = await controller.isRecordingForTesting(camA) + XCTAssertFalse(recAafter) + } + + // MARK: - Removal + + func testRemoveCameraDropsTheLaneAndRefocuses() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + + controller.removeCamera(camA) + await controller.waitForIdle() + let lanes = await controller.lanesForTesting() + XCTAssertEqual(lanes.map(\.peerID), [camB]) + let focusedAfter = await controller.focusedPeerForTesting() + XCTAssertEqual(focusedAfter, camB) + } + + // MARK: - Rig quality (intersection) + timer + + /// Caps whose current (back) camera advertises a resolution/fps matrix. + private func capsWith(_ matrix: [VideoResolution: [VideoFrameRate]], + heif: Bool = true, hdr: Bool = true) -> RemoteCmd.CameraCapabilitiesResp { + let info = RemoteCmd.CameraInfo( + availableLenses: [.wideAngle], hasFlash: true, hasTorch: true, + zoomCapabilities: [:], + supportedResolutions: Array(matrix.keys), + supportedFrameRates: Array(Set(matrix.values.flatMap { $0 })), + resolutionFrameRates: matrix, supportsHEIF: heif, supportsHDR: hdr) + return RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: info, + currentCamera: .back, currentLens: .wideAngle, currentZoom: 1.0, + supportsMulticam: true, error: nil) + } + + private let full4K: [VideoResolution: [VideoFrameRate]] = + [.uhd4k: [.fps30, .fps60], .hd1080p: [.fps30, .fps60]] + private let only1080: [VideoResolution: [VideoFrameRate]] = [.hd1080p: [.fps30]] + + func testSetVideoQualityFansOutToEveryLane() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + transport.sentMessages.removeAll() + + controller.setVideoQuality(resolution: .uhd4k, frameRate: .fps30) + await controller.waitForIdle() + + let sends = sent(transport, RemoteCmd.SetVideoQuality.self) + XCTAssertEqual(Set(sends.flatMap(\.peers)), [camA, camB]) + for s in sends { + let q = s.msg as? RemoteCmd.SetVideoQuality + XCTAssertEqual(q?.resolution, .uhd4k) + XCTAssertEqual(q?.frameRate, .fps30) + } + let active = await controller.activeVideoQualityForTesting() + XCTAssertEqual(active?.0, .uhd4k) + } + + /// Manual toggle between two shared options fans out each time (Dario's + /// first-class-manual requirement). + func testManualQualityToggleFansOutEachTime() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.didReceiveMessage(capsWith(full4K), from: camA) + controller.didReceiveMessage(capsWith(full4K), from: camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + controller.setVideoQuality(resolution: .hd1080p, frameRate: .fps30) + controller.setVideoQuality(resolution: .uhd4k, frameRate: .fps30) + await controller.waitForIdle() + + let sends = sent(transport, RemoteCmd.SetVideoQuality.self) + // Two toggles × two lanes = four sends. + XCTAssertEqual(sends.count, 4) + } + + func testAutomaticPicksBestInIntersectionAndFansOut() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + // camA does 4K; camB only 1080p → the rig can only agree on 1080p30. + controller.didReceiveMessage(capsWith(full4K), from: camA) + controller.didReceiveMessage(capsWith(only1080), from: camB) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + controller.applyAutomaticVideoQuality() + await controller.waitForIdle() + + let sends = sent(transport, RemoteCmd.SetVideoQuality.self) + XCTAssertEqual(Set(sends.flatMap(\.peers)), [camA, camB]) + let q = sends.first?.msg as? RemoteCmd.SetVideoQuality + XCTAssertEqual(q?.resolution, .hd1080p) + XCTAssertEqual(q?.frameRate, .fps30) + } + + func testLateJoinerThatCannotMatchIsFlagged() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + controller.didReceiveMessage(capsWith(full4K), from: camA) + controller.didReceiveMessage(capsWith(full4K), from: camB) + await controller.waitForIdle() + // Rig set to 4K30 (both can). + controller.setVideoQuality(resolution: .uhd4k, frameRate: .fps30) + await controller.waitForIdle() + let flaggedBefore = await controller.needsRematchForTesting(camB) + XCTAssertFalse(flaggedBefore) + + // camB device-switches to a 1080-only camera → can't match → flagged. + controller.didReceiveMessage(capsWith(only1080), from: camB) + await controller.waitForIdle() + let flaggedAfter = await controller.needsRematchForTesting(camB) + let camAstillOK = await controller.needsRematchForTesting(camA) + XCTAssertTrue(flaggedAfter) + XCTAssertFalse(camAstillOK) + } + + func testRigTimerCountsDownFansOutAndFiresCapture() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + // Seed offsets so the fired capture takes the scheduled path. + await controller.seedLaneForTesting(camA, supportsMulticam: true, offsetMillis: 0) + await controller.seedLaneForTesting(camB, supportsMulticam: true, offsetMillis: 0) + // Large interval so the production tick Task never fires during the + // test — the ticks are driven deterministically through the inbox. + await controller.setTimerTickInterval(1000) + controller.setRigTimer(3) + await controller.waitForIdle() + transport.sentMessages.removeAll() + + controller.capturePhoto() + await controller.waitForIdle() // arms the countdown, fans tick 3 + var remaining = await controller.countdownRemainingForTesting() + XCTAssertEqual(remaining, 3) + + // Drive the countdown 3 → 2 → 1 → 0 (fires) through the pump. + for _ in 0..<3 { + await controller.advanceTimerForTesting() + await controller.waitForIdle() + } + remaining = await controller.countdownRemainingForTesting() + XCTAssertNil(remaining, "countdown finished") + + let ticks = sent(transport, RemoteCmd.TimerCountdown.self).compactMap { $0.msg as? RemoteCmd.TimerCountdown } + XCTAssertTrue(ticks.contains { $0.value == 3 }) + XCTAssertTrue(ticks.contains { $0.value == 0 }) + XCTAssertFalse(sent(transport, RemoteCmd.ScheduledCapture.self).isEmpty, + "expiry fired the synced capture") + } + + func testSetPhotoQualityFansOut() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + transport.sentMessages.removeAll() + controller.setPhotoQuality(format: .heif, hdr: .on) + await controller.waitForIdle() + let sends = sent(transport, RemoteCmd.SetPhotoQuality.self) + XCTAssertEqual(Set(sends.flatMap(\.peers)), [camA, camB]) + let q = sends.first?.msg as? RemoteCmd.SetPhotoQuality + XCTAssertEqual(q?.format, .heif) + XCTAssertEqual(q?.hdrMode, .on) + } + + // MARK: - Auto-collect + + func testVideoResourceTransferUpdatesLaneStateAndSaves() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + let progress = Progress(totalUnitCount: 100) + + controller.didStartReceivingResource(name: "RS_a_b_cam1.mov", from: camA, progress: progress) + await controller.waitForIdle() + var stateA = await controller.collectionStateForTesting(camA) + XCTAssertEqual(stateA, .transferring(0)) + + let url = FileManager.default.temporaryDirectory.appendingPathComponent("clip.mov") + controller.didFinishReceivingResource(name: "RS_a_b_cam1.mov", from: camA, at: url, error: nil) + await controller.waitForIdle() + stateA = await controller.collectionStateForTesting(camA) + XCTAssertEqual(stateA, .collected) + // camB untouched. + let stateB = await controller.collectionStateForTesting(camB) + XCTAssertEqual(stateB, .idle) + } + + func testFailedTransferMarksLaneAndRetryReRequests() async { + let (controller, transport, _) = await makeController(peers: [camA, camB]) + controller.didFinishReceivingResource( + name: "RS_a_b_cam1.mov", from: camA, at: nil, + error: NSError(domain: "x", code: 1)) + await controller.waitForIdle() + let failed = await controller.collectionStateForTesting(camA) + XCTAssertEqual(failed, .failed) + + transport.sentMessages.removeAll() + controller.retryCollection(for: camA) + await controller.waitForIdle() + // Re-request goes to that camera, and its lane returns to transferring. + let resends = sent(transport, RemoteCmd.RequestVideoResend.self) + XCTAssertEqual(resends.map(\.peers), [[camA]]) + let retrying = await controller.collectionStateForTesting(camA) + XCTAssertEqual(retrying, .transferring(0)) + } + + func testReturnedPhotoMarksLaneCollected() async { + let (controller, _, _) = await makeController(peers: [camA, camB]) + // A returned still (non-nil pic, no error) is collected on the director. + let jpeg = Data([0xFF, 0xD8, 0xFF, 0xE0, 0, 0, 0, 0]) + controller.didReceiveMessage( + RemoteCmd.TakePicResp(sender: nil, pic: jpeg, error: nil), from: camB) + await controller.waitForIdle() + let state = await controller.collectionStateForTesting(camB) + XCTAssertEqual(state, .collected) + } + + // MARK: - Fixtures + + private func multicamCaps() -> RemoteCmd.CameraCapabilitiesResp { + RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: nil, + currentCamera: .back, currentLens: .wideAngle, currentZoom: 1.0, + supportsMulticam: true, error: nil) + } + + private func sendFrame() -> RemoteCmd.SendFrame { + RemoteCmd.SendFrame(data: Data([1, 2, 3]), sender: nil, + fps: 30, camPosition: .back, camOrientation: .portrait, + codec: .vp9, sequenceNumber: 1) + } +} diff --git a/RemoteCamTests/MulticamViewModelTests.swift b/RemoteCamTests/MulticamViewModelTests.swift new file mode 100644 index 00000000..d94f37d7 --- /dev/null +++ b/RemoteCamTests/MulticamViewModelTests.swift @@ -0,0 +1,64 @@ +// +// MulticamViewModelTests.swift +// RemoteShutterTests +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import MPCCompat +import XCTest +@testable import RemoteShutter + +final class MulticamViewModelTests: XCTestCase { + + private let camA = MCPeerID(displayName: "CameraA") + private let camB = MCPeerID(displayName: "CameraB") + + private func info(_ peer: MCPeerID, status: CameraLink.Status = .linked, + focused: Bool = false) -> MulticamLaneInfo { + MulticamLaneInfo(peerID: peer, displayName: peer.displayName, + status: status, isFocused: focused, clockOffsetMillis: nil, + captureOutcome: nil, isRecording: false, needsQualityRematch: false, + collection: .idle) + } + + func testApplyAddsLanesAndReportsCreated() { + let vm = MulticamViewModel() + let created = vm.apply([info(camA, focused: true), info(camB)]) + XCTAssertEqual(vm.lanes.map(\.peerID), [camA, camB]) + XCTAssertEqual(created.map(\.peerID), [camA, camB]) + } + + func testApplyPreservesExistingLaneInstances() { + let vm = MulticamViewModel() + vm.apply([info(camA, focused: true), info(camB)]) + let laneABefore = vm.lane(for: camA) + + // Second apply changes only status; the CameraLane (and its live + // frames/receiver) must be the same instance, not rebuilt. + let created = vm.apply([info(camA, status: .reconnecting, focused: true), info(camB)]) + XCTAssertTrue(created.isEmpty, "no new lanes should be created") + XCTAssertTrue(vm.lane(for: camA) === laneABefore) + XCTAssertEqual(vm.lane(for: camA)?.status, .reconnecting) + } + + func testApplyDropsGoneLanes() { + let vm = MulticamViewModel() + vm.apply([info(camA, focused: true), info(camB)]) + vm.apply([info(camA, focused: true)]) + XCTAssertEqual(vm.lanes.map(\.peerID), [camA]) + XCTAssertNil(vm.lane(for: camB)) + } + + func testFocusedAndOtherLanesPartition() { + let vm = MulticamViewModel() + vm.apply([info(camA, focused: true), info(camB)]) + XCTAssertEqual(vm.focusedLane?.peerID, camA) + XCTAssertEqual(vm.otherLanes.map(\.peerID), [camB]) + + // Refocusing moves the partition without rebuilding lanes. + vm.apply([info(camA), info(camB, focused: true)]) + XCTAssertEqual(vm.focusedLane?.peerID, camB) + XCTAssertEqual(vm.otherLanes.map(\.peerID), [camA]) + } +} diff --git a/RemoteCamTests/RemoteCamSessionTests.swift b/RemoteCamTests/RemoteCamSessionTests.swift index e748d145..ee77a8b8 100644 --- a/RemoteCamTests/RemoteCamSessionTests.swift +++ b/RemoteCamTests/RemoteCamSessionTests.swift @@ -117,6 +117,64 @@ class SessionCoordinatorTests: XCTestCase { await harness.coordinator.seed(state: .scanning, lobby: harness.lobbyWrapper) } + // MARK: - Multicam "Connect (N)": invite the selected set, retry, report + + /// The device-test regression, guarded at the transport: selecting rows + /// (pure VM toggles, the production tap path) must invite nothing. + func testSelectingRowsSendsZeroInvites() async { + await seedScanning() + await harness.deliver(UICmd.SetMulticamCollecting(on: true)) + let vm = harness.lobby.scannerViewModel + let peers = (0..<5).map { MCPeerID(displayName: "Cam\($0)") } + peers.forEach { vm.addPeer($0) } + peers.forEach { vm.toggleMulticamSelection($0) } // taps = pure selection + + await harness.coordinator.waitForIdle() + XCTAssertTrue(harness.fakeMP.invitedPeers.isEmpty, + "selecting must never invite — the device-test regression") + } + + func testMulticamConnectInvitesEachSelectedPeer() async { + await seedScanning() + await harness.deliver(UICmd.SetMulticamCollecting(on: true)) + harness.fakeMP.connectedPeers = [] + let camA = MCPeerID(displayName: "CamA") + let camB = MCPeerID(displayName: "CamB") + + // "Connect" fires one invite per selected camera — no `link` clobbering. + await harness.deliver(ConnectToDevice(peer: camA, sender: nil)) + await harness.deliver(ConnectToDevice(peer: camB, sender: nil)) + XCTAssertEqual(Set(harness.fakeMP.invitedPeers.map(\.peer)), [camA, camB]) + + // As each connects, the coordinator reports the growing connected set. + harness.fakeMP.connectedPeers = [camA] + await harness.deliver(OnConnectToDevice(peer: camA, sender: nil)) + harness.fakeMP.connectedPeers = [camA, camB] + await harness.deliver(OnConnectToDevice(peer: camB, sender: nil)) + let count = await harness.coordinator.multicamConnectedCount() + XCTAssertEqual(count, 2) + } + + func testMulticamInviteRetriesOnceThenReportsFailure() async { + await seedScanning() + await harness.deliver(UICmd.SetMulticamCollecting(on: true)) + harness.fakeMP.connectedPeers = [] + let camA = MCPeerID(displayName: "CamA") + + await harness.deliver(ConnectToDevice(peer: camA, sender: nil)) + XCTAssertEqual(harness.fakeMP.invitedPeers.count, 1) + + // First drop → retry (a second invite), no failure yet. + await harness.deliver(DisconnectPeer(peer: camA, sender: nil)) + XCTAssertEqual(harness.fakeMP.invitedPeers.count, 2) + XCTAssertTrue(harness.lobby.failedPeers.isEmpty) + + // Second drop → reported failed, no third invite. + await harness.deliver(DisconnectPeer(peer: camA, sender: nil)) + XCTAssertEqual(harness.fakeMP.invitedPeers.count, 2, "no third invite") + XCTAssertEqual(harness.lobby.failedPeers, [camA]) + } + func testConnectInvitesWithLongTimeout() async { await seedScanning() await harness.deliver(ConnectToDevice(peer: harness.peer, sender: nil)) @@ -260,6 +318,177 @@ class SessionCoordinatorTests: XCTestCase { XCTAssertEqual(frameRequests[0].mode, .reliable) } + /// Seam B: a frame's ack goes only to the camera that sent it. With two + /// peers connected, a broadcast ack would advance the credit window of a + /// camera whose frame was never consumed. + func testFrameAckTargetsOnlyTheSendingPeer() async { + let secondCamera = MCPeerID(displayName: "SecondCamera") + await enterMonitor(.Photo) + harness.fakeMP.connectedPeers.append(secondCamera) + harness.fakeMP.sendResult = true + + await harness.deliver(RemoteCmd.OnFrame( + data: Data([1, 2, 3]), sender: nil, peerId: secondCamera, + fps: 30, camPosition: .back, camOrientation: .portrait, + codec: .jpeg, sequenceNumber: 1)) + + let acks = sent(RemoteCmd.RequestFrame.self) + XCTAssertEqual(acks.count, 1) + XCTAssertEqual(acks[0].peers, [secondCamera], + "ack must address the frame's source, not all connected peers") + + harness.fakeMP.sentMessages.removeAll() + await harness.deliver(RemoteCmd.OnFrame( + data: Data([4, 5, 6]), sender: nil, peerId: harness.peer, + fps: 30, camPosition: .back, camOrientation: .portrait, + codec: .jpeg, sequenceNumber: 2)) + XCTAssertEqual(sent(RemoteCmd.RequestFrame.self).map(\.peers), [[harness.peer]]) + } + + /// A camera answers a clock-sync ping immediately, from any state, with + /// the pong addressed to the pinging peer — off the actor inbox so the + /// timestamp isn't smeared by queued state-machine work. + func testClockSyncPingIsAnsweredDirectlyToTheSource() async { + harness.coordinator.didReceiveMessage( + RemoteCmd.ClockSyncPing(t0Millis: 424_242), from: harness.peer) + + let pongs = sent(RemoteCmd.ClockSyncPong.self) + XCTAssertEqual(pongs.count, 1) + XCTAssertEqual(pongs[0].peers, [harness.peer]) + XCTAssertEqual((pongs[0].msg as? RemoteCmd.ClockSyncPong)?.echoT0Millis, 424_242) + XCTAssertGreaterThan( + (pongs[0].msg as? RemoteCmd.ClockSyncPong)?.cameraClockMillis ?? 0, 0) + } + + // MARK: - Scheduled (multicam) capture, camera side + + func testScheduledCaptureInThePastNacks() async { + await enterCamera() + harness.fakeMP.sendResult = true + + await harness.deliver(RemoteCmd.ScheduledCapture( + fireAtCameraClockMillis: 1, // long past + anchorMillis: 1, captureId: "CAP-1", sessionId: "S", cameraIndex: 1)) + + let acks = sent(RemoteCmd.ScheduledCaptureAck.self) + .compactMap { $0.msg as? RemoteCmd.ScheduledCaptureAck } + XCTAssertEqual(acks.count, 1) + XCTAssertEqual(acks[0].captureId, "CAP-1") + XCTAssertNotNil(acks[0].error, "a fire time in the past is refused") + XCTAssertTrue(camera.takePictureCalls.isEmpty, "no shutter on a nack") + } + + func testValidScheduledCaptureAcksImmediatelyAndFires() async { + await enterCamera() + harness.fakeMP.sendResult = true + + // Fire ~now: acked immediately, then the shutter pulls a moment later. + await harness.deliver(RemoteCmd.ScheduledCapture( + fireAtCameraClockMillis: SyncClock.nowMillis(), + anchorMillis: SyncClock.nowMillis(), captureId: "CAP-2", + sessionId: "S", cameraIndex: 2)) + + let acks = sent(RemoteCmd.ScheduledCaptureAck.self) + .compactMap { $0.msg as? RemoteCmd.ScheduledCaptureAck } + XCTAssertEqual(acks.map(\.captureId), ["CAP-2"]) + XCTAssertNil(acks[0].error, "accepted") + + // The fire is enqueued by an off-actor delay task; wait for it. + for _ in 0..<200 where camera.takePictureCalls.isEmpty { + try? await Task.sleep(nanoseconds: 10_000_000) + } + XCTAssertEqual(camera.takePictureCalls, [true], + "the scheduled shutter fires, saving locally AND returning the still to the director") + } + + // MARK: - Resilient camera (multicam only) + + /// In a multicam session the camera keeps recording through a director + /// drop — the rest of the rig is still rolling. It enters the reconnect + /// path without stopping the clip. + func testMulticamDisconnectMidRecordingKeepsRecording() async { + await enterCamera() + harness.fakeMP.sendResult = true + + // A real scheduled start latches the multicam session and rolls tape. + await harness.deliver(RemoteCmd.ScheduledStartRecording( + fireAtCameraClockMillis: SyncClock.nowMillis(), + anchorMillis: SyncClock.nowMillis(), + captureId: "R", sessionId: "S", cameraIndex: 1)) + let latched = await harness.coordinator.inMulticamSessionForTesting() + XCTAssertTrue(latched) + + for _ in 0..<200 where camera.startRecordingCalls == 0 { + try? await Task.sleep(nanoseconds: 10_000_000) + } + XCTAssertEqual(camera.startRecordingCalls, 1) + await harness.coordinator.waitForIdle() + var name = await harness.stateName() + XCTAssertEqual(name, .cameraRecordingVideo) + + // Director drops. + harness.fakeMP.connectedPeers = [] + await harness.deliver(DisconnectPeer(peer: harness.peer, sender: nil)) + + XCTAssertTrue(camera.stopRecordingCalls.isEmpty, + "the clip keeps rolling through a director drop in multicam") + name = await harness.stateName() + XCTAssertEqual(name, .reconnecting) + } + + /// A single-camera session is unchanged: a disconnect mid-recording stops + /// the clip exactly as before (the resilient behavior must not leak here). + func testSingleCamDisconnectMidRecordingStops() async { + await enterCamera() + harness.fakeMP.sendResult = true + + await harness.deliver(RemoteCmd.StartRecordingVideo(sender: nil)) + let name = await harness.stateName() + XCTAssertEqual(name, .cameraRecordingVideo) + let latched = await harness.coordinator.inMulticamSessionForTesting() + XCTAssertFalse(latched, "no multicam command arrived") + + harness.fakeMP.connectedPeers = [] + await harness.deliver(DisconnectPeer(peer: harness.peer, sender: nil)) + + XCTAssertEqual(camera.stopRecordingCalls, [false], + "single-camera recording still stops on disconnect") + } + + /// A scheduled start latches the session and stamps the recording with sync + /// metadata; a scheduled stop later fires the stop. + func testScheduledRecordingStampsMetadataAndFires() async { + await enterCamera() + harness.fakeMP.sendResult = true + + await harness.deliver(RemoteCmd.ScheduledStartRecording( + fireAtCameraClockMillis: SyncClock.nowMillis(), + anchorMillis: 42, captureId: "R7", sessionId: "S3", cameraIndex: 4)) + + let acks = sent(RemoteCmd.ScheduledRecordingAck.self) + .compactMap { $0.msg as? RemoteCmd.ScheduledRecordingAck } + XCTAssertEqual(acks.map(\.captureId), ["R7"]) + XCTAssertFalse(acks[0].isStop) + + for _ in 0..<200 where camera.startRecordingCalls == 0 { + try? await Task.sleep(nanoseconds: 10_000_000) + } + // The recording carries its sync metadata (anchor as opaque key). + XCTAssertEqual(camera.videoSyncMetadata?.captureID, "R7") + XCTAssertEqual(camera.videoSyncMetadata?.cameraIndex, 4) + XCTAssertEqual(camera.videoSyncMetadata?.anchorMillis, 42) + + // A scheduled stop fires the stop. + await harness.deliver(RemoteCmd.ScheduledStopRecording( + fireAtCameraClockMillis: SyncClock.nowMillis(), + anchorMillis: 42, captureId: "R7", sessionId: "S3", cameraIndex: 4)) + for _ in 0..<200 where camera.stopRecordingCalls.isEmpty { + try? await Task.sleep(nanoseconds: 10_000_000) + } + XCTAssertEqual(camera.stopRecordingCalls, [true], + "scheduled stop saves locally AND pushes the clip to the director") + } + func testMonitorPhotoModeUnbecomeMonitorPopsToConnected() async { await enterMonitor(.Photo) await harness.deliver(UICmd.UnbecomeMonitor(sender: nil)) diff --git a/RemoteCamTests/RemoteCmdSerializationTests.swift b/RemoteCamTests/RemoteCmdSerializationTests.swift index a045eccf..e154d147 100644 --- a/RemoteCamTests/RemoteCmdSerializationTests.swift +++ b/RemoteCamTests/RemoteCmdSerializationTests.swift @@ -47,6 +47,15 @@ final class RemoteCmdSerializationTests: XCTestCase { case let m as RemoteCmd.SendFrame: return m.toFlatBuffer() case let m as RemoteCmd.RequestFrame: return m.toFlatBuffer() case let m as RemoteCmd.RequestKeyframe: return m.toFlatBuffer() + case let m as RemoteCmd.ClockSyncPing: return m.toFlatBuffer() + case let m as RemoteCmd.ClockSyncPong: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledCapture: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledCaptureAck: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledStartRecording: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledStopRecording: return m.toFlatBuffer() + case let m as RemoteCmd.ScheduledRecordingAck: return m.toFlatBuffer() + case let m as RemoteCmd.SetStreamProfile: return m.toFlatBuffer() + case let m as RemoteCmd.RequestVideoResend: return m.toFlatBuffer() case let m as RemoteCmd.SetZoom: return m.toFlatBuffer() case let m as RemoteCmd.SetZoomResp: return m.toFlatBuffer() case let m as RemoteCmd.FocusAtPoint: return m.toFlatBuffer() @@ -1182,4 +1191,104 @@ extension RemoteCmdSerializationTests { XCTAssertFalse(result.supportsPreviewMode) XCTAssertEqual(result.previewMode, .on) } + + func testClockSyncPing_roundTrip() { + let result = roundTrip(RemoteCmd.ClockSyncPing(t0Millis: 987_654_321_012)) + XCTAssertEqual(result.t0Millis, 987_654_321_012) + } + + func testClockSyncPong_roundTrip() { + let result = roundTrip(RemoteCmd.ClockSyncPong( + echoT0Millis: 987_654_321_012, cameraClockMillis: 123_456_789_345)) + XCTAssertEqual(result.echoT0Millis, 987_654_321_012) + XCTAssertEqual(result.cameraClockMillis, 123_456_789_345) + } + + func testScheduledCapture_roundTrip() { + let result = roundTrip(RemoteCmd.ScheduledCapture( + fireAtCameraClockMillis: 1_754_800_000_123, + anchorMillis: 1_754_800_000_000, + captureId: "CAP-123", + sessionId: "SESS-9", + cameraIndex: 3)) + XCTAssertEqual(result.fireAtCameraClockMillis, 1_754_800_000_123) + XCTAssertEqual(result.anchorMillis, 1_754_800_000_000) + XCTAssertEqual(result.captureId, "CAP-123") + XCTAssertEqual(result.sessionId, "SESS-9") + XCTAssertEqual(result.cameraIndex, 3) + } + + func testScheduledCaptureAck_roundTrip() { + let ok = roundTrip(RemoteCmd.ScheduledCaptureAck(captureId: "CAP-42")) + XCTAssertEqual(ok.captureId, "CAP-42") + XCTAssertNil(ok.error) + + let nack = roundTrip(RemoteCmd.ScheduledCaptureAck( + captureId: "CAP-43", + error: NSError(domain: "too late", code: 0))) + XCTAssertEqual(nack.captureId, "CAP-43") + XCTAssertNotNil(nack.error) + } + + func testScheduledStartRecording_roundTrip() { + let result = roundTrip(RemoteCmd.ScheduledStartRecording( + fireAtCameraClockMillis: 111, anchorMillis: 100, + captureId: "REC-1", sessionId: "S", cameraIndex: 2)) + XCTAssertEqual(result.fireAtCameraClockMillis, 111) + XCTAssertEqual(result.anchorMillis, 100) + XCTAssertEqual(result.captureId, "REC-1") + XCTAssertEqual(result.cameraIndex, 2) + } + + func testScheduledStopRecording_roundTrip() { + let result = roundTrip(RemoteCmd.ScheduledStopRecording( + fireAtCameraClockMillis: 222, anchorMillis: 200, + captureId: "REC-1", sessionId: "S", cameraIndex: 2)) + XCTAssertEqual(result.fireAtCameraClockMillis, 222) + XCTAssertEqual(result.captureId, "REC-1") + } + + /// The start/stop distinction (`isStop`) rides the response action, so the + /// director routes each ack to the right aggregation. + func testScheduledRecordingAck_roundTripPreservesIsStop() { + let start = roundTrip(RemoteCmd.ScheduledRecordingAck(captureId: "R", isStop: false)) + XCTAssertFalse(start.isStop) + XCTAssertEqual(start.captureId, "R") + + let stop = roundTrip(RemoteCmd.ScheduledRecordingAck(captureId: "R", isStop: true)) + XCTAssertTrue(stop.isStop) + + let nack = roundTrip(RemoteCmd.ScheduledRecordingAck( + captureId: "R", isStop: false, error: NSError(domain: "x", code: 0))) + XCTAssertNotNil(nack.error) + } + + func testSetStreamProfile_roundTrip() { + let result = roundTrip(RemoteCmd.SetStreamProfile( + maxLongEdge: 640, bitrateKbps: 500, fps: 20)) + XCTAssertEqual(result.maxLongEdge, 640) + XCTAssertEqual(result.bitrateKbps, 500) + XCTAssertEqual(result.fps, 20) + } + + func testRequestVideoResend_roundTrip() { + let result = roundTrip(RemoteCmd.RequestVideoResend(captureId: "R7")) + XCTAssertEqual(result.captureId, "R7") + } + + /// The multicam capability survives the wire, and a peer that predates it + /// (absent field) decodes as not-multicam-capable. + func testCapabilitiesCarryMulticamSupport() { + let caps = RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: nil, + currentCamera: .back, currentLens: .wideAngle, currentZoom: 1.0, + supportsMulticam: true, error: nil) + XCTAssertTrue(roundTrip(caps).supportsMulticam) + + let legacy = RemoteCmd.CameraCapabilitiesResp( + frontCamera: nil, backCamera: nil, + currentCamera: .back, currentLens: .wideAngle, currentZoom: 1.0, + error: nil) + XCTAssertFalse(roundTrip(legacy).supportsMulticam) + } } diff --git a/RemoteCamTests/RigQualityMenuTests.swift b/RemoteCamTests/RigQualityMenuTests.swift new file mode 100644 index 00000000..c0321061 --- /dev/null +++ b/RemoteCamTests/RigQualityMenuTests.swift @@ -0,0 +1,119 @@ +// +// RigQualityMenuTests.swift +// RemoteShutterTests +// +// Copyright © 2026 Security Union LLC. All rights reserved. +// + +import XCTest +@testable import RemoteShutter + +final class RigQualityMenuTests: XCTestCase { + + /// A CameraInfo advertising a resolution/fps matrix (+ HEIF/HDR). + private func info(_ matrix: [VideoResolution: [VideoFrameRate]], + heif: Bool = true, hdr: Bool = true) -> RemoteCmd.CameraInfo { + RemoteCmd.CameraInfo( + availableLenses: [.wideAngle], hasFlash: true, hasTorch: true, + zoomCapabilities: [:], + supportedResolutions: Array(matrix.keys), + supportedFrameRates: Array(Set(matrix.values.flatMap { $0 })), + resolutionFrameRates: matrix, + supportsHEIF: heif, supportsHDR: hdr) + } + + private func lane(_ name: String, _ matrix: [VideoResolution: [VideoFrameRate]], + heif: Bool = true, hdr: Bool = true) -> RigQualityMenu.Lane { + RigQualityMenu.Lane(name: name, info: info(matrix, heif: heif, hdr: hdr)) + } + + private let full4K: [VideoResolution: [VideoFrameRate]] = + [.uhd4k: [.fps24, .fps30, .fps60], .hd1080p: [.fps24, .fps30, .fps60]] + private let only1080: [VideoResolution: [VideoFrameRate]] = + [.hd1080p: [.fps24, .fps30]] + + // MARK: - Intersection + + func testHomogeneousRigOffersEverything() { + let menu = RigQualityMenu(lanes: [lane("Cam 1", full4K), lane("Cam 2", full4K)]) + let opts = Set(menu.videoOptions().map { "\($0.resolution.rawValue):\($0.frameRate.rawValue)" }) + XCTAssertTrue(opts.contains("2:3")) // 4K60 + XCTAssertTrue(opts.contains("1:2")) // 1080p30 + XCTAssertTrue(menu.blockingLanes(resolution: .uhd4k, frameRate: .fps60).isEmpty) + } + + func testHeterogeneousRigIntersectsAndNamesBlocker() { + // Cam 2 can only do 1080p{24,30}. The rig loses 4K and 1080p60. + let menu = RigQualityMenu(lanes: [lane("Cam 1", full4K), lane("Cam 2", only1080)]) + XCTAssertEqual(menu.blockingLanes(resolution: .uhd4k, frameRate: .fps30), ["Cam 2"]) + XCTAssertEqual(menu.blockingLanes(resolution: .hd1080p, frameRate: .fps60), ["Cam 2"]) + // 1080p30 survives (both do it). + XCTAssertTrue(menu.blockingLanes(resolution: .hd1080p, frameRate: .fps30).isEmpty) + } + + /// Dario's clarification: manual selection within the intersection is + /// first-class — both 1080p30 and 4K30 must be enabled for a rig where both + /// cameras do both, so the tray can toggle directly between them. + func testManualOptionsBothEnabledWhenSharedByAllCameras() { + let shared: [VideoResolution: [VideoFrameRate]] = [.uhd4k: [.fps30], .hd1080p: [.fps30]] + let menu = RigQualityMenu(lanes: [lane("Cam 1", shared), lane("Cam 2", shared)]) + XCTAssertTrue(menu.blockingLanes(resolution: .hd1080p, frameRate: .fps30).isEmpty) + XCTAssertTrue(menu.blockingLanes(resolution: .uhd4k, frameRate: .fps30).isEmpty) + } + + // MARK: - Automatic + + func testAutomaticPicksHighestResThenFps() { + let menu = RigQualityMenu(lanes: [lane("Cam 1", full4K), lane("Cam 2", full4K)]) + let auto = menu.automaticVideo() + XCTAssertEqual(auto.resolution, .uhd4k) + XCTAssertEqual(auto.frameRate, .fps60) + } + + func testAutomaticFallsBackToFloorOnEmptyIntersection() { + // Cam 1 does 4K only; Cam 2 does 1080p only → nothing shared but the + // floor, so Automatic is 1080p30. + let menu = RigQualityMenu(lanes: [ + lane("Cam 1", [.uhd4k: [.fps30]]), + lane("Cam 2", [.hd1080p: [.fps30]])]) + let auto = menu.automaticVideo() + XCTAssertEqual(auto.resolution, RigQualityMenu.floor.resolution) + XCTAssertEqual(auto.frameRate, RigQualityMenu.floor.frameRate) + } + + func testEmptyRigAutomaticIsFloor() { + let menu = RigQualityMenu(lanes: []) + XCTAssertEqual(menu.automaticVideo().resolution, .hd1080p) + XCTAssertEqual(menu.automaticVideo().frameRate, .fps30) + } + + // MARK: - Photo + + func testPhotoIntersection() { + let both = RigQualityMenu(lanes: [lane("A", only1080, heif: true, hdr: true), + lane("B", only1080, heif: true, hdr: true)]) + XCTAssertTrue(both.supportsHEIF()) + XCTAssertTrue(both.supportsHDR()) + XCTAssertEqual(both.automaticPhoto().format, .heif) + XCTAssertEqual(both.automaticPhoto().hdr, .on) + + let mixed = RigQualityMenu(lanes: [lane("A", only1080, heif: true, hdr: false), + lane("B", only1080, heif: false, hdr: true)]) + XCTAssertFalse(mixed.supportsHEIF()) + XCTAssertFalse(mixed.supportsHDR()) + XCTAssertEqual(mixed.lanesBlockingHEIF(), ["B"]) + XCTAssertEqual(mixed.lanesBlockingHDR(), ["A"]) + XCTAssertEqual(mixed.automaticPhoto().format, .jpeg) + } + + // MARK: - Late joiner + + func testLateJoinerThatCannotMatchIsDetected() { + let menu = RigQualityMenu(lanes: [lane("Cam 1", full4K), lane("Cam 2", only1080)]) + // The rig is running at 4K30; Cam 2 (1080-only) can't match. + XCTAssertFalse(menu.laneCanMatch(lane("Cam 2", only1080), + resolution: .uhd4k, frameRate: .fps30)) + XCTAssertTrue(menu.laneCanMatch(lane("Cam 1", full4K), + resolution: .uhd4k, frameRate: .fps30)) + } +} diff --git a/RemoteCamTests/SessionTestSupport.swift b/RemoteCamTests/SessionTestSupport.swift index 90b60dd7..c5ebd765 100644 --- a/RemoteCamTests/SessionTestSupport.swift +++ b/RemoteCamTests/SessionTestSupport.swift @@ -91,10 +91,14 @@ class FakeScannerLobby: ScannerLobby, @unchecked Sendable { var roleScreenShown = 0 var returnsToLobby = 0 var scanningErrors = 0 + var collectedReports: [[MCPeerID]] = [] + var failedPeers: [MCPeerID] = [] func goToRole() { roleScreenShown += 1 } func returnToLobby() { returnsToLobby += 1 } func presentScanningError() { scanningErrors += 1 } + func didCollectMulticamCameras(_ peers: [MCPeerID]) { collectedReports.append(peers) } + func didFailMulticamCamera(_ peer: MCPeerID) { failedPeers.append(peer) } } // MARK: - Fake camera (CameraControlling) @@ -133,6 +137,10 @@ class FakeCameraControlling: CameraControlling, @unchecked Sendable { func takePicture(_ sendMediaToRemote: Bool) { takePictureCalls.append(sendMediaToRemote) } func startRecordingVideo() { startRecordingCalls += 1 } func stopRecordingVideo(_ shouldSendVideo: Bool) { stopRecordingCalls.append(shouldSendVideo) } + var videoSyncMetadata: CaptureSyncMetadata? + func setVideoSyncMetadata(_ metadata: CaptureSyncMetadata?) { videoSyncMetadata = metadata } + var appliedProfiles: [StreamProfile] = [] + func applyStreamProfile(_ profile: StreamProfile) { appliedProfiles.append(profile) } // swiftlint:disable:next large_tuple func setZoom(zoomFactor: CGFloat) async throws -> (CGFloat, CameraLensType, RemoteCmd.ZoomRange) { diff --git a/RemoteCamTests/StoreManagerTests.swift b/RemoteCamTests/StoreManagerTests.swift index 9aa6f0c6..cb3d103f 100644 --- a/RemoteCamTests/StoreManagerTests.swift +++ b/RemoteCamTests/StoreManagerTests.swift @@ -46,6 +46,22 @@ final class StoreManagerTests: XCTestCase { XCTAssertFalse(store.hasTapToFocusFeature()) } + // MARK: - Multicam camera cap + + func testFreeTierGetsTwoCameras() { + XCTAssertEqual(StoreManager.shared.maxCameras(), 2) + } + + func testProModeUnlocksFourCameras() { + UserDefaults.standard.set(true, forKey: proModeKey) + XCTAssertEqual(StoreManager.shared.maxCameras(), 4) + } + + func testProSubscriptionUnlocksFourCameras() { + UserDefaults.standard.set(true, forKey: proSubscriptionKey) + XCTAssertEqual(StoreManager.shared.maxCameras(), 4) + } + // MARK: - Individual Feature Flags func testHasAdRemovalWhenPurchased() { diff --git a/RemoteCamTests/StormoLoopbackTests.swift b/RemoteCamTests/StormoLoopbackTests.swift index 9a4a1f7f..0b424b36 100644 --- a/RemoteCamTests/StormoLoopbackTests.swift +++ b/RemoteCamTests/StormoLoopbackTests.swift @@ -57,7 +57,7 @@ final class StormoLoopbackTests: XCTestCase { func peerDidConnect(_ peer: PeerID) { connected.fulfill() } - func didReceiveMessage(_ message: Message) { + func didReceiveMessage(_ message: Message, from peer: PeerID) { lock.lock(); _receivedMessages.append(message); lock.unlock() messageReceived.fulfill() } @@ -67,8 +67,8 @@ final class StormoLoopbackTests: XCTestCase { func didReceiveFrameRequest(_ request: RemoteCmd.RequestFrame) {} func didReceiveFrame(_ frame: RemoteCmd.SendFrame, from peer: PeerID) {} func didDetectIncompatibility() {} - func didStartReceivingResource(name: String, progress: Progress) {} - func didFinishReceivingResource(name: String, at localURL: URL?, error: Error?) {} + func didStartReceivingResource(name: String, from peer: PeerID, progress: Progress) {} + func didFinishReceivingResource(name: String, from peer: PeerID, at localURL: URL?, error: Error?) {} func browserDidLosePeer(_ peer: PeerID) {} func browserDidFail(_ error: Error) {} func advertiserDidFail(_ error: Error) {} diff --git a/RemoteShutter.xcodeproj/project.pbxproj b/RemoteShutter.xcodeproj/project.pbxproj index 223ad010..8abfb2ec 100644 --- a/RemoteShutter.xcodeproj/project.pbxproj +++ b/RemoteShutter.xcodeproj/project.pbxproj @@ -201,6 +201,21 @@ FADEC0DE0002000000000002 /* PeerLinkStatus.swift in Sources */ = {isa = PBXBuildFile; fileRef = FADEC0DE0002000000000001 /* PeerLinkStatus.swift */; }; FC0CF5A101FE65A9800F0B238 /* FocusPointMapping.swift in Sources */ = {isa = PBXBuildFile; fileRef = FC0CF5A201FE65A9800F0B238 /* FocusPointMapping.swift */; }; CAFEBABE0120000000000001 /* MonitorChrome.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0120000000000002 /* MonitorChrome.swift */; }; + CAFEBABE0130000000000001 /* CaptureSyncMetadata.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0130000000000002 /* CaptureSyncMetadata.swift */; }; + CAFEBABE0132000000000001 /* ClockOffsetEstimator.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0132000000000002 /* ClockOffsetEstimator.swift */; }; + CAFEBABE0160000000000001 /* RigQualityMenu.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0160000000000002 /* RigQualityMenu.swift */; }; + CAFEBABE0161000000000002 /* RigQualityMenuTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0161000000000001 /* RigQualityMenuTests.swift */; }; + CAFEBABE0150000000000001 /* MultiCamChrome.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0150000000000002 /* MultiCamChrome.swift */; }; + CAFEBABE0151000000000002 /* MultiCamChromeTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0151000000000001 /* MultiCamChromeTests.swift */; }; + CAFEBABE0140000000000001 /* CameraLink.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0140000000000002 /* CameraLink.swift */; }; + CAFEBABE0141000000000001 /* MulticamController.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0141000000000002 /* MulticamController.swift */; }; + CAFEBABE0142000000000001 /* MulticamViewModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0142000000000002 /* MulticamViewModel.swift */; }; + CAFEBABE0143000000000001 /* MulticamView.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0143000000000002 /* MulticamView.swift */; }; + CAFEBABE0144000000000001 /* MulticamViewController.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0144000000000002 /* MulticamViewController.swift */; }; + CAFEBABE0145000000000002 /* MulticamControllerTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0145000000000001 /* MulticamControllerTests.swift */; }; + CAFEBABE0146000000000002 /* MulticamViewModelTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0146000000000001 /* MulticamViewModelTests.swift */; }; + CAFEBABE0133000000000002 /* ClockOffsetEstimatorTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0133000000000001 /* ClockOffsetEstimatorTests.swift */; }; + CAFEBABE0131000000000002 /* CaptureSyncMetadataTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = CAFEBABE0131000000000001 /* CaptureSyncMetadataTests.swift */; }; FEEDFACE0000000000000001 /* StormoLoopbackTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = FEEDFACE0000000000000002 /* StormoLoopbackTests.swift */; }; /* End PBXBuildFile section */ @@ -442,6 +457,21 @@ FADEC0DE0002000000000001 /* PeerLinkStatus.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = PeerLinkStatus.swift; sourceTree = ""; }; FC0CF5A201FE65A9800F0B238 /* FocusPointMapping.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = FocusPointMapping.swift; sourceTree = ""; }; CAFEBABE0120000000000002 /* MonitorChrome.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MonitorChrome.swift; sourceTree = ""; }; + CAFEBABE0130000000000002 /* CaptureSyncMetadata.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CaptureSyncMetadata.swift; sourceTree = ""; }; + CAFEBABE0132000000000002 /* ClockOffsetEstimator.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = ClockOffsetEstimator.swift; sourceTree = ""; }; + CAFEBABE0160000000000002 /* RigQualityMenu.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = RigQualityMenu.swift; sourceTree = ""; }; + CAFEBABE0161000000000001 /* RigQualityMenuTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = RigQualityMenuTests.swift; sourceTree = ""; }; + CAFEBABE0150000000000002 /* MultiCamChrome.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MultiCamChrome.swift; sourceTree = ""; }; + CAFEBABE0151000000000001 /* MultiCamChromeTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MultiCamChromeTests.swift; sourceTree = ""; }; + CAFEBABE0140000000000002 /* CameraLink.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = CameraLink.swift; sourceTree = ""; }; + CAFEBABE0141000000000002 /* MulticamController.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MulticamController.swift; sourceTree = ""; }; + CAFEBABE0142000000000002 /* MulticamViewModel.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MulticamViewModel.swift; sourceTree = ""; }; + CAFEBABE0143000000000002 /* MulticamView.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MulticamView.swift; sourceTree = ""; }; + CAFEBABE0144000000000002 /* MulticamViewController.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = MulticamViewController.swift; sourceTree = ""; }; + CAFEBABE0145000000000001 /* MulticamControllerTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MulticamControllerTests.swift; sourceTree = ""; }; + CAFEBABE0146000000000001 /* MulticamViewModelTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MulticamViewModelTests.swift; sourceTree = ""; }; + CAFEBABE0133000000000001 /* ClockOffsetEstimatorTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ClockOffsetEstimatorTests.swift; sourceTree = ""; }; + CAFEBABE0131000000000001 /* CaptureSyncMetadataTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CaptureSyncMetadataTests.swift; sourceTree = ""; }; FCB0F6BA2086BF9BB4242D66 /* Pods_RemoteShutterWatch.framework */ = {isa = PBXFileReference; explicitFileType = wrapper.framework; includeInIndex = 0; path = Pods_RemoteShutterWatch.framework; sourceTree = BUILT_PRODUCTS_DIR; }; FEEDFACE0000000000000002 /* StormoLoopbackTests.swift */ = {isa = PBXFileReference; includeInIndex = 1; lastKnownFileType = sourcecode.swift; path = StormoLoopbackTests.swift; sourceTree = ""; }; /* End PBXFileReference section */ @@ -591,6 +621,12 @@ CAFEBABE0001000000000001 /* CropRectTests.swift */, CAFEBABE00F0000000000001 /* FocusPointMappingTests.swift */, CAFEBABE0121000000000001 /* MonitorChromeTests.swift */, + CAFEBABE0131000000000001 /* CaptureSyncMetadataTests.swift */, + CAFEBABE0133000000000001 /* ClockOffsetEstimatorTests.swift */, + CAFEBABE0161000000000001 /* RigQualityMenuTests.swift */, + CAFEBABE0151000000000001 /* MultiCamChromeTests.swift */, + CAFEBABE0145000000000001 /* MulticamControllerTests.swift */, + CAFEBABE0146000000000001 /* MulticamViewModelTests.swift */, CAFEBABE0002000000000001 /* WatchCaptureCountdownTests.swift */, CAFEBABE0003000000000001 /* WatchSerializationTests.swift */, CAFEBABE0006000000000001 /* WatchPreviewStreamerTests.swift */, @@ -683,6 +719,15 @@ 0684A2D01BE65A9800F0B238 /* OrientationUtils.swift */, FC0CF5A201FE65A9800F0B238 /* FocusPointMapping.swift */, CAFEBABE0120000000000002 /* MonitorChrome.swift */, + CAFEBABE0130000000000002 /* CaptureSyncMetadata.swift */, + CAFEBABE0132000000000002 /* ClockOffsetEstimator.swift */, + CAFEBABE0160000000000002 /* RigQualityMenu.swift */, + CAFEBABE0150000000000002 /* MultiCamChrome.swift */, + CAFEBABE0140000000000002 /* CameraLink.swift */, + CAFEBABE0141000000000002 /* MulticamController.swift */, + CAFEBABE0142000000000002 /* MulticamViewModel.swift */, + CAFEBABE0143000000000002 /* MulticamView.swift */, + CAFEBABE0144000000000002 /* MulticamViewController.swift */, 060B1E141BE7079800077BCC /* Helpers */, 06E965202535199400E5A8B3 /* MediaProcessors.swift */, 06E9652625351E3F00E5A8B3 /* SwiftConstants.swift */, @@ -1175,6 +1220,15 @@ 0684A2D11BE65A9800F0B238 /* OrientationUtils.swift in Sources */, FC0CF5A101FE65A9800F0B238 /* FocusPointMapping.swift in Sources */, CAFEBABE0120000000000001 /* MonitorChrome.swift in Sources */, + CAFEBABE0130000000000001 /* CaptureSyncMetadata.swift in Sources */, + CAFEBABE0132000000000001 /* ClockOffsetEstimator.swift in Sources */, + CAFEBABE0160000000000001 /* RigQualityMenu.swift in Sources */, + CAFEBABE0150000000000001 /* MultiCamChrome.swift in Sources */, + CAFEBABE0140000000000001 /* CameraLink.swift in Sources */, + CAFEBABE0141000000000001 /* MulticamController.swift in Sources */, + CAFEBABE0142000000000001 /* MulticamViewModel.swift in Sources */, + CAFEBABE0143000000000001 /* MulticamView.swift in Sources */, + CAFEBABE0144000000000001 /* MulticamViewController.swift in Sources */, 06BB79B82E374D410094E085 /* FeatureFlags.swift in Sources */, 0692844F1BE5C0E600AF4678 /* MultipeerMessages.swift in Sources */, 069284531BE5C0E600AF4678 /* RemoteCamStateNames.swift in Sources */, @@ -1247,6 +1301,12 @@ CAFEBABE0001000000000002 /* CropRectTests.swift in Sources */, CAFEBABE00F0000000000002 /* FocusPointMappingTests.swift in Sources */, CAFEBABE0121000000000002 /* MonitorChromeTests.swift in Sources */, + CAFEBABE0131000000000002 /* CaptureSyncMetadataTests.swift in Sources */, + CAFEBABE0133000000000002 /* ClockOffsetEstimatorTests.swift in Sources */, + CAFEBABE0161000000000002 /* RigQualityMenuTests.swift in Sources */, + CAFEBABE0151000000000002 /* MultiCamChromeTests.swift in Sources */, + CAFEBABE0145000000000002 /* MulticamControllerTests.swift in Sources */, + CAFEBABE0146000000000002 /* MulticamViewModelTests.swift in Sources */, CAFEBABE0002000000000002 /* WatchCaptureCountdownTests.swift in Sources */, CAFEBABE0003000000000002 /* WatchSerializationTests.swift in Sources */, CAFEBABE0006000000000002 /* WatchPreviewStreamerTests.swift in Sources */,