🖥️ MacDeck

Low-Latency Remote Desktop & Element-Snapping Controller for Mac & iPhone

Built natively with Swift 5.9, Xcode, ScreenCaptureKit, VideoToolbox & Network.framework

MacDeck is a high-performance, low-latency remote desktop and Mac control suite designed for iPhone. Built natively in Swift for macOS 14+ and iOS 17+, it leverages hardware-accelerated H.264 encoding via VideoToolbox, real-time display capture via ScreenCaptureKit, encrypted local discovery via Network.framework, and AX Element Snapping.

---

🏗️ Workspace Targets

graph TD
    A["MacDeckHost (macOS 14+ Menu Bar App)"] <-->|Network.framework (Bonjour TLS)| B["MacDeck (iOS 17+ Client App)"]
    A --- C["MacDeckKit (Shared Swift Package)"]
    B --- C

Loading

1. MacDeckHost: macOS 14+ SwiftUI Menu Bar app that handles display/window capture, H.264 video encoding, CoreGraphics input injection, and Accessibility UI tree walking.
2. MacDeck: iOS 17+ Client SwiftUI app featuring Bonjour host discovery, hardware-accelerated video decoding, multi-touch trackpad, and interactive element snapping overlays.
3. MacDeckKit: Shared Swift Package defining packet framing, binary protocols, delta payloads, and element rect models to guarantee zero protocol drift.

---

⚡ Core Technical Features

• 📺 ScreenCaptureKit Streamer: Real-time display capture targeting 60 FPS, 32BGRA pixel format, queueDepth 5, resized directly to client display dimensions.
• 🚀 Ultra-Low-Latency H.264 Encoding: Uses VideoToolbox with kVTCompressionPropertyKey_AllowFrameReordering = false (eliminates B-frames for ~30ms end-to-end latency).
• 🔒 Encrypted Bonjour Discovery: Discovers hosts via _macdeck._tcp over Network.framework NWListener/NWBrowser using a 6-digit Preshared Key (TLS PSK).
• 🎯 Signature AX Element Snapping: Walks frontmost application UI trees (AXUIElement) for AXButton, AXTextField, AXMenuItem. Tapping near an element on iPhone snaps the cursor directly to its center.
• ⌨️ CoreGraphics Input Injection: Posts hid events (.cghidEventTap) for mouse deltas, clicks, pixel scrolling (wheelCount:2), and CGEventKeyboardSetUnicodeString text typing.
• ⚡ Hardware Decoding on iOS: Uses AVSampleBufferDisplayLayer with kCMSampleAttachmentKey_DisplayImmediately = true for zero-buffer display.

---

📋 Permissions Checklist

Permission	Platform	Reason
Screen Recording	macOS	Required by ScreenCaptureKit to stream screen
Accessibility (TCC)	macOS	Required for AXUIElement tree walking & CGEvent input injection
Local Network	iOS & macOS	Required by Network.framework for Bonjour discovery & socket stream

---

🛠️ How to Build & Run

1. Open the master workspace in Xcode:

   open MacDeck.xcworkspace

2. Select target MacDeckHost and run on your Mac (runs as a SwiftUI Menu Bar App).
3. Select target MacDeck and run on your iPhone or iOS Simulator.
4. Enter the 6-digit pairing code displayed in the Mac menu bar into the iPhone app to establish the TLS encrypted session!

---

📄 License

Licensed under the MIT License.