ard-rs

Pure Rust, platform-independent parsing and framebuffer decoding for Apple Remote Desktop (ARD) screen sharing.

This repository is a Cargo workspace with two packages. ard-core contains the protocol client, authentication, codecs, input messages, native RFB framebuffer bytes, and reconnect support without a GUI dependency. ard-viewer contains the current native viewer and is the only package that depends on windowing, GPU, and clipboard backends.

Native GPU viewer

ard-viewer is an interactive desktop viewer for macOS, Windows, and Linux. It connects directly to the ARD TCP service and supports mouse, keyboard, clipboard, and IME text input.

The pre-Iced standalone Winit/WGPU viewer is retained as an ard-core example:

cargo run -p ard-core --release --example ard_viewer -- \
  192.168.1.20:5900 username

The example prompts for the ARD password interactively without echoing it.

cargo build -p ard-viewer --release --bin ard-viewer
ARD_PASSWORD='screen-sharing-password' \
  cargo run -p ard-viewer --release --bin ard-viewer -- \
  192.168.1.20:5900 username

The viewer defaults to Apple adaptive MVS (1011) with GPU tile/DCT decoding. It also supports the native low, medium, and high profiles plus RDM-compatible full quality ([Zlib, ZRLE]):

# Maximum fidelity and bandwidth
ard-viewer --quality full 192.168.1.20:5900 username

# Apple MVS adaptive streaming and GPU tile decoding (the default)
ard-viewer --quality adaptive 192.168.1.20:5900 username

# macOS high-performance AVC/H.264 or HEVC over UDP/SRTP
ard-viewer --quality high-performance 192.168.1.20:5900 username

# Optional minimum server update interval; zero is native maximum rate
ard-viewer --frame-interval-ms 16 192.168.1.20:5900 username

High-performance mode advertises AVC (1010) as the preferred encoding, sends the media-stream offer, and decodes the negotiated video with macOS VideoToolbox or Windows Media Foundation Transforms. Keyboard, pointer, clipboard, and session control stay on the encrypted TCP RFB channel while video uses the negotiated UDP/RTP stream. This mode is strict: it does not advertise MVS/zlib as a visual fallback, ignores RFB visual rectangles while AVC negotiation is pending, and reports an explicit error on unsupported platforms or negotiation failure.

The four native media SSRCs are decoded in their shared DON/DONL order, with legitimate sparse bands preserved. UDP receive runs independently of hardware decode so bitrate bursts cannot starve the socket. Fixed logical modes request their exact 2x physical framebuffer from the initial update request onward; decoded dimensions and codec padding are validated rather than scaled or arbitrarily cropped. The optional performance HUD exposes RTP reassembly, DON reorder, receive/decode/render-command timing, input queue/write latency, and requested-versus-decoded resolution.

For a remote Mac behind explicit port forwarding, the connection form accepts independent external UDP overrides for audio, primary video, and secondary video. Empty fields use the ports advertised by the server. An override changes only the remote destination: the viewer continues to bind the original negotiated local port required by Screen Sharing. For example, a router mapping external UDP 15900-15902 to the Mac's UDP 5900-5902 should be configured as audio 15900, primary video 15901, and secondary video 15902.

After encrypted transport activation, the client requests and decodes one non-incremental full-frame baseline, then sends Apple's automatic frame-update subscription with a zero interval by default. This preserves MVS copy/cache state while letting screensharingd push later changes without waiting for a decode/render cycle. The viewer logs server update rate, actual encrypted inbound Mbit/s, and successfully presented display frames separately. Its window title stays stable so a viewer connected to the local ARD server does not create a screen-update feedback loop by changing the captured title every frame.

In adaptive mode, the CPU performs MVS state and Rice/Huffman parsing and emits bounded 8x8 tile commands with native DCT coefficients. A wgpu compute pipeline performs inverse DCT, chroma expansion, tile composition, and color conversion into a GPU-only presentation texture.

TCP -> authenticated decrypt -> MVS tile/DCT parser
    -> GPU storage buffers -> compute IDCT/tile expansion
    -> GPU presentation texture -> Metal / D3D12 / Vulkan

Input is normalized before it reaches the wire. Mouse coordinates are mapped from the letterboxed, DPI-scaled window to the remote framebuffer. Keyboard events use stable RFB/X11 keysyms: printable text follows the active host layout through Unicode input, while modifiers, navigation, function, and numpad keys use layout-independent symbols. The viewer sends and receives UTF-8 clipboard text, including committed IME text, on all supported desktop platforms.

The full, high, medium, and low profiles use the CPU decoder for Raw, Zlib, ZRLE, and Apple's three sub-zlib encodings. ard-core retains the negotiated RFB pixel bytes; the viewer converts them at its presentation boundary before uploading complete snapshots to the GPU texture. Pending full-frame snapshots are coalesced so a slow window cannot grow latency or memory without bound.

Reverse-engineering notes are in docs/SCREENSHARING_RE.md. A step-by-step playbook for reproducing the native-code investigation and oracle validation is in docs/REVERSE_ENGINEERING_PLAYBOOK.md.

ARD reuses RFB message framing, but it is not merely a normal VNC session. This crate implements Apple-specific protocol behavior directly and does not depend on a VNC library or a native operating-system library.

Implemented

• Apple protocol banner RFB 003.889
• Apple security-type recognition (30..=36) without pretending those methods are ordinary VNC authentication
• bounded parsing of Apple type-30 Diffie-Hellman parameters and encrypted credential responses
• pure-Rust construction of the Apple type-30 client exchange from caller-supplied random input
• Apple ClientInit flag-byte and client message-10 session-options parsing
• bounded parsing of the 66-byte RFBViewerInformation (0x21) message
• byte-exact construction and bounded parsing of the RFBSetEncryptionLevel (0x12) proposal and its eight-byte activation record, matching the installed client and screensharingd handler
• explicit handling of the zero-sized encryption-control rectangle (1103 / 0x044f)
• incremental, transactional encrypted-record framing across arbitrary TCP fragmentation, with persistent AES-128-CBC state, implicit sequence validation, and SHA-1 verification before plaintext is returned
• a bounded incremental dispatcher that turns verified record payloads into server messages, routing FramebufferUpdate (including MVS 1011) rectangles into the persistent decoder state and exposing 1103 controls
• a pure-Rust encrypted-transport oracle server that completes the type-30 exchange, sends a real 1103 control rectangle, validates the client's activation and automatic-update subscription, and exchanges AES-CBC records carrying either MVS or persistent full-colour zlib frames
• bounded parsing of security offers, ServerInit, and FramebufferUpdate
• Apple's extended ServerInit command-support block, including the 0x12-advertising bitfield that gates the encrypted transport
• client message generation for pixel format, encodings, and update requests
• Apple's server-driven automatic frame-update message (0x09), including a configurable interval and the native zero-interval maximum-rate default
• RDM-compatible low, medium, high, adaptive MVS, and full-quality encoding profiles
• raw and full-colour zlib rectangles
• ZRLE tiles (raw, solid, packed palette, plain RLE, and palette RLE)
• Apple encoding 1000: zlib-compressed 1-bit halftone
• Apple encoding 1001: zlib-compressed 4-bit grayscale
• Apple encoding 1002: zlib-compressed RGB555 “thousands of colors”
• Apple encoding 1011 MVS framing, type-2 quantization-table updates, and partial-update solid, bilevel, repeat, left-copy, above-copy, and general Rice/DCT tile modes with zigzag, quantization, and inverse DCT, partial explicit/sequential cache records, plus full-update unchanged, copy-replay, differential DCT, JPEG chrominance Huffman, and explicit/sequential cache records
• independent persistent zlib state for every Apple stream
• native RFB framebuffer updates with checked dimensions, allocations, runs, and palette indexes
• client keyboard, pointer, and UTF-8 clipboard message generation
• optional interactive native GUI with direct TCP authentication, server-driven encrypted session streaming, selectable quality, GPU-native MVS tile/DCT output, viewer-side RGBA upload, mouse/keyboard/IME input, bidirectional clipboard synchronization, live FPS/traffic metrics, and Metal/D3D12/Vulkan presentation
• high-performance AVC media mode (1010) with binary-plist negotiation, authenticated cipher-suite-5 SRTP/AES-256-CTR, cross-SSRC DON/DONL-ordered H.264/HEVC depacketization, dedicated UDP receive, macOS VideoToolbox or Windows Media Foundation NV12 decode, strict 2x fixed-resolution validation, and GPU presentation

Apple MVS (1011) is identified as a distinct codec and is never fed to a VNC or zlib decoder. Its two bitstreams, Rice/DCT state, per-tile differential baseline, copy metadata, and 1–64999 DCT cache ring are decoded directly.

Current end-to-end boundary

The 0x21, 0x12, 1103, type-30 computation, encrypted-record, decrypted-payload dispatch, and bidirectional interaction layers are implemented and covered by focused tests, including in-process client↔oracle sessions that decode both adaptive MVS and full-quality persistent zlib frames from encrypted records while exercising keyboard, pointer, and clipboard messages. A Rust client has also completed a private live session against macOS screensharingd and decoded a fully covered framebuffer; the captured payload and pixels are deliberately not stored in this repository. See docs/SCREENSHARING_RE.md for confirmed evidence and the exact remaining work.

Pure Rust guarantee

Core runtime dependencies are Rust-only crates: flate2 with its rust_backend, RustCrypto AES/digest primitives, num-bigint, and subtle. The separate ard-viewer package adds winit, wgpu, and arboard for the operating system's normal window, GPU, and clipboard backends. Project code retains #![forbid(unsafe_code)].

Verification

cargo test -p ard-core --all-targets
cargo clippy -p ard-core --all-targets -- -D warnings
cargo test --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings
cargo check -p ard-core --target aarch64-unknown-linux-musl
cargo check -p ard-core --target x86_64-unknown-linux-musl
cargo check -p ard-core --target x86_64-pc-windows-gnu
cargo check -p ard-core --target wasm32-wasip1
cargo check -p ard-viewer --target x86_64-unknown-linux-musl --bin ard-viewer
cargo check -p ard-viewer --target x86_64-pc-windows-gnu --bin ard-viewer

The test suite includes the exact ARD banner and security offer captured from the local macOS Screen Sharing server, Apple type-30 authentication framing, independent byte-level vectors for all three Apple zlib subencodings, a persistent-stream test, a complete FramebufferUpdate, and ZRLE compact-pixel decoding. A local isolated Rust server was also connected to macOS Screen Sharing 6.1 (760.4): the native client advertised encoding 1011 and displayed the exact 15-byte MVS type-0/repeat packet as a stable 64x64 white framebuffer. A second dual-bitstream packet rendered one gray type-4 YCbCr tile followed by 63 white tiles and also remained connected. A third packet used the minimum type-5 Rice/DCT record (zero DC predictors and an immediate AC end-of-block); Apple's decoder rendered the expected gray tile. An additional nonzero-AC frame produced the same eight-value luminance ramp in Apple's decoder and this Rust implementation. The native decoder also accepted a stateful partial-Rice → full-differential sequence containing a nonzero standard-JPEG chrominance Huffman coefficient, then accepted both partial and full explicit cache recalls of the generated DCT tile.