feat(clips): add local Rewind memory and semantic chapters

[3mdistal]

Summary

• add an opt-in, local-only Rewind buffer that shares the existing screen and audio capture substrate with ordinary Clips and meeting transcription
• organize retained evidence into stable semantic work chapters that connected local agents can search, inspect through exact frames or contact sheets, and promote into a bounded private Clip only through the explicit handoff flow
• add explicit 30-second and five-minute retrospective Clip editing, including an owner-visible private-first flow that refuses active direct shares at the transition, request, and apply boundaries
• add meeting-start recovery, privacy exclusions, rolling retention, local OCR/transcription, content-free activity provenance, and prompt-first Codex/Claude Code MCP setup
• preserve current main recording finalization, pending-upload recovery, deep-link behavior, and native/browser recorder safeguards

Product and privacy contract

• Rewind is disabled by default and raw archive media stays on the Mac.
• Ordinary Clips begin after the 3–2–1 countdown; earlier media is never included automatically.
• Rewind, Clips, and meeting notes do not start duplicate physical recorders for the same source.
• Local agents receive bounded text or path-free frames. Media becomes an ordinary private Clip only after one explicit bounded request.
• Review and automatic local preview default on but can be disabled independently. Agent-created Clips are retained forever by default.
• Durable activity receipts retain timestamps, opaque source IDs, counts, and disposition—not agent questions, transcript excerpts, chapter excerpts, image bytes, or archive paths.

Semantic chapters

• derive retention-bound evidence atoms and stable 30-second scenes from capture continuity, app/document context, bounded accessibility fingerprints, OCR, and source-attributed transcripts
• treat real gaps, exclusions, epochs, tainted media, and capture-mode changes as hard boundaries
• use persistent document/topic evidence for soft boundaries while absorbing short foreground interruptions and window-title churn
• keep ambient microphone speech subordinate to corroborated visible work when naming chapters
• preserve stable IDs/revisions, exact coverage, aliases, representative moments, and truthful frame-budget truncation

Verification

• 155 native library tests passed; one opt-in private-store audit is ignored by default and passed separately against three real 20-minute local ranges
• 623 Clips tests passed across 125 files
• 71 desktop tests passed across 18 files
• 24 focused Core Screen Memory MCP tests passed
• Core, Clips, and desktop typechecks passed
• Core, Toolkit, Clips production, and Alpha Tauri builds passed
• Alpha bundle passed strict codesign verification
• independent latency QA passed on the installed Alpha: Return completed the visible countdown in 758 ms, capture acknowledged start in 3 ms, Stop closed the logical boundary in 0 ms, and long-form runs uploaded progressively from the one existing Rewind producer
• the focused Preparing recording card and countdown cancellation cleanup passed real native-app visual QA
• Rust formatting, oxfmt, changeset status, and git diff --check passed
• independent code review: ready to land with zero blockers after two repair passes
• independent native-app successful-user-story QA: passed against commit 75fa94905, including a cross-session retained preview whose QuickTime artifact matched the exact request ID, decline with no upload, and terminal reason scrubbing
• one immediate post-decline QA screenshot captured a white frame while accessibility reported the normal surface; the next refresh was usable and the story passed

Known release boundaries

• this PR does not deploy the hosted retrospective editor/action; its real clips.agent-native.com round trip remains a post-deployment acceptance story
• local Alpha builds are ad-hoc signed when the Builder.io Developer ID identity is unavailable
• archive encryption/LocalAuthentication and sustained eight-hour battery/thermal measurement remain explicitly deferred release work

Governing brief

Architecture fingerprint: rewind-semantic-chapters-r5

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
34948924	Triggered	Bearer Token	956e95c	packages/core/src/mcp/screen-memory-stdio.spec.ts	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[github-actions]

Here's a visual recap of what changed:

Open the full interactive recap

Large diff — this recap is a summarized view (top files + schema/API deltas).

[3mdistal]

Landing verification update:

• Shared Rewind-backed Clips now prepare before the 3–2–1, accept Return through an app-local key monitor, and upload progressively without starting a second physical recorder.
• Installed Alpha QA: Return completed in 758 ms, countdown-to-start acknowledgement 3 ms, Stop logical boundary 0 ms, sustained recording did not auto-stop, hosted video/audio completed, and the focused Preparing card/cancellation flow passed.
• Post-main local verification: full workspace typecheck, all guards, 623 Clips tests, 71 desktop tests, and 155 Rust tests passed (1 intentional ignore).
• Independent final review: no blockers.

No merge performed; maintainer approval is still required.

• Codex AI

[builder-io-integration]

Builder reviewed your changes and found 1 potential issue 🔴

Review Details

Code Review Summary

This incremental review covers the latest Rewind changes, including durable recording boundaries, portable OCR helper builds, localization, and the new privacy/share safeguards. The three findings from the prior review were rechecked and are fixed: stored private-window exclusions are preserved, tainted/unretained rows are filtered from MCP context, and the Rewind privacy transition now uses a conditional update. Those review threads were resolved.

The remaining implementation is generally well-structured, with explicit local-only boundaries, bounded MCP egress, atomic privacy transition semantics, and focused desktop/UI coverage. However, this remains a high-risk change because screen and audio evidence can contain sensitive user data.

Key Findings

• 🔴 HIGH — Screen Memory exclusion settings are parsed and persisted in the local MCP client, but context retrieval does not apply excludedBundleIds or excludePrivateWindows. If a store contains rows produced before native filtering, or is accessed through a standalone/local MCP configuration, explicitly excluded sensitive-app or private-window evidence can still be returned to the agent.

🧪 Browser testing: Will run after this review (PR touches UI code)

[3mdistal]

Landing addendum (Codex): final head e7b115456 is current with main (0 behind), mergeable, and clean after a three-minute quiet-period recheck. All checks are green across macOS universal, Windows, Ubuntu, type/lint/security, scaffold E2E, and the full Fast tests matrix. The final review finding was addressed by enforcing configured bundle/private-window exclusions before local MCP query matching and evidence normalization, with bounded privacy-gap reporting and focused tests. The first Fast tests attempt was cancelled only by the workflow’s 20-minute timeout; the failed-job-only retry completed successfully. All review threads are resolved. No merge performed; maintainer approval remains required.