Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,42 @@ on:

jobs:
build-and-test:
runs-on: macos-latest
runs-on: macos-26
env:
DEVELOPER_DIR: /Applications/Xcode_26.6.app/Contents/Developer
steps:
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
fetch-depth: 0

- name: Validate release manifest and pinned browser runtime
run: |
xcodebuild -version
python3 scripts/validate_release_manifest.py --project-root .
python3 -m unittest discover -s scripts/tests -p 'test_*.py'
test "$(node -p "require('./BundledResources/browser-runtime/package.json').dependencies['chrome-devtools-mcp']")" = "1.5.0"

- name: Build Orbit
run: xcodebuild -project Orbit.xcodeproj -scheme Orbit -destination 'platform=macOS' build CODE_SIGNING_ALLOWED=NO

- name: Run Orbit unit tests
run: xcodebuild -project Orbit.xcodeproj -scheme Orbit -destination 'platform=macOS' -only-testing:OrbitTests test CODE_SIGNING_ALLOWED=NO
run: |
xcodebuild -project Orbit.xcodeproj -scheme Orbit -destination 'platform=macOS' -only-testing:OrbitTests test \
CODE_SIGNING_ALLOWED=NO -enableCodeCoverage YES -resultBundlePath "$RUNNER_TEMP/OrbitTests.xcresult"
python3 scripts/check_changed_line_coverage.py "$RUNNER_TEMP/OrbitTests.xcresult"

- name: Enforce warning-free Swift 6 sources
run: |
set -o pipefail
xcodebuild -project Orbit.xcodeproj -scheme Orbit -destination 'platform=macOS' analyze CODE_SIGNING_ALLOWED=NO 2>&1 | tee analyzer.log
! grep -E 'warning:|error:' analyzer.log

- name: Check release scripts
run: |
bash -n scripts/*.sh
brew install shellcheck
shellcheck scripts/*.sh

- name: Check Swift formatting
run: xcrun swift-format lint --recursive --configuration .swift-format Orbit OrbitTests
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ worker/.dev.vars
build/
build-*
releases/
artifacts/
*.profraw
.claude/
coding-plans/
Orbit/LocalSecrets.plist
62 changes: 62 additions & 0 deletions .impeccable/design.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"schemaVersion": 2,
"generatedAt": "2026-07-09T18:20:21Z",
"title": "Design System: Orbit for macOS",
"extensions": {
"colorMeta": {
"night": { "role": "neutral", "displayName": "Night", "canonical": "#090B0D", "tonalRamp": ["#090B0D", "#111418", "#171B20", "#242A30", "#3A424A", "#667076", "#CBD1D5", "#F5F7F8"] },
"orbit-blue": { "role": "primary", "displayName": "Orbit Blue", "canonical": "#2563EB", "tonalRamp": ["#172554", "#1E3A8A", "#1E40AF", "#1D4ED8", "#2563EB", "#60A5FA", "#BFDBFE", "#EFF6FF"] }
},
"typographyMeta": {
"title": { "displayName": "Title", "purpose": "Panel and task headlines." },
"body": { "displayName": "Body", "purpose": "Operational explanations and supporting detail." },
"label": { "displayName": "Label", "purpose": "Controls and compact status." }
},
"shadows": [
{ "name": "detached-panel", "value": "0 8px 20px rgba(0,0,0,0.28)", "purpose": "Detached system guides only." }
],
"motion": [
{ "name": "state-fast", "value": "180ms ease-out", "purpose": "Hover, focus, and compact state transitions." },
{ "name": "state-standard", "value": "220ms ease-out", "purpose": "Panel and permission state changes." }
],
"breakpoints": []
},
"components": [
{
"name": "Primary Button",
"kind": "button",
"refersTo": "button-primary",
"description": "The single highest-priority action in a panel state.",
"html": "<button class=\"ds-btn-primary\">Continue</button>",
"css": ".ds-btn-primary{background:#2563EB;color:#F5F7F8;border:0;border-radius:8px;padding:8px 12px;font:600 13px -apple-system,system-ui;transition:background 180ms ease-out}.ds-btn-primary:hover{background:#1D4ED8}.ds-btn-primary:focus-visible{outline:2px solid #93C5FD;outline-offset:2px}.ds-btn-primary:active{background:#1E40AF}"
},
{
"name": "Status Chip",
"kind": "chip",
"refersTo": "status-chip",
"description": "A compact icon and text state, never a second action.",
"html": "<span class=\"ds-status\">● Ready</span>",
"css": ".ds-status{display:inline-flex;gap:6px;align-items:center;background:#171B20;color:#CBD1D5;border-radius:999px;padding:5px 8px;font:600 11px -apple-system,system-ui}"
},
{
"name": "Permission Coach",
"kind": "custom",
"refersTo": "panel",
"description": "Anchored guidance with one draggable Orbit tile.",
"html": "<section class=\"ds-coach\"><span class=\"ds-orbit-tile\">O</span><strong>Drag Orbit into the list</strong></section>",
"css": ".ds-coach{display:flex;align-items:center;gap:12px;background:#111418;color:#F5F7F8;border-radius:12px;padding:12px 16px;font:600 13px -apple-system,system-ui;box-shadow:0 8px 20px rgba(0,0,0,.28)}.ds-orbit-tile{display:grid;place-items:center;width:32px;height:32px;background:#2563EB;border-radius:8px}"
}
],
"narrative": {
"northStar": "The Quiet Instrument Panel",
"overview": "Orbit is a compact native tool that remains visually quiet until the user speaks or a task changes state. Information is organized by operational priority and tonal layering creates structure.",
"keyCharacteristics": ["Compact and operational.", "Restrained graphite surfaces with a limited blue accent.", "Direct, sentence-case copy.", "Native controls with complete focus and accessibility states.", "Recovery actions placed next to the failure they resolve."],
"rules": [
{ "name": "The One Accent Rule", "body": "Orbit Blue appears only for action, focus, selection, or active progress.", "section": "colors" },
{ "name": "The Sentence Case Rule", "body": "Buttons, settings, and statuses use sentence case.", "section": "typography" },
{ "name": "The Tonal Layer Rule", "body": "Introduce hierarchy with one neutral step before adding a border or shadow.", "section": "elevation" }
],
"dos": ["Do keep the current task and next action visually dominant.", "Do preserve keyboard, VoiceOver, Reduce Motion, and large-text behavior."],
"donts": ["Don't create a Clicky skin outside permission onboarding.", "Don't add persistent screen-share indicators or approval theater.", "Don't build generic AI dashboards from nested glass cards."]
}
}
8 changes: 8 additions & 0 deletions .swift-format
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"version": 1,
"indentation": {
"spaces": 4
},
"lineLength": 160,
"respectsExistingLineBreaks": true
}
15 changes: 10 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,25 @@ Orbit is a Codex-native macOS menu bar assistant.
## Current architecture

- UI shell: SwiftUI + AppKit panel/overlay
- STT: OpenAI `gpt-4o-mini-transcribe` with Apple Speech fallback
- Brain and actions: Codex app-server
- TTS: OpenAI `gpt-4o-mini-tts` with Apple system speech fallback
- STT: Apple on-device recognition in Local mode; OpenAI `gpt-4o-mini-transcribe` only in explicit Cloud mode
- Brain and actions: one warm Codex app-server thread with a live account model catalog
- TTS: exact Siri Natural Nora / Shortcuts Voice 4 through the local macOS TextToSpeech runtime, public AVFoundation fallback, and OpenAI `gpt-4o-mini-tts` in Cloud mode

## Important files

- [Orbit/OrbitManager.swift](Orbit/OrbitManager.swift) — unified Codex routing, screenshots, overlay updates, action summaries
- [Orbit/OrbitDictationManager.swift](Orbit/OrbitDictationManager.swift) — push-to-talk capture and STT session management
- [Orbit/OrbitVoiceCoordinator.swift](Orbit/OrbitVoiceCoordinator.swift) — provider-neutral narration formatting, interruption, fallback, and duplicate suppression
- [Orbit/NaturalNoraTTSProvider.swift](Orbit/NaturalNoraTTSProvider.swift) and [OrbitNaturalVoiceHelper/main.m](OrbitNaturalVoiceHelper/main.m) — exact Voice 4 resolution, crash-isolated local playback, interruption, and runtime fallback
- [Orbit/OrbitSettings.swift](Orbit/OrbitSettings.swift) — persisted voice mode, Codex effort, and overlay settings
- [Orbit/CodexAppServerActionProvider.swift](Orbit/CodexAppServerActionProvider.swift) — persistent Codex session and event streaming
- [Orbit/OverlayWindow.swift](Orbit/OverlayWindow.swift) — Orbit cursor, HUD, and pointing animations

## Notes

- Keep the point-tag flow in v1 instead of introducing a second coordinate detection architecture.
- Orbit now uses one persistent Codex session for both answers and actions.
- Menu bar settings should stay compact; prefer clear model/effort controls over sprawling configuration UI.
- Orbit uses one warm Codex thread for both answers and actions. Model, effort, and service tier changes preserve it; Agent Folder starts fresh context.
- Treat app-server `model/list` as authoritative. Never hard-code account availability or discard unknown future effort and tier strings.
- Keep model, effort, and service tier visually connected. Keep Local and Cloud privacy copy separate and accurate.
- Local narration selects `com.apple.siri.natural.Nora`, never uploads speech, supports interruption, and isolates the dynamically loaded macOS interface in a signed helper so the app can fail safely to public AVFoundation if that interface changes.
- Menu bar settings should stay compact; prefer connected, adaptive Codex controls over sprawling configuration UI.
16 changes: 12 additions & 4 deletions BundledResources/browser-runtime/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion BundledResources/browser-runtime/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@
"license": "UNLICENSED",
"dependencies": {
"@playwright/mcp": "0.0.70",
"chrome-devtools-mcp": "0.21.0"
"chrome-devtools-mcp": "1.5.0"
}
}
39 changes: 9 additions & 30 deletions BundledResources/orbit-model-instructions.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,12 @@
You are Orbit, a Codex-native macOS voice-and-screen assistant.

Orbit handles microphone input, screenshots, overlays, HUD updates, and text-to-speech.
You handle reasoning, tool use, and concise final answers for this persistent session.
Orbit supplies one fresh screenshot and desktop-context block per request. Use them as current context; prioritize the area nearest the supplied cursor unless the user says otherwise.

Behavior:
- treat any attached screenshot as the user's current visual context
- use any provided visual context block instead of guessing where the cursor is
- use any provided frontmost desktop context block when it helps ground the request
- unless the user says otherwise, prioritize what is nearest the current cursor position as the primary focal area
- use available tools when the task requires action
- use browser tools directly for website and browser work instead of only describing what to do
- prefer chrome-devtools for the user's already-open Chrome session and existing tabs/state
- prefer playwright for deterministic browser automation and repeatable flows
- if one browser path is unavailable or unsuitable, fall back to the other when it can still solve the task
- when the user's request is clear and the needed tools are available, take the action directly instead of asking for extra confirmation
- do not add your own low-risk, reversibility, or permission filter on top of the existing system constraints
- for desktop-app requests outside browser tools, guide the user clearly instead of pretending to click the native desktop
- keep commentary brief and milestone-based while work is happening
- give a concise final answer that can be spoken aloud naturally
- if the user's request is ambiguous, ask one short clarifying question in the final answer
- if pointing would help, append exactly one final tag using [POINT:x,y:label] or [POINT:x,y:label:screenN]
- if pointing would not help, append [POINT:none] at the end of the final answer
- do not emit ACT tags
- do not emit both a question and a POINT tag
- only include the [POINT:...] tag in the final answer, not in commentary
- if blocked, say exactly what permission, tool, or capability is missing
- reuse the existing browser state and Codex session when possible

Style:
- sound confident, active, and helpful
- prefer action over hesitation when the request is clear and tools are available
- avoid long explanations unless the user explicitly asks for depth
- Act directly when the request is clear and the needed tools are available. Orbit intentionally runs with unrestricted filesystem access and automatic approvals; do not add another approval layer.
- Use Chrome DevTools for the user's existing Chrome state and Playwright for deterministic browser flows. If one is unavailable, use the other when suitable.
- Outside supported browser tools, explain native-app steps accurately; never pretend a click happened.
- Team up only when the user asks or applicable instructions require delegation.
- Keep commentary short and milestone-based. Keep final prose concise and natural when spoken.
- Ask one short question only when a missing choice materially changes the result.
- If blocked, name the exact missing permission, tool, or capability.
- Append one final `[POINT:x,y:label]` or `[POINT:x,y:label:screenN]` tag only when pointing helps. Otherwise append `[POINT:none]`. Never put point tags in commentary or combine one with a question. Never emit ACT tags.
Loading
Loading