Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Info.plist
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@
<string>FluidVoice needs Accessibility permission to listen for global hotkeys and type text into other apps when you stop recording.</string>
<key>NSMicrophoneUsageDescription</key>
<string>FluidVoice needs microphone access to record your voice for speech-to-text transcription. Your audio is processed locally and never sent to external servers.</string>
<key>NSAudioCaptureUsageDescription</key>
<string>FluidVoice needs system audio access to record the meeting application you explicitly select. Meeting audio stays on this Mac.</string>
<key>NSScreenCaptureUsageDescription</key>
<string>FluidVoice needs Screen &amp; System Audio access to record audio from the meeting application you explicitly select.</string>
<key>LSUIElement</key>
<false/>
<key>NSAppleEventsUsageDescription</key>
Expand Down
25 changes: 23 additions & 2 deletions MEETING_TRANSCRIPTION_PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Checkboxes should be checked only when the requirement and its acceptance criter

### Document map

- [Agreed product requirements](#3-agreed-product-requirements--implementation-status)
- [Confirmed decisions](#3-confirmed-product-decisions)
- [Goals and non-goals](#4-goals-and-non-goals)
- [High-level milestones](#5-high-level-milestones)
- [User experience and canvas states](#7-user-experience-and-canvas-states)
Expand Down Expand Up @@ -128,6 +128,15 @@ The direction below is agreed. Its checkboxes remain open until the correspondin

## 5. High-level milestones

### Milestone delivery policy

Each milestone is implemented on a stacked branch targeting `1.6.8/meeting`; only the integration branch targets `main`. A milestone is not complete when code merely compiles.

- [ ] **[MUST · FOUNDATION] DELIVERY-001** Define the milestone's deterministic tests, real-system proof, performance/resource checks, and regression boundaries before implementation is called complete.
- [ ] **[MUST · FOUNDATION] DELIVERY-002** Assign independent architecture/reliability and UX/accessibility reviewers who did not own the implementation.
- [ ] **[MUST · FOUNDATION] DELIVERY-003** Resolve every blocking finding and record deferred non-blockers against a named later milestone.
- [ ] **[MUST · FOUNDATION] DELIVERY-004** Merge the reviewed milestone PR into `1.6.8/meeting`; open one final integration PR from `1.6.8/meeting` to `main` after the intended milestone set is complete.

### Milestone 0 — Decisions and durable foundations

- [ ] **[MUST · FOUNDATION] M0-EXIT-001** Resolve FOUNDATION and V1 open decisions in Section 22; M3/M4 decisions remain gated by their own milestones.
Expand All @@ -146,6 +155,18 @@ Goal: prove the complete path before polishing secondary automation.
- [ ] **[MUST · M1] M1-EXIT-005** The menu-bar menu shows **Stop Meeting Recording** first while capture is active.
- [ ] **[MUST · M1] M1-EXIT-006** Window closure and sidebar navigation do not stop the session.

#### Milestone 1 test and review gate

- [ ] **[MUST · M1] M1-TEST-001** Deterministic tests cover legal state transitions, concurrent/repeated Start and Stop, stale-generation callbacks, and exactly one finalization/processing job.
- [ ] **[MUST · M1] M1-TEST-002** A real online-call capture proves separate non-empty application/system and microphone files with timestamps inside one session manifest.
- [ ] **[MUST · M1] M1-TEST-003** A real in-room capture proves microphone-only recording does not request Screen/System Audio permission solely for that mode.
- [ ] **[MUST · M1] M1-TEST-004** Stop proves the complete offline English path: finalized audio → transcription/diarization → timestamped canvas result.
- [ ] **[MUST · M1] M1-TEST-005** Apple Silicon fixtures cover remote speaker separation, clean confirmed personal-mic **You**, and ambiguous/leaked microphone speech remaining unknown.
- [ ] **[MUST · M1] M1-TEST-006** UI proof covers menu item order and Stop action, menu-bar recording mark, sidebar navigation, main-window close/reopen, and the four canvas states.
- [ ] **[MUST · M1] M1-REVIEW-001** An independent audio/architecture reviewer approves capture ownership, track separation, timestamp handling, provider serialization, and no dictation first-PCM regression.
- [ ] **[MUST · M1] M1-REVIEW-002** An independent UX/accessibility reviewer approves sidebar placement, state clarity, menu-bar behavior, keyboard/VoiceOver labels, and low-resource rendering.
- [ ] **[MUST · M1] M1-REVIEW-003** Every blocking review finding is fixed or explicitly recorded as a later-milestone item before the M1 stacked PR merges.

### Milestone 2 — Trustworthy offline V1

Goal: make the working slice safe to ship.
Expand Down Expand Up @@ -719,8 +740,8 @@ flowchart TD
- [ ] **[MUST · V1] TEST-REPO-002** Validate implementation builds with `sh build_with_FI_incremental.sh` and inspect the installed `/Applications/FluidVoice.app`.
- [ ] **[MUST · V1] TEST-REPO-003** Do not treat Debug/XCTest-host runs as release proof; use focused logic checks plus installed-app, real-meeting evidence.
- [ ] **[MUST · V1] TEST-REPO-004** Verify the minimum supported macOS path and low-resource Apple Silicon behavior; describe Intel as reduced capability until separately proven.
- [ ] **[MUST · V1] TEST-REPO-005** Update the requested release-notes version locally, credit contributors as required, and never commit ignored release notes.
- [ ] **[MUST · V1] TEST-REPO-006** Run real Intel capture plus plain English transcription before claiming Intel meeting support; universal build slices alone are insufficient.
- [ ] **[MUST · V1] TEST-REPO-005** Update the requested release-notes version locally, credit contributors as required, and never commit ignored release notes.

### 21.7 Measurable budgets to set before implementation

Expand Down
4 changes: 3 additions & 1 deletion Sources/Fluid/AppDelegate.swift
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ class AppDelegate: NSObject, NSApplicationDelegate, UNUserNotificationCenterDele
didFinishShutdown = true
}

let deadline = Date().addingTimeInterval(8)
// Meeting capture can spend up to three seconds stopping its runtime and
// four seconds finalizing audio before the durable session save.
let deadline = Date().addingTimeInterval(12)
while !didFinishShutdown, Date() < deadline {
RunLoop.current.run(mode: .default, before: Date().addingTimeInterval(0.01))
}
Expand Down
57 changes: 49 additions & 8 deletions Sources/Fluid/ContentView.swift
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ enum SidebarItem: Hashable {
case voiceEngine
case aiEnhancements
case preferences
case meetingTools
case fileTranscription
case meetingTranscription
case customDictionary
case stats
case history
Expand Down Expand Up @@ -591,6 +592,9 @@ struct ContentView: View {
DispatchQueue.main.asyncAfter(deadline: .now() + 1.5) {
DebugLogger.shared.info("🚦 Startup delay complete, signaling UI ready...", source: "ContentView")
self.appServices.signalUIReady()
self.menuBarManager.configure(
meetingCoordinator: self.appServices.meetingSessionCoordinator
)

Task { @MainActor in
await AudioStartupGate.shared.scheduleOpenAfterInitialUISettled()
Expand Down Expand Up @@ -958,6 +962,8 @@ struct ContentView: View {
switch destination {
case .customDictionary:
self.selectedSidebarItem = .customDictionary
case .meetingTranscription:
self.selectedSidebarItem = .meetingTranscription
case .preferences:
self.selectedSidebarItem = .preferences
}
Expand Down Expand Up @@ -1166,7 +1172,12 @@ struct ContentView: View {

Section {
self.sidebarNavigationLink(.commandMode, title: "Command Mode", systemImage: "terminal.fill")
self.sidebarNavigationLink(.meetingTools, title: "File Transcription", systemImage: "doc.text.fill")
self.sidebarNavigationLink(.fileTranscription, title: "File Transcription", systemImage: "doc.text.fill")
self.sidebarNavigationLink(
.meetingTranscription,
title: "Meeting Transcription",
systemImage: "person.2.wave.2.fill"
)
} header: {
self.sidebarSectionHeader("Use")
}
Expand Down Expand Up @@ -1264,8 +1275,10 @@ struct ContentView: View {
))
case .preferences:
return AnyView(self.preferencesView)
case .meetingTools:
return AnyView(self.meetingToolsView)
case .fileTranscription:
return AnyView(self.fileTranscriptionView)
case .meetingTranscription:
return AnyView(self.meetingTranscriptionView)
case .customDictionary:
return AnyView(CustomDictionaryView())
case .stats:
Expand Down Expand Up @@ -1500,10 +1513,21 @@ struct ContentView: View {
})
}

// MARK: - Meeting Transcription (Coming Soon)
// MARK: - File and Meeting Transcription

private var fileTranscriptionView: some View {
FileTranscriptionView(
asrService: self.asr,
transcriptionService: self.appServices.fileTranscriptionService
)
}

private var meetingToolsView: some View {
MeetingTranscriptionView(asrService: self.asr)
private var meetingTranscriptionView: some View {
MeetingTranscriptionView(
coordinator: self.appServices.meetingSessionCoordinator,
asrService: self.asr,
onOpenVoiceEngine: { self.selectedSidebarItem = .voiceEngine }
)
}

// MARK: - Stats View
Expand Down Expand Up @@ -3068,6 +3092,7 @@ struct ContentView: View {

/// Capture app context at start to avoid mismatches if the user switches apps mid-session
private func startRecording() {
guard !self.presentExclusiveActivityBlockIfNeeded() else { return }
let model = SettingsStore.shared.selectedSpeechModel
DebugLogger.shared.info(
"ContentView: startRecording() for model=\(model.displayName), supportsStreaming=\(model.supportsStreaming)",
Expand All @@ -3077,7 +3102,6 @@ struct ContentView: View {
DebugLogger.shared.debug("ContentView: start ignored because capture is already active", source: "ContentView")
return
}

self.advanceOverlayLifecycle()
self.setActiveRecordingMode(.dictate)
let shouldShowDictationOverlay = !self.isRecordingForCommand
Expand Down Expand Up @@ -3128,6 +3152,20 @@ struct ContentView: View {
}
}

private func presentExclusiveActivityBlockIfNeeded() -> Bool {
guard let activity = self.asr.activeExclusiveActivity else { return false }
let message = "Wait for the active \(activity.displayName) to finish."
self.asr.errorTitle = "Dictation Unavailable"
self.asr.errorMessage = message
self.asr.showError = true
AccessibilityNotification.Announcement("Dictation unavailable. \(message)").post()
DebugLogger.shared.info(
"ContentView: dictation start blocked by \(activity.rawValue)",
source: "ContentView"
)
return true
}

private func prewarmPrivateAIDictationIfNeeded(for slot: SettingsStore.DictationShortcutSlot) {
let appBundleID = self.recordingAppInfo?.bundleId
guard PrivateAIProviderPromptFormat.isAvailable(settings: SettingsStore.shared),
Expand Down Expand Up @@ -3337,6 +3375,7 @@ struct ContentView: View {
self.beginDictationRecording(for: selection, mode: .promptMode)
},
commandModeCallback: {
guard !self.presentExclusiveActivityBlockIfNeeded() else { return }
DebugLogger.shared.info("Command mode triggered", source: "ContentView")
self.captureRecordingContext()

Expand Down Expand Up @@ -3368,6 +3407,7 @@ struct ContentView: View {
}
},
rewriteModeCallback: {
guard !self.presentExclusiveActivityBlockIfNeeded() else { return }
guard !self.showPrivateAIEditModeUnavailableIfNeeded() else { return }

self.captureRecordingContext()
Expand Down Expand Up @@ -3721,6 +3761,7 @@ extension ContentView {
}

private func beginDictationRecording(for slot: SettingsStore.DictationShortcutSlot, mode: ActiveRecordingMode) {
guard !self.presentExclusiveActivityBlockIfNeeded() else { return }
DebugLogger.shared.debug("Begin dictation recording for slot \(slot.rawValue)", source: "ContentView")
self.appBench("begin_recording slot=\(slot.rawValue) mode=\(mode.rawValue)")
if self.isOnboardingVoicePlaygroundStepActive {
Expand Down
18 changes: 18 additions & 0 deletions Sources/Fluid/Persistence/SettingsStore.swift
Original file line number Diff line number Diff line change
Expand Up @@ -1813,6 +1813,23 @@ final class SettingsStore: ObservableObject {
set { self.defaults.set(newValue, forKey: Keys.preferredInputDeviceUID) }
}

var meetingRecordingDefaults: MeetingRecordingDefaults {
get {
guard let data = self.defaults.data(forKey: Keys.meetingRecordingDefaults),
let saved = try? JSONDecoder().decode(MeetingRecordingDefaults.self, from: data)
else {
return .unconfigured
}
return saved
}
set {
guard let data = try? JSONEncoder().encode(newValue) else {
return
}
self.defaults.set(data, forKey: Keys.meetingRecordingDefaults)
}
}

var preferredOutputDeviceUID: String? {
get { self.defaults.string(forKey: Keys.preferredOutputDeviceUID) }
set { self.defaults.set(newValue, forKey: Keys.preferredOutputDeviceUID) }
Expand Down Expand Up @@ -4885,6 +4902,7 @@ private extension SettingsStore {
static let primaryDictationShortcutsKey = "PrimaryDictationShortcuts"
static let preferredInputDeviceUID = "PreferredInputDeviceUID"
static let preferredOutputDeviceUID = "PreferredOutputDeviceUID"
static let meetingRecordingDefaults = "MeetingRecordingDefaults"
static let microphoneSelectionMode = "MicrophoneSelectionMode"
static let appMicSelectionMigrationVersion = "AppOnlyMicrophoneSelectionMigrationVersion"
static let visualizerNoiseThreshold = "VisualizerNoiseThreshold"
Expand Down
Loading
Loading