Skip to content

Defer update prompts until dictation completes #746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
joeyjooste wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Defer update prompts until dictation completes #746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b05c490
Defer update prompts during dictation
joeyjooste Jul 31, 2026
8e23a8b
Keep update deferral active through output delivery
joeyjooste Jul 31, 2026
957d43c
Wait for typing before ending update deferral
joeyjooste Jul 31, 2026
d5232e1
Keep update deferral across dictation boundaries
joeyjooste Jul 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Sources/Fluid/AppDelegate.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,20 @@ class AppDelegate: NSObject, NSApplicationDelegate, UNUserNotificationCenterDele

@MainActor
private func showUpdateNotification(version: String) {
guard !AppServices.shared.hasActiveDictation else {
DebugLogger.shared.info(
"Deferring update notification for \(version) until dictation finishes",
source: "AppDelegate"
)
Task { @MainActor in
while AppServices.shared.hasActiveDictation {
try? await Task.sleep(nanoseconds: 500_000_000)
}
self.showUpdateNotification(version: version)
}
return
}

DebugLogger.shared.info("Showing update notification for version \(version)", source: "AppDelegate")

let alert = NSAlert()
Expand Down
30 changes: 22 additions & 8 deletions Sources/Fluid/ContentView.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2012,6 +2012,9 @@ struct ContentView: View {
// MARK: - Stop and Process Transcription

private func stopAndProcessTranscription(route: DictationOutputRoute = .normal) async {
self.appServices.setDictationOutputDeliveryActive(true)
defer { self.appServices.setDictationOutputDeliveryActive(false) }

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep deferral active until typing finishes

In the external typing route, this defer clears hasActiveDictation as soon as stopAndProcessTranscription returns, but the actual insertion has only been dispatched: ASRService.typeOutputPlanToActiveField calls TypingService.typeOutputPlanInstantly, which queues a background worker and can wait before inserting/pasting. Fresh evidence in this revision is that the new higher-level flag still drops before that worker completes, so a deferred automatic update alert can wake during that window, take focus, and cause the dictated output to miss or target the wrong UI; keep the flag active until the typing/paste operation has actually completed.

Useful? React with 👍 / 👎.


DebugLogger.shared.debug("stopAndProcessTranscription called", source: "ContentView")
DebugLogger.shared.info("Output route selected: \(route.rawValue)", source: "ContentView")
self.appBench("stop_path_enter route=\(route.rawValue)")
Expand Down Expand Up @@ -2378,20 +2381,23 @@ struct ContentView: View {

if shouldTypeExternally {
let typingTarget = self.resolveTypingTargetPID()
// Dispatch insertion as soon as the destination app is ready; the
// overlay hides asynchronously after output so it cannot delay paste.
// Insert as soon as the destination app is ready, then keep update
// prompts deferred until the background typing worker completes.
if typingTarget.shouldRestoreOriginalFocus {
await self.restoreFocusToRecordingTarget()
}
self.appBench(
"text_ready_to_type_request elapsedMs=\(Int(((ProcessInfo.processInfo.systemUptime - finalTextReadyAt) * 1000).rounded()))"
)
self.asr.typeOutputPlanToActiveField(
finalOutputPlan,
preferredTargetPID: typingTarget.pid,
textReadyAt: finalTextReadyAt,
tracksDictionaryCorrections: true
)
await withCheckedContinuation { continuation in
self.asr.typeOutputPlanToActiveField(
finalOutputPlan,
preferredTargetPID: typingTarget.pid,
textReadyAt: finalTextReadyAt,
tracksDictionaryCorrections: true,
completion: { continuation.resume() }
)
}
didTypeExternally = true
if !shouldShowAIProcessingFailure, !didRequestOverlayHideOnStop {
self.hideOverlayAfterOutput()
Expand Down Expand Up @@ -3104,7 +3110,9 @@ struct ContentView: View {
)
}

self.appServices.beginDictationStartup()
Task {
defer { self.appServices.endDictationStartup() }
let startOutcome = await self.asr.start(onCaptureStarted: {
if shouldPlayStartSound {
TranscriptionSoundPlayer.shared.playStartSound()
Expand Down Expand Up @@ -3361,7 +3369,9 @@ struct ContentView: View {
"Starting voice recording for command",
source: "ContentView"
)
self.appServices.beginDictationStartup()
Task {
defer { self.appServices.endDictationStartup() }
let startOutcome = await self.asr.start(onCaptureStarted: {
TranscriptionSoundPlayer.shared.playStartSound()
self.appBench("overlay_phase phase=recording trigger=first_pcm mode=command")
Expand Down Expand Up @@ -3407,7 +3417,9 @@ struct ContentView: View {

// Start recording immediately for the edit instruction
DebugLogger.shared.info("Starting voice recording for edit mode", source: "ContentView")
self.appServices.beginDictationStartup()
Task {
defer { self.appServices.endDictationStartup() }
let startOutcome = await self.asr.start(onCaptureStarted: {
TranscriptionSoundPlayer.shared.playStartSound()
self.appBench("overlay_phase phase=recording trigger=first_pcm mode=edit")
Expand Down Expand Up @@ -3752,7 +3764,9 @@ extension ContentView {
self.appBench("overlay_mode_requested mode=Dictation")
self.appBench("overlay_phase phase=connecting")
}
self.appServices.beginDictationStartup()
Task {
defer { self.appServices.endDictationStartup() }
let asrStartStartedAt = ProcessInfo.processInfo.systemUptime
DebugLogger.shared.benchmark("APP_BENCH", message: "asr_start_call", source: "AppBenchmark")
let startOutcome = await self.asr.start(onCaptureStarted: {
Expand Down
9 changes: 7 additions & 2 deletions Sources/Fluid/Services/ASRService.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ final class ASRService: ObservableObject {
private(set) var dictionaryTrainingAudioGeneration = 0

@Published private(set) var isStarting: Bool = false // Guard against re-entrant start() calls
@Published private(set) var isFinalizing: Bool = false
private var audioCaptureStartWaiters: [CheckedContinuation<Void, Never>] = []
var isRunningOrStarting: Bool {
self.isRunning || self.isStarting
Expand Down Expand Up @@ -1797,8 +1798,10 @@ final class ASRService: ObservableObject {
DebugLogger.shared.warning("⚠️ STOP() - not running, returning empty string", source: "ASRService")
return ""
}
self.isFinalizing = true
let useDictionaryTrainingPath = forDictionaryTraining || self.isDictionaryTrainingCaptureActive
defer {
self.isFinalizing = false
self.applyPendingParakeetVocabularyReloadIfNeeded()
self.isDictionaryTrainingCaptureActive = false
}
Expand Down Expand Up @@ -3997,7 +4000,8 @@ final class ASRService: ObservableObject {
_ plan: DictationLiteralOutputPlan,
preferredTargetPID: pid_t?,
textReadyAt: TimeInterval? = nil,
tracksDictionaryCorrections: Bool = false
tracksDictionaryCorrections: Bool = false,
completion: (() -> Void)? = nil
) {
let requestedAt = ProcessInfo.processInfo.systemUptime
let textReadyAge = textReadyAt.map { Int(((requestedAt - $0) * 1000).rounded()) }
Expand All @@ -4011,7 +4015,8 @@ final class ASRService: ObservableObject {
plan,
preferredTargetPID: preferredTargetPID,
textReadyAt: textReadyAt,
tracksDictionaryCorrections: tracksDictionaryCorrections
tracksDictionaryCorrections: tracksDictionaryCorrections,
completion: completion
)
let dispatchedAt = ProcessInfo.processInfo.systemUptime
let textReadyToDispatchMs = textReadyAt.map {
Expand Down
20 changes: 20 additions & 0 deletions Sources/Fluid/Services/AppServices.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ final class AppServices: ObservableObject {

/// Automatic speech recognition service (lazily initialized)
private var _asr: ASRService?
private var pendingDictationStartCount = 0
private var isDeliveringDictationOutput = false
var asr: ASRService {
if let existing = self._asr {
return existing
Expand All @@ -68,6 +70,24 @@ final class AppServices: ObservableObject {
return service
}

var hasActiveDictation: Bool {
self.pendingDictationStartCount > 0 ||
self.isDeliveringDictationOutput ||
(self._asr.map { $0.isRunningOrStarting || $0.isFinalizing } ?? false)
Comment on lines +73 to +76

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark dictation active before scheduling ASR startup

When a user triggers dictation as the automatic update check completes, startRecording() schedules asr.start() in a new Task, so there is a main-actor scheduling window before isStarting becomes true. During that window this expression reports no active dictation, allowing showUpdateNotification to enter its modal alert even though recording startup has already been requested; track the pending start synchronously or set the activity flag before creating the startup task.

Useful? React with 👍 / 👎.

}

func beginDictationStartup() {
self.pendingDictationStartCount += 1
}

func endDictationStartup() {
self.pendingDictationStartCount = max(0, self.pendingDictationStartCount - 1)
}

func setDictationOutputDeliveryActive(_ active: Bool) {
self.isDeliveringDictationOutput = active
Comment on lines +87 to +88

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Count overlapping output-delivery operations

When two stop triggers queue stopAndProcessTranscription before the first has finished—for example, a recording-control action and the global shortcut—the second call can quickly return empty because the first already cleared isRunning, then set this shared Boolean to false. Once the first asr.stop() returns, isFinalizing is also false even though its AI processing and typing are still active, so the deferred update alert can wake prematurely. Use a balanced counter/token rather than a Boolean so one invocation cannot clear another invocation's deferral.

Useful? React with 👍 / 👎.

}
Comment on lines +73 to +89

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep update deferral active through output delivery

When dictation uses AI post-processing, ContentView.stopAndProcessTranscription continues refining, saving, and typing after ASRService.stop() returns; this helper flips false as soon as ASR finalization ends. A deferred update alert can therefore wake during that post-stop processing and take focus before the dictated output is delivered, so update prompts can still interrupt the active dictation workflow. Track the higher-level stop/output processing state, or keep the deferral active until callers finish delivering the transcription.

Useful? React with 👍 / 👎.


private var _microphonePreferenceCoordinator: MicrophonePreferenceCoordinator?
var microphonePreferenceCoordinator: MicrophonePreferenceCoordinator {
if let existing = self._microphonePreferenceCoordinator {
Expand Down
21 changes: 14 additions & 7 deletions Sources/Fluid/Services/TypingService.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,8 @@ final class TypingService {
_ plan: DictationLiteralOutputPlan,
preferredTargetPID: pid_t?,
textReadyAt: TimeInterval?,
tracksDictionaryCorrections: Bool = false
tracksDictionaryCorrections: Bool = false,
completion: (() -> Void)? = nil
) {
let requestedAt = ProcessInfo.processInfo.systemUptime
let text = plan.plainText
Expand All @@ -315,13 +316,15 @@ final class TypingService {
guard text.isEmpty == false else {
self.bench("request_return reason=empty_text")
self.log("[TypingService] ERROR: Empty text provided, aborting")
completion?()
return
}

// Prevent concurrent typing operations
guard !self.isCurrentlyTyping else {
self.bench("request_return reason=already_typing")
self.log("[TypingService] WARNING: Skipping text injection - already in progress")
completion?()
return
}

Expand All @@ -330,6 +333,7 @@ final class TypingService {
self.bench("request_return reason=accessibility_not_trusted")
self.log("[TypingService] ERROR: Accessibility permissions required for text injection")
self.log("[TypingService] Current accessibility status: \(AXIsProcessTrusted())")
completion?()
return
}

Expand All @@ -341,12 +345,15 @@ final class TypingService {
self.bench("worker_start queueDelayMs=\(Self.elapsedMs(from: requestedAt, to: workerStartedAt))")

defer {
let completedAt = ProcessInfo.processInfo.systemUptime
self.isCurrentlyTyping = false
self.bench(
"complete totalMs=\(Self.elapsedMs(from: requestedAt, to: completedAt)) textReadyToCompleteMs=\(textReadyAt.map { String(Self.elapsedMs(from: $0, to: completedAt)) } ?? "nil")"
)
self.log("[TypingService] Typing operation completed, isCurrentlyTyping set to false")
Self.pasteboardRestoreQueue.async {
let completedAt = ProcessInfo.processInfo.systemUptime
self.isCurrentlyTyping = false
self.bench(
"complete totalMs=\(Self.elapsedMs(from: requestedAt, to: completedAt)) textReadyToCompleteMs=\(textReadyAt.map { String(Self.elapsedMs(from: $0, to: completedAt)) } ?? "nil")"
)
self.log("[TypingService] Typing operation completed, isCurrentlyTyping set to false")
completion?()
}
}

self.log("[TypingService] Starting async text insertion process")
Expand Down
6 changes: 5 additions & 1 deletion Sources/Fluid/Views/CommandModeView.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -593,7 +593,11 @@ struct CommandModeView: View {
}
}
} else {
Task { await self.asr.start() }
self.appServices.beginDictationStartup()

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Track voice-command processing through completion

When recording from CommandModeView, this registers only the startup phase; after asr.stop() finishes, hasActiveDictation becomes false while processUserCommand is still awaiting the generated response. A deferred automatic update alert can therefore appear during a voice-originated command and, if the user installs it, restart the app before the response is produced. Keep output-delivery activity set around the stop-and-command-processing task, as the main ContentView dictation route does.

Useful? React with 👍 / 👎.

Task {
defer { self.appServices.endDictationStartup() }
await self.asr.start()
}
}
}

Expand Down
6 changes: 5 additions & 1 deletion Sources/Fluid/Views/RewriteModeView.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,11 @@ struct RewriteModeView: View {
_ = self.asr.consumeLastCompletedAudioSnapshot()
}
} else {
Task { await self.asr.start() }
self.appServices.beginDictationStartup()
Task {
defer { self.appServices.endDictationStartup() }
await self.asr.start()
}
}
}

Expand Down
30 changes: 30 additions & 0 deletions Tests/FluidDictationIntegrationTests/DictationE2ETests.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2051,6 +2051,36 @@ final class DictationE2ETests: XCTestCase {
XCTAssertFalse(SimpleUpdater.isRollbackVersion(nil, differentFrom: "1.5.11-beta.3"))
}

@MainActor
func testUpdatePromptDeferralCoversOutputDelivery() {
AppServices.shared.setDictationOutputDeliveryActive(true)
defer { AppServices.shared.setDictationOutputDeliveryActive(false) }

XCTAssertTrue(AppServices.shared.hasActiveDictation)
}

@MainActor
func testUpdatePromptDeferralCoversScheduledStartup() {
AppServices.shared.beginDictationStartup()
AppServices.shared.beginDictationStartup()
AppServices.shared.endDictationStartup()
defer { AppServices.shared.endDictationStartup() }

XCTAssertTrue(AppServices.shared.hasActiveDictation)
}

func testTypingCompletionRunsWhenNoOutputIsQueued() async {
let completed = self.expectation(description: "Typing completion")
TypingService().typeOutputPlanInstantly(
.plain(""),
preferredTargetPID: nil,
textReadyAt: nil,
completion: { completed.fulfill() }
)

await self.fulfillment(of: [completed], timeout: 1)
}

// MARK: - Model download HTML/markup rejection (#353)

func testLooksLikeHTML_rejectsMarkupVariants() {
Expand Down
Loading