fix(voice): reset finalization gate on new turn + attach remote audio to DOM#19
Closed
sergiopesch wants to merge 1 commit into
Closed
fix(voice): reset finalization gate on new turn + attach remote audio to DOM#19sergiopesch wants to merge 1 commit into
sergiopesch wants to merge 1 commit into
Conversation
…to DOM
Two small but real issues in useVoiceInteraction.ts:
1. Dropped turn on fast re-speak.
finalizeTurn() sets isFinalizingTurnRef.current = true, then awaits
onFinalTranscript (a fetch to /api/chat). If the user starts a new
turn before that fetch resolves, the server-emitted
'input_audio_buffer.speech_started' event clears the buffers but
does NOT clear the gate. The next 'input_audio_transcription.
completed' event then hits:
if (isFinalizingTurnRef.current) return;
and gets silently dropped. The user's second utterance is lost.
Fix: reset isFinalizingTurnRef to false on speech_started. The
previous turn's in-flight fetch still completes and updates state;
it just no longer blocks the next turn from finalizing.
2. Remote <audio> element not in the DOM.
On pc.ontrack we lazily created an HTMLAudioElement with new Audio()
and never appended it to the document. Most desktop browsers will
play audio from a detached element, but Safari on iOS is known to
be inconsistent -- autoplay can silently fail and audio-route
selection (speaker vs earpiece vs Bluetooth) can misbehave on
detached nodes.
Fix: append to document.body with display:none + aria-hidden on
first creation, and remove the element from its parent during
teardown so repeated sessions don't leave orphaned <audio> nodes.
No changes to the public hook API. Verified with type-check, lint,
prettier, tests (12/12), production build.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related but independent fixes in
hooks/useVoiceInteraction.ts.1. Dropped turn on fast re-speak
finalizeTurnsetsisFinalizingTurnRef.current = true, thenawaitsonFinalTranscript(which is a fetch to/api/chat). If the user starts speaking again before that fetch resolves:The user's second utterance is silently dropped.
Fix: reset
isFinalizingTurnRefoninput_audio_buffer.speech_started. The previous turn's in-flight fetch still completes normally; it just no longer blocks the next turn from finalizing.2. Remote
<audio>element not attached to DOMOn
pc.ontrackwe lazily constructed:…and never appended it to the document. Most desktop browsers accept detached
<audio>elements, but Safari on iOS is known to be inconsistent — autoplay can silently fail, and audio-route selection (speaker / earpiece / Bluetooth) can misbehave on detached nodes.Fix: append to
document.bodywithdisplay: none+aria-hidden="true"on first creation. Remove it from its parent during teardown so repeated sessions don't leave orphaned nodes behind.Why both in one PR
They're two lines of code each, both in the same hook, both about the lifetime of connection-scoped resources. Splitting them would create more review overhead than either deserves.
Verification
npm run validate(type-check + lint + prettier)npm test— 12/12npm run build— production build succeedsCan't easily unit-test the race without a full Realtime-API harness; adding one was explicitly out of scope (would also have required jsdom or playwright). The fix is a two-line reset and the comment explains the invariant.
Depends on #12. Independent of #13/#14/#15/#16/#17/#18.