Prototype avatar popout local voice#123
Conversation
Add an avatar-first popout window with local Windows speech recognition and Edge TTS playback for the developer prototype. Rebased PR ianphil#123 onto current master after the apps/desktop + apps/web + packages/services + packages/shared monorepo restructure. New file locations: - apps/desktop/src/main/ipc/voice.ts (was src/main/ipc/voice.ts) - apps/web/src/renderer/components/avatar/* (was src/renderer/components/avatar/*) - apps/web/src/renderer/types/talkinghead.d.ts - packages/shared/src/types.ts: + VoiceRecognitionResult, VoiceSynthesisResult, ElectronAPI.voice - apps/web/vite.config.ts: + optimizeDeps.exclude for @met4citizen/talkinghead - apps/web/src/renderer/components/layout/AppShell.tsx: render avatar in popout when window.electronAPI.voice is available; gracefully fall back to chat in browser mode - apps/web/src/browserApi.ts: + voice stub returning 'unavailable in browser' errors - apps/desktop/src/main/ipc/mind.ts: popout window 860x900 + backgroundThrottling: false Original PR's sdkPaths.ts dev-mode fix is no longer needed: master already enforces repo-local SDK in dev (packages/services/src/sdk/sdkPaths.ts) more strictly. Validation: typecheck + eslint + deps:check (315 modules, 593 deps, 0 violations); all 4 ported test files pass at new locations (23/23); full suite passes (1050/1050). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4802533 to
d1cccb7
Compare
|
Hey @ianphil - Rebased onto current What changed in this revision Ported the prototype onto the new monorepo layout ( A couple things worth flagging:
Validation
Follow-ups (intentionally out of scope) Foundry/Speech avatar will land as a separate optional provider per our chat. Voice provider abstraction deferred until the second provider exists. Also naturally absorbs #19 (Whisper Light onboarding voice) which you closed as superseded - happy to split that out if you'd prefer something narrower. Let me know if you'd like me to break anything out, tighten scope, or address review comments - happy to iterate. 🎙️ |
|
One more thing worth surfacing while we're on this topic, Ian - I have an Azure Foundry-based avatar/speech version sitting on a separate fork branch that's the natural next step beyond this local prototype. Posting it here in case you want to take a look so we can decide how (or whether) to bring it into Branch: https://github.com/HashwanthVen/chamber/tree/voice-avatar-foundry |

Summary
Avatar-first popped-out agent window for the developer prototype.
Updated for monorepo restructure
This PR was originally opened before master's monorepo restructure into
apps/desktop,apps/web,apps/server,packages/services,packages/shared. The full prototype has been rebased onto the new layout:src/)src/main/ipc/voice.tsapps/desktop/src/main/ipc/voice.tssrc/main.ts(wiring)apps/desktop/src/main.ts(+ setupVoiceIPC())src/preload.ts(voice namespace)apps/desktop/src/preload.tssrc/main/ipc/mind.ts(popout window tweaks)apps/desktop/src/main/ipc/mind.ts(860×900 +backgroundThrottling: false)src/shared/types.ts(voice types)packages/shared/src/types.tssrc/renderer/components/avatar/*apps/web/src/renderer/components/avatar/*src/renderer/types/talkinghead.d.tsapps/web/src/renderer/types/talkinghead.d.tssrc/renderer/index.css(avatar stage CSS)apps/web/src/renderer/index.csssrc/renderer/components/layout/AppShell.tsxapps/web/src/renderer/components/layout/AppShell.tsx(with browser-vs-Electron guard viawindow.electronAPI?.voice)src/test/helpers.ts(mockElectronAPI voice)apps/web/src/test/helpers.tsvite.renderer.config.ts(optimizeDeps.exclude)apps/web/vite.config.tssrc/main/services/sdk/sdkPaths.ts(dev-mode fix)packages/services/src/sdk/sdkPaths.tsalready enforces repo-local SDK in dev mode (more strictly than the original PR fix).A new
apps/web/src/browserApi.tsvoice stub returns"unavailable in browser mode"errors so the browser variant doesn't crash if a developer opens the popout URL there. The avatar stage activates only whenwindow.electronAPI.voiceis present (Electron desktop).Validation
npm run lint— typecheck + eslint + dependency-cruiser clean (315 modules, 593 deps, 0 violations)npm test— full vitest suite clean (1050/1050 passing)apps/web/src/renderer/components/avatar/PopoutAvatarWindow.test.tsx(4)apps/web/src/renderer/components/avatar/AgentAvatarPanel.logic.test.ts(7)packages/services/src/sdk/sdkPaths.test.ts(4)packages/services/src/sdk/CopilotClientFactory.test.ts(8)npm start): avatar visible, mic captures speech via WindowsSystem.Speech, agent reply plays via Edge TTS, red ✕ returns to transcript, main chat unchanged.Dependencies added
@met4citizen/talkinghead@^1.7.0(MIT) →apps/web— avatar renderingedge-tts-universal@^1.4.0(AGPL-3.0) →apps/desktop— local TTS outputFollow-ups (intentionally out of scope here)
VoiceProviderinterface should happen alongside any second provider (e.g. Foundry).apps/web. Adding a Web Speech API + browser TTS shim could enable a browser-only path.