feat(cv): on-demand Python CV worker lifecycle + Windows setup#18
Conversation
Run the CV worker only while detection is wanted instead of for the whole
app session. The renderer requests/releases the worker via IPC; it is
reference-counted in the main process and started on the first request,
stopped a short grace period after the last release.
- python-bridge.ts: reference-counted request/release with a stop grace
period, crash auto-restart (capped to avoid crash loops), and a loud
check that fails clearly when the venv is missing
- index.ts: drop launch-time start; keep quit-time stop as a safety net
- ipc-handlers / preload / vite-env: expose window.taskmaster.cv.{request,release}
- useCvDetection: request the worker before connecting, release on cleanup
- onboarding: face/phone check pages wired to the worker via useCvDetection
- setup.ps1 + README: Windows PowerShell setup mirroring setup.sh
# Conflicts: # electron/src/main/index.ts # electron/src/main/ipc-handlers.ts # electron/src/preload/index.js # electron/src/renderer/vite-env.d.ts
|
@copilot check for possible security issues |
Addressed in I reviewed the CV worker path and added hardening in
Validation run: Electron lint/build, Python compile check, secret scan, and CodeQL (0 alerts). |
|
Hey Saatvik, this PR looks good overall and I like the direction of the on-demand CV worker. I just noticed 2 small things I think we should fix before merging. First, cd python
.\.venv\Scripts\Activate.ps1
python -m uvicorn main:app --port 8765or simply say that after running setup, Electron will start the CV worker automatically when detection is needed. Second, in I’d fix it by tracking whether the CV worker was requested, then calling Apart from that, the architecture makes sense to me. |
…un hints useCvDetection requested the CV worker before opening the camera, but if getUserMedia failed (camera denied/missing/busy, invalid saved device id) the catch only logged — leaving the worker requested until unmount even though no frames were being sent. Release it immediately on startup failure, guarded so the unmount cleanup doesn't double-release the reference count. Also fix the post-setup hints in setup.ps1/setup.sh: they pointed at the old cv/detection_loop.py entry point. The worker now runs as uvicorn main:app and is started on demand by Electron, so the scripts now say that and give the standalone uvicorn command for testing.
# Conflicts: # electron/src/main/index.ts # electron/src/main/ipc-handlers.ts
Summary
Connects the Electron app to the Python CV worker, and changes the worker so it only runs while something actually needs detection instead of staying up for the whole app session. Also adds the onboarding camera-check screens that use it, plus a Windows setup script.
What this PR does
On-demand CV worker lifecycle (
python-bridge.ts)requestand stops a short grace period (2s) after the lastrelease, so nothing runs while no session or check is active. The grace period keeps it from dying and respawning on a quick pause/resume, a page change, or React StrictMode's double-mount in dev.index.ts. The quit-time stop stays as a safety net.Renderer to main wiring
ipc-handlers,preload, andvite-envexposewindow.taskmaster.cv.request/release.useCvDetectionasks for the worker before opening its WebSocket and releases it on cleanup.Onboarding camera checks
OnboardingFaceCheckandOnboardingPhoneCheckpages stream the camera to the worker throughuseCvDetectionand show live feedback. Wired intoOnboardingPage.Windows setup
setup.ps1is a PowerShell version ofsetup.sh(venv 3.11, pip deps, model downloads, npm install). The README points to both.Testing
I ran the full app, not just the build. The worker started only once the renderer asked for it, MediaPipe gaze ran on a live camera frame over the WebSocket, and it shut down cleanly on quit.
tsc -b && vite buildandeslint .are both clean.Notes
This branch already has the latest
mainmerged in, including #16 (mini-timer). The two features don't touch each other and run side by side.