wip scope server cloud #383
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add src/scope/server/fal_client.py with WebSocket + WebRTC client - Add tests/server/test_fal_client.py with 9 unit tests - Add dependencies: aiohttp, websockets, pytest-asyncio - Update docs/fal-server-integration-plan.md with detailed Phase 1 testing guide FalClient handles: - Token acquisition from fal REST API - WebSocket URL construction - WebRTC peer connection setup - ICE candidate exchange - Frame sending/receiving via tracks Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add custom MediaStreamTrack classes for fal.ai WebRTC communication: - FalOutputTrack: Sends frames from a queue to fal.ai via WebRTC - Manages frame queue with overflow handling (drops oldest) - Sets proper pts and time_base for WebRTC encoding - Provides async and sync put methods - FalInputTrack: Receives processed frames from fal.ai - Wraps incoming track with consume loop - Queues received frames for retrieval - Includes clean shutdown via stop() Also includes: - 18 unit tests covering both track classes - Updated Phase 2 testing documentation with detailed instructions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…routing - Add fal_client, fal_enabled, and _fal_received_frames attributes to FrameProcessor - Add connect_to_fal() and disconnect_from_fal() async methods - Add _on_fal_frame_received() callback for frames from fal cloud - Modify put() to route frames to fal WebRTC when fal_enabled=True - Modify get() to return frames from fal received queue when enabled - Add stop_async() for proper async cleanup of fal connection - Update stop() with fallback fal cleanup - Add 14 unit tests for fal integration in FrameProcessor - Update plan doc to mark Phase 3 complete Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add three new API endpoints for managing fal.ai cloud connections: - POST /api/v1/fal/connect - Connect sessions to fal cloud - POST /api/v1/fal/disconnect - Disconnect sessions from fal cloud - GET /api/v1/fal/status - Get current fal connection status Also adds FalConnectRequest and FalStatusResponse schemas, plus 11 unit tests covering all endpoints and edge cases. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
The Spout receiver loop was bypassing put() and writing directly to the local pipeline queue, which meant Spout input wouldn't route to fal even when fal mode was enabled. Changes: - Modify _spout_receiver_loop to check fal_enabled and route through put() - Convert numpy frames to av.VideoFrame for WebRTC compatibility with fal - Add 2 unit tests for Spout fal routing behavior - Update Phase 5 documentation with accurate description and diagram Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
Backend: - Add data channel to FalClient for parameter forwarding - Modify FrameProcessor.update_parameters() to route to fal when enabled - Spout parameters always handled locally, pipeline params forwarded Frontend: - Add CloudModeState type and cloudMode to SettingsState - Add CloudModeToggle component with connect/disconnect functionality - Add cloudMode section to SettingsPanel - Add localStorage persistence for cloud credentials Tests: - Add 6 tests for FalClient parameter forwarding - Add 4 tests for FrameProcessor parameter routing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The CloudModeToggle component was added to SettingsPanel but the cloudMode and onCloudModeChange props were not being passed from StreamPage where SettingsPanel is rendered. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Skip JWT token auth for public fal apps (use_auth=False default) - Add compression=None to websockets.connect() to avoid protocol errors - Increase connection timeout to 120s to handle cold starts - Forward initialParameters (including pipeline_ids) with WebRTC offer - Auto-load pipeline from initialParameters in fal_app.py - Add verbose logging for debugging connection issues - Fix RTCPeerConnection config to use RTCConfiguration object Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update Docker image tag to 9c0146b - Increase keep_alive from 300s to 1800s (30 min) to reduce cold starts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
- Poll /api/v1/pipeline/status until pipeline is ready (up to 3 min) - Remove pipeline_ids from initialParameters to avoid double-load race - Update Docker image tag Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
- Use pipeline_id (singular) instead of loaded_pipelines - Check status == "loaded" to confirm pipeline is ready Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
Frame processor needs pipeline_ids to know which pipeline to use. Since we now wait for pipeline to fully load, double-load is a no-op. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
Remove pre-loading and status polling from fal_app.py. The backend's offer endpoint already handles pipeline loading: - Auto-loads from initialParameters if pipeline not loaded - Awaits the load (blocks until complete) - Then processes the WebRTC offer This matches the backend-fal-v2 approach and avoids race conditions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: emranemran <emran.mah@gmail.com>
- Fix handle_get_ice_servers() call in fal_app.py (was passing unused arg) - Add null check for frame_processor in input_loop to prevent race condition during initialization Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.