Conversation
There was a problem hiding this comment.
Pull request overview
Adds client-side (local) voice activity detection (VAD) by exposing a local track audio level API from the WebRTC layer up to the React hook layer, and updates the mobile example UI to highlight active speakers.
Changes:
- Expose
getLocalTrackAudioLevel(trackId)fromwebrtc-client→ts-clientfor consumers. - Add
useLocalVADand integrate it intouseVADto report speaking state for the local peer when included inpeerIds. - Update the mobile chat example to use
useVADand visually indicate speaking peers.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/webrtc-client/src/webRTCEndpoint.ts | Adds a public method to fetch local track audio level via the local track manager. |
| packages/webrtc-client/src/tracks/LocalTrack.ts | Implements getAudioLevel() by reading RTCRtpSender.getStats() for audio-level stats. |
| packages/webrtc-client/src/tracks/Local.ts | Wires getLocalTrackAudioLevel() to look up the local track and query its audio level. |
| packages/ts-client/src/FishjamClient.ts | Exposes getLocalTrackAudioLevel() to SDK users (via internal webrtc endpoint). |
| packages/react-client/src/hooks/useVAD.ts | Integrates local VAD into the existing VAD hook and adjusts mic-track selection. |
| packages/react-client/src/hooks/useLocalVAD.ts | New polling-based local VAD hook using the local microphone track’s audio level. |
| examples/mobile-client/fishjam-chat/components/VideosGrid.tsx | Uses useVAD and adds a speaking indicator border in the example UI. |
| examples/mobile-client/fishjam-chat/app.json | Adds iOS entitlements and a hard-coded Apple Team ID for the example app. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
packages/webrtc-client/tests/methods/getLocalTrackAudioLevelMethod.test.ts
Show resolved
Hide resolved
packages/webrtc-client/tests/methods/getLocalTrackAudioLevelMethod.test.ts
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
packages/webrtc-client/tests/methods/getLocalTrackAudioLevelMethod.test.ts
Show resolved
Hide resolved
MiloszFilimowski
left a comment
There was a problem hiding this comment.
Great work! Just few more questions.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description
Added local VAD implementation that shows whether local peer is talking or not. I added useLocalVAD hook that is being called in useVAD and displays the VAD on local peer depending on whether he's present in PeerIDs array
Motivation and Context
This change was requested by our client
Documentation impact
Types of changes
not work as expected)