Skip to content

local VAD implementation#507

Merged
Magmusacy merged 19 commits intomainfrom
FCE-3021
Mar 25, 2026
Merged

local VAD implementation#507
Magmusacy merged 19 commits intomainfrom
FCE-3021

Conversation

@Magmusacy
Copy link
Collaborator

Description

Added local VAD implementation that shows whether local peer is talking or not. I added useLocalVAD hook that is being called in useVAD and displays the VAD on local peer depending on whether he's present in PeerIDs array

Motivation and Context

This change was requested by our client

Documentation impact

  • Documentation update required
  • Documentation updated in another PR
  • No documentation update required

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to
    not work as expected)

@linear
Copy link

linear bot commented Mar 24, 2026

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds client-side (local) voice activity detection (VAD) by exposing a local track audio level API from the WebRTC layer up to the React hook layer, and updates the mobile example UI to highlight active speakers.

Changes:

  • Expose getLocalTrackAudioLevel(trackId) from webrtc-clientts-client for consumers.
  • Add useLocalVAD and integrate it into useVAD to report speaking state for the local peer when included in peerIds.
  • Update the mobile chat example to use useVAD and visually indicate speaking peers.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
packages/webrtc-client/src/webRTCEndpoint.ts Adds a public method to fetch local track audio level via the local track manager.
packages/webrtc-client/src/tracks/LocalTrack.ts Implements getAudioLevel() by reading RTCRtpSender.getStats() for audio-level stats.
packages/webrtc-client/src/tracks/Local.ts Wires getLocalTrackAudioLevel() to look up the local track and query its audio level.
packages/ts-client/src/FishjamClient.ts Exposes getLocalTrackAudioLevel() to SDK users (via internal webrtc endpoint).
packages/react-client/src/hooks/useVAD.ts Integrates local VAD into the existing VAD hook and adjusts mic-track selection.
packages/react-client/src/hooks/useLocalVAD.ts New polling-based local VAD hook using the local microphone track’s audio level.
examples/mobile-client/fishjam-chat/components/VideosGrid.tsx Uses useVAD and adds a speaking indicator border in the example UI.
examples/mobile-client/fishjam-chat/app.json Adds iOS entitlements and a hard-coded Apple Team ID for the example app.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Magmusacy Magmusacy marked this pull request as ready for review March 24, 2026 17:13
Copy link
Collaborator

@MiloszFilimowski MiloszFilimowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Just few more questions.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Collaborator

@MiloszFilimowski MiloszFilimowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@Magmusacy Magmusacy merged commit 0af2f17 into main Mar 25, 2026
2 checks passed
@Magmusacy Magmusacy deleted the FCE-3021 branch March 25, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants