Skip to content

Latest commit

 

History

History
1950 lines (929 loc) · 52.7 KB

File metadata and controls

1950 lines (929 loc) · 52.7 KB

Protocol Documentation

Table of Contents

Top

fishjam/agent_notifications.proto

AgentRequest

Defines any type of message passed from agent peer to Fishjam

Field Type Label Description
auth_request AgentRequest.AuthRequest
add_track AgentRequest.AddTrack
remove_track AgentRequest.RemoveTrack
track_data AgentRequest.TrackData
interrupt_track AgentRequest.InterruptTrack
capture_image AgentRequest.CaptureImage

AgentRequest.AddTrack

Request to add a track of the specified type

Field Type Label Description
track notifications.Track Specification of the track to be added
codec_params AgentRequest.AddTrack.CodecParameters Parameters of the input data stream

AgentRequest.AddTrack.CodecParameters

Field Type Label Description
encoding notifications.TrackEncoding The format of the input stream, defaults to pcm16
sample_rate uint32 The sample rate of the input stream, may be omitted for opus
channels uint32 The number of channels. 1 means mono, 2 means stereo

AgentRequest.AuthRequest

Request sent by agent, to authenticate to Fishjam server

Field Type Label Description
token string

AgentRequest.CaptureImage

Field Type Label Description
track_id string

AgentRequest.InterruptTrack

Interrupts an agent's outgoing track, preventing already queued audio from being played

Field Type Label Description
track_id string

AgentRequest.RemoveTrack

Removes the given track

Field Type Label Description
track_id string

AgentRequest.TrackData

Notification containing a chunk of an agent's track's data stream

Field Type Label Description
track_id string
data bytes

AgentResponse

Defines any type of message passed from Fishjam to agent peer

Field Type Label Description
authenticated AgentResponse.Authenticated
track_data AgentResponse.TrackData
track_image AgentResponse.TrackImage

AgentResponse.Authenticated

Response confirming successful authentication

AgentResponse.TrackData

Notification containing a chunk of a track's data stream

Field Type Label Description
peer_id string
track notifications.Track
data bytes

AgentResponse.TrackImage

Field Type Label Description
track_id string
content_type string
data bytes

Top

fishjam/media_events/peer/peer.proto

MediaEvent

Defines any type of message sent from Peer to Membrane RTC Engine

SCHEMAS

Field Type Label Description
connect MediaEvent.Connect
disconnect MediaEvent.Disconnect
update_endpoint_metadata MediaEvent.UpdateEndpointMetadata
update_track_metadata MediaEvent.UpdateTrackMetadata
renegotiate_tracks MediaEvent.RenegotiateTracks
candidate fishjam.media_events.Candidate
sdp_offer MediaEvent.SdpOffer
track_bitrates MediaEvent.TrackBitrates
enable_track_variant MediaEvent.EnableTrackVariant
disable_track_variant MediaEvent.DisableTrackVariant
set_target_track_variant MediaEvent.SetTargetTrackVariant
unmute_track MediaEvent.UnmuteTrack

MediaEvent.Connect

Sent when a peer wants to join WebRTC Endpoint.

Field Type Label Description
metadata_json string

MediaEvent.DisableTrackVariant

Sent when client disables one of the track variants

Field Type Label Description
track_id string
variant fishjam.media_events.Variant

MediaEvent.Disconnect

Sent when a peer disconnects from WebRTC Endpoint.

MediaEvent.EnableTrackVariant

Sent when client enables one of the track variants

Field Type Label Description
track_id string
variant fishjam.media_events.Variant

MediaEvent.RenegotiateTracks

Sent when peer wants to renegotiate connection due to adding a track or removing a track

MediaEvent.SdpOffer

Sent as a response to offerData media event during renegotiation Maps contain only information about current peer's sendonly tracks. The "mid" is an identifier used to associate an RTP packet with an MLine from the SDP offer/answer.

Field Type Label Description
sdp string The value of the sessionDescription.sdp
track_id_to_metadata_json MediaEvent.SdpOffer.TrackIdToMetadataJsonEntry repeated
track_id_to_bitrates MediaEvent.SdpOffer.TrackIdToBitratesEntry repeated Maps track_id to its bitrate. The track_id in the TrackBitrates message is ignored (we use the map key), so it can be omitted.
mid_to_track_id MediaEvent.SdpOffer.MidToTrackIdEntry repeated

MediaEvent.SdpOffer.MidToTrackIdEntry

Field Type Label Description
key string
value string

MediaEvent.SdpOffer.TrackIdToBitratesEntry

Field Type Label Description
key string
value MediaEvent.TrackBitrates

MediaEvent.SdpOffer.TrackIdToMetadataJsonEntry

Field Type Label Description
key string
value string

MediaEvent.SetTargetTrackVariant

Field Type Label Description
track_id string
variant fishjam.media_events.Variant

MediaEvent.TrackBitrates

Sent when Peer wants to update its track's bitrate

Field Type Label Description
track_id string
variant_bitrates MediaEvent.VariantBitrate repeated Bitrate of each variant. For non-simulcast tracks use VARIANT_UNSPECIFIED.

MediaEvent.UnmuteTrack

Field Type Label Description
track_id string

MediaEvent.UpdateEndpointMetadata

Sent when a peer wants to update its metadata

Field Type Label Description
metadata_json string

MediaEvent.UpdateTrackMetadata

Sent when a peer wants to update its track's metadata

Field Type Label Description
track_id string
metadata_json string

MediaEvent.VariantBitrate

Field Type Label Description
variant fishjam.media_events.Variant
bitrate int32

Top

fishjam/media_events/server/server.proto

MediaEvent

Defines any type of message sent from Membrane RTC Engine to Peer

Field Type Label Description
endpoint_updated MediaEvent.EndpointUpdated
track_updated MediaEvent.TrackUpdated
tracks_added MediaEvent.TracksAdded
tracks_removed MediaEvent.TracksRemoved
endpoint_added MediaEvent.EndpointAdded
endpoint_removed MediaEvent.EndpointRemoved
connected MediaEvent.Connected
error MediaEvent.Error
offer_data MediaEvent.OfferData
candidate fishjam.media_events.Candidate
sdp_answer MediaEvent.SdpAnswer
vad_notification MediaEvent.VadNotification
track_variant_switched MediaEvent.TrackVariantSwitched
track_variant_disabled MediaEvent.TrackVariantDisabled
track_variant_enabled MediaEvent.TrackVariantEnabled

MediaEvent.Connected

Sent to the peer after connecting to the WebRTC Endpoint.

Field Type Label Description
endpoint_id string
endpoint_id_to_endpoint MediaEvent.Connected.EndpointIdToEndpointEntry repeated
ice_servers MediaEvent.IceServer repeated

MediaEvent.Connected.EndpointIdToEndpointEntry

Field Type Label Description
key string
value MediaEvent.Endpoint

MediaEvent.Endpoint

Field Type Label Description
endpoint_type string
metadata_json string
track_id_to_track MediaEvent.Endpoint.TrackIdToTrackEntry repeated

MediaEvent.Endpoint.TrackIdToTrackEntry

Field Type Label Description
key string
value MediaEvent.Track

MediaEvent.EndpointAdded

Sent to all peers in the room after a new endpoint was added.

Field Type Label Description
endpoint_id string
metadata_json string

MediaEvent.EndpointRemoved

Sent to all remaining peers in the room after some endpoint was removed.

Field Type Label Description
endpoint_id string

MediaEvent.EndpointUpdated

Sent when metadata of one of the endpoints was updated

Field Type Label Description
endpoint_id string
metadata_json string

MediaEvent.Error

Sent to inform that an error occurred on the server providing a message to show

Field Type Label Description
message string

MediaEvent.IceServer

Field Type Label Description
credential string
urls string repeated
username string

MediaEvent.OfferData

Sent to inform about the number of audio and video tracks that will be sent from the engine to the peer

Field Type Label Description
tracks_types MediaEvent.OfferData.TrackTypes

MediaEvent.OfferData.TrackTypes

Field Type Label Description
audio int32
video int32

MediaEvent.SdpAnswer

Sent after receiving SdpOffer from Peer

Field Type Label Description
sdp string The value of the sessionDescription.sdp
mid_to_track_id MediaEvent.SdpAnswer.MidToTrackIdEntry repeated

MediaEvent.SdpAnswer.MidToTrackIdEntry

Field Type Label Description
key string
value string

MediaEvent.Track

SCHEMAS

Field Type Label Description
metadata_json string
simulcast_config MediaEvent.Track.SimulcastConfig

MediaEvent.Track.SimulcastConfig

Field Type Label Description
enabled bool
enabled_variants fishjam.media_events.Variant repeated
disabled_variants fishjam.media_events.Variant repeated

MediaEvent.TrackUpdated

Sent when metadata of one of the tracks was updated

Field Type Label Description
endpoint_id string
track_id string
metadata_json string

MediaEvent.TrackVariantDisabled

Sent when track's variant has been disabled

Field Type Label Description
endpoint_id string
track_id string
variant fishjam.media_events.Variant

MediaEvent.TrackVariantEnabled

Sent when track's variant has been enabled

Field Type Label Description
endpoint_id string
track_id string
variant fishjam.media_events.Variant

MediaEvent.TrackVariantSwitched

Informs that track's variant has been changed

Field Type Label Description
endpoint_id string
track_id string
variant fishjam.media_events.Variant

MediaEvent.TracksAdded

Sent to informs that one of the peers has added one or more tracks.

Field Type Label Description
endpoint_id string
track_id_to_track MediaEvent.TracksAdded.TrackIdToTrackEntry repeated

MediaEvent.TracksAdded.TrackIdToTrackEntry

Field Type Label Description
key string
value MediaEvent.Track

MediaEvent.TracksRemoved

Sent to informs that one of the peers has removed one or more tracks.

Field Type Label Description
endpoint_id string
track_ids string repeated

MediaEvent.VadNotification

Sent to inform that the track denoted by trackId has changed their voice activity For this notification to work, the server must be configured to use VAD extension and the sender must support it.

Field Type Label Description
track_id string
status MediaEvent.VadNotification.Status

MediaEvent.VadNotification.Status

Name Number Description
STATUS_UNSPECIFIED 0
STATUS_SILENCE 1
STATUS_SPEECH 2

Top

fishjam/media_events/shared.proto

Candidate

Contains information about an ICE candidate which will be sent to the peer/server

Field Type Label Description
candidate string
sdp_m_line_index int32
sdp_mid string
username_fragment string

ChannelMessage

Field Type Label Description
source string
destinations string repeated
binary ChannelMessageBinaryPayload

ChannelMessageBinaryPayload

Field Type Label Description
data bytes

Variant

Name Number Description
VARIANT_UNSPECIFIED 0
VARIANT_LOW 1
VARIANT_MEDIUM 2
VARIANT_HIGH 3

Top

fishjam/notifications/shared.proto

Track

Describes a media track

Field Type Label Description
id string
type TrackType
metadata string

TrackEncoding

Name Number Description
TRACK_ENCODING_UNSPECIFIED 0
TRACK_ENCODING_PCM16 1
TRACK_ENCODING_OPUS 2

TrackType

Defines types of tracks being published by peers and component

Name Number Description
TRACK_TYPE_UNSPECIFIED 0
TRACK_TYPE_VIDEO 1
TRACK_TYPE_AUDIO 2

Top

fishjam/peer_notifications.proto

PeerMessage

Defines any type of message sent between FJ and a peer

Field Type Label Description
authenticated PeerMessage.Authenticated
auth_request PeerMessage.AuthRequest
media_event PeerMessage.MediaEvent
rtc_stats_report PeerMessage.RTCStatsReport
peer_media_event media_events.peer.MediaEvent
server_media_event media_events.server.MediaEvent

PeerMessage.AuthRequest

Request sent by peer, to authenticate to FJ server

Field Type Label Description
token string
sdk_version string

PeerMessage.Authenticated

Response sent by FJ, confirming successful authentication

Field Type Label Description
room_type PeerMessage.RoomType
sdk_deprecation PeerMessage.SdkDeprecation

PeerMessage.MediaEvent

Any type of WebRTC messages passed between FJ and peer

Field Type Label Description
data string

PeerMessage.RTCStatsReport

PeerConnection stats sent by peer https://developer.mozilla.org/en-US/docs/Web/API/RTCStatsReport#the_statistic_types

Field Type Label Description
data string

PeerMessage.SdkDeprecation

Deprecation status for SDK version

Field Type Label Description
status PeerMessage.SdkDeprecation.Status
message string

PeerMessage.RoomType

Defines types of rooms peers may connect to

Name Number Description
ROOM_TYPE_UNSPECIFIED 0
ROOM_TYPE_CONFERENCE 1
ROOM_TYPE_AUDIO_ONLY 2
ROOM_TYPE_LIVESTREAM 3

PeerMessage.SdkDeprecation.Status

Name Number Description
STATUS_UNSPECIFIED 0
STATUS_UP_TO_DATE 1
STATUS_DEPRECATED 2
STATUS_UNSUPPORTED 3

Top

fishjam/server_notifications.proto

ServerMessage

Defines any type of message passed between FJ and server peer

Field Type Label Description
authenticated ServerMessage.Authenticated
auth_request ServerMessage.AuthRequest
subscribe_request ServerMessage.SubscribeRequest
subscribe_response ServerMessage.SubscribeResponse
room_created ServerMessage.RoomCreated
room_deleted ServerMessage.RoomDeleted
room_crashed ServerMessage.RoomCrashed
peer_connected ServerMessage.PeerConnected
peer_disconnected ServerMessage.PeerDisconnected
peer_crashed ServerMessage.PeerCrashed
peer_metadata_updated ServerMessage.PeerMetadataUpdated
track_added ServerMessage.TrackAdded
track_removed ServerMessage.TrackRemoved
track_metadata_updated ServerMessage.TrackMetadataUpdated
peer_added ServerMessage.PeerAdded
peer_deleted ServerMessage.PeerDeleted
channel_added ServerMessage.ChannelAdded
channel_removed ServerMessage.ChannelRemoved
track_forwarding ServerMessage.TrackForwarding
track_forwarding_removed ServerMessage.TrackForwardingRemoved
vad_notification ServerMessage.VadNotification
viewer_connected ServerMessage.ViewerConnected
viewer_disconnected ServerMessage.ViewerDisconnected
streamer_connected ServerMessage.StreamerConnected
streamer_disconnected ServerMessage.StreamerDisconnected
stream_connected ServerMessage.StreamConnected Deprecated.
stream_disconnected ServerMessage.StreamDisconnected Deprecated.
hls_playable ServerMessage.HlsPlayable Deprecated.
hls_uploaded ServerMessage.HlsUploaded Deprecated.
hls_upload_crashed ServerMessage.HlsUploadCrashed Deprecated.
component_crashed ServerMessage.ComponentCrashed Deprecated.

ServerMessage.AuthRequest

Request sent by peer, to authenticate to FJ server

Field Type Label Description
token string

ServerMessage.Authenticated

Response sent by FJ, confirming successfull authentication

ServerMessage.ChannelAdded

Notification sent when a peer creates a channel

Field Type Label Description
room_id string
peer_id string
component_id string
channel_id string

ServerMessage.ChannelRemoved

Notification sent when a peer deletes a channel

Field Type Label Description
room_id string
peer_id string
component_id string
channel_id string

ServerMessage.ComponentCrashed

Notification sent when a component crashes

Field Type Label Description
room_id string
component_id string

ServerMessage.HlsPlayable

Notification sent when the HLS stream becomes available in a room

Field Type Label Description
room_id string
component_id string

ServerMessage.HlsUploadCrashed

Notification sent when the upload of HLS recording to AWS S3 fails

Field Type Label Description
room_id string

ServerMessage.HlsUploaded

Notification sent when the HLS recording is successfully uploaded to AWS S3

Field Type Label Description
room_id string

ServerMessage.PeerAdded

Notification sent when a peer is added

Field Type Label Description
room_id string
peer_id string
peer_type ServerMessage.PeerType

ServerMessage.PeerConnected

Notification sent when a peer connects

Field Type Label Description
room_id string
peer_id string
peer_type ServerMessage.PeerType

ServerMessage.PeerCrashed

Notification sent when a peer crashes

Field Type Label Description
room_id string
peer_id string
reason string
peer_type ServerMessage.PeerType

ServerMessage.PeerDeleted

Notification sent when a peer is removed

Field Type Label Description
room_id string
peer_id string
peer_type ServerMessage.PeerType

ServerMessage.PeerDisconnected

Notification sent when a peer disconnects from FJ

Field Type Label Description
room_id string
peer_id string
peer_type ServerMessage.PeerType

ServerMessage.PeerMetadataUpdated

Notification sent when peer updates its metadata

Field Type Label Description
room_id string
peer_id string
metadata string
peer_type ServerMessage.PeerType

ServerMessage.RoomCrashed

Notification sent when a room crashes

Field Type Label Description
room_id string

ServerMessage.RoomCreated

Notification sent when a room is created

Field Type Label Description
room_id string

ServerMessage.RoomDeleted

Notification sent when a room is deleted

Field Type Label Description
room_id string

ServerMessage.StreamConnected

Notification sent when streamer successfully connects

Field Type Label Description
stream_id string

ServerMessage.StreamDisconnected

Notification sent when streamer disconnects

Field Type Label Description
stream_id string

ServerMessage.StreamerConnected

Field Type Label Description
stream_id string
streamer_id string

ServerMessage.StreamerDisconnected

Field Type Label Description
stream_id string
streamer_id string

ServerMessage.SubscribeRequest

Request sent by peer to subscribe for certain message type

Field Type Label Description
event_type ServerMessage.EventType

ServerMessage.SubscribeResponse

Response sent by FJ, confirming subscription for message type

Field Type Label Description
event_type ServerMessage.EventType

ServerMessage.TrackAdded

Notification sent when peer or component adds new track

Field Type Label Description
room_id string
peer_id string
component_id string
track notifications.Track

ServerMessage.TrackForwarding

Sent when there is an upsert to track forwardings from Fishjam to Composition

Field Type Label Description
room_id string
peer_id string
composition_url string
input_id string
audio_track notifications.Track optional Track has id, type, and metadata
video_track notifications.Track optional

ServerMessage.TrackForwardingRemoved

Sent when track forwarding is removed

Field Type Label Description
room_id string
peer_id string
composition_url string
input_id string

ServerMessage.TrackMetadataUpdated

Notification sent when metadata of a multimedia track is updated

Field Type Label Description
room_id string
peer_id string
component_id string
track notifications.Track

ServerMessage.TrackRemoved

Notification sent when a track is removed

Field Type Label Description
room_id string
peer_id string
component_id string
track notifications.Track

ServerMessage.VadNotification

Notification sent when voice activity changes on a track

Field Type Label Description
room_id string
peer_id string
track_id string
status ServerMessage.VadNotification.Status

ServerMessage.ViewerConnected

Notification sent when viewer successfully connects

Field Type Label Description
stream_id string
viewer_id string

ServerMessage.ViewerDisconnected

Notification sent when viewer disconnects

Field Type Label Description
stream_id string
viewer_id string

ServerMessage.EventType

Defines message groups for which peer can subscribe

Name Number Description
EVENT_TYPE_UNSPECIFIED 0
EVENT_TYPE_SERVER_NOTIFICATION 1

ServerMessage.PeerType

Name Number Description
PEER_TYPE_UNSPECIFIED 0
PEER_TYPE_WEBRTC 1
PEER_TYPE_AGENT 2

ServerMessage.VadNotification.Status

Name Number Description
STATUS_UNSPECIFIED 0
STATUS_SILENCE 1
STATUS_SPEECH 2

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)