VideoAvatar handling does not allow per-agent session customization

[Viktoriagrg]

Bug Description

The VideoAvatar logic assumes that a single AgentSession is tied to one video avatar. This design prevents assigning separate avatars to different agents within the same session, even though logically it should be possible.

As a result:

• We cannot have multiple agents within the same session each using a distinct video avatar.
• The current implementation restricts flexibility, forcing one avatar per session instead of allowing per-agent avatar configuration.

Expected Behavior

• Each agent within a session should be able to have its own video avatar.
• The system should still support shared session data while allowing per-agent avatar customization.

Reproduction Steps

async def start(
        self,
        agent_session: AgentSession,

Operating System

macOS

Models Used

No response

Package Versions

livekit==1.0.20
livekit-agents==1.3.6

Session/Room/Call IDs

No response

Proposed Solution

Additional Context

No response

Screenshots and Recordings

No response

[longcw]

hot swap avatar is supported in #5863