Skip to content

perf: switch podcast TTS to multi-speaker single request #10

Description

@logosc

Problem

The podcast agent currently generates audio by making one TTS API call per script segment sequentially. A typical podcast has ~47 segments, each taking ~8s → ~6 minutes total for audio generation.

Benchmark: 47 segments, 9.27MB MP3, 5m59s generation time.

Solution

Gemini TTS supports multiSpeakerVoiceConfig which allows two speakers in a single request. Since our podcast uses exactly two hosts (A and B), the entire script can be sent as one conversation.

Tested with a 6-segment sample:

  • 1 API call instead of 47
  • 10.6s response time → 21s MP3 with two distinct voices (Kore + Charon)

Request format

{
  "generationConfig": {
    "responseModalities": ["AUDIO"],
    "speechConfig": {
      "multiSpeakerVoiceConfig": {
        "speakerVoiceConfigs": [
          {"speaker": "A", "voiceConfig": {"prebuiltVoiceConfig": {"voiceName": "Kore"}}},
          {"speaker": "B", "voiceConfig": {"prebuiltVoiceConfig": {"voiceName": "Charon"}}}
        ]
      }
    }
  }
}

Implementation plan

  1. Add generateMultiSpeakerTTS() — formats segments as A: text\nB: text\n..., uses multiSpeakerVoiceConfig
  2. Chunking fallback — if script exceeds ~8000 chars, split into segment groups and run up to 4 parallel workers
  3. Replace the sequential for-loop in GenerateAudioTool.Execute
  4. Remove old single-voice generateTTS() function

Expected result

Audio generation drops from ~6min to <30s for a typical podcast.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions