Skip to content

fix: gracefully truncate context on long conversations - #32

Open
ArvindShah wants to merge 1 commit into
MisoLabsAI:mainfrom
ArvindShah:unboundedContext
Open

fix: gracefully truncate context on long conversations#32
ArvindShah wants to merge 1 commit into
MisoLabsAI:mainfrom
ArvindShah:unboundedContext

Conversation

@ArvindShah

Copy link
Copy Markdown

Summary

  • Bug: generate() concatenates all context segments unconditionally, then raises a ValueError only after tokenization is complete. For multi-turn conversations this causes unpredictable crashes once the accumulated frames exceed the 2048 limit.
  • Fix: Implements a sliding-window truncation strategy that drops the oldest segments to fit within the frame budget, keeping the most recent conversational context intact.
  • Warning: Emits a warnings.warn() when segments are dropped, so callers can detect and handle the truncation without catching exceptions.

Changes

  • Compute the frame budget upfront (max_seq_len − max_generation_len − gen_text_frames)
  • Tokenize context segments individually, tracking per-segment frame counts
  • Drop oldest segments iteratively until total context fits within budget
  • Improved error message when input text alone exceeds the budget
  • No API changes — existing callers work without modification

Test plan

  • Run python run_misotts.py with the default 4-turn conversation (fits within context — no truncation, same output as before)
  • Extend the conversation list to 10+ long utterances to trigger truncation — verify a UserWarning is emitted and audio still generates correctly
  • Pass a very long single text string that exceeds the budget on its own — verify a clear ValueError is raised with an actionable message

…tions

Previously, generate() concatenated all context segments unconditionally
and raised a ValueError only after all tokenization was complete. For
multi-turn conversations this made the function unpredictably crash once
the accumulated frames exceeded the 2048 limit.

Now the method uses a sliding-window strategy: it computes the frame
budget upfront, then drops the oldest segments until the context fits.
A warnings.warn() is emitted when segments are dropped so callers can
detect and handle the truncation without catching exceptions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant