feat(generator): add streaming generation API with chunked audio decode

[mvanhorn]

Summary

Audio can now be consumed while generation is still running. generate_stream() yields ~2 second watermarked PCM chunks as frames decode, so playback can start after the first chunk instead of waiting for the full clip. generate() is unchanged, and no new dependencies are added: the chunked decode reuses Mimi's streaming mode from the moshi package that is already pinned in pyproject.toml.

Why this matters

The generation loop in generator.py is already frame-incremental (one 80 ms Mimi frame per step), but everything buffers until a single batch decode at the end. For a 10 second clip that means ~125 sequential frame generations before any audio exists.

• Users of the parent CSM architecture ask for exactly this: SesameAILabs/csm#163 (streaming decode guidance) and SesameAILabs/csm#78 (realtime voice agents discussion).
• Launch coverage positions MisoTTS for low latency and voiceover work (@kimmonismus, @omarsar0); a streaming path is what makes that real for self-hosters.
• Comparable open models in this class ship streaming as a headline feature, and it is a common axis in community comparisons on r/LocalLLaMA.

Demo

Simulated demo (the 8B model needs a CUDA GPU, so this recreates examples/stream_demo.py output rather than a live capture):

Changes

• The frame loop moves out of generate() into _prepare_prompt() + _generate_frames(). generate() consumes the same iterator and stays byte-identical in behavior, including RNG consumption order.
• Prompt tokenization (including Mimi encode of voice-cloning context audio) happens before entering Mimi streaming mode, so streamed generation conditions on the same context tokens as generate().
• Only full chunk_frames-sized chunks are decoded inside the streaming context. On CUDA, moshi wraps the streaming decoder in a CUDA graph captured at the first chunk's shape, so a shorter residual chunk would raise a shape mismatch. Residual frames flow through a batch tail decode instead, which also keeps total output length equal to a batch decode of the same codes.
• Each chunk is computed under torch.inference_mode() but yielded outside it, so caller code between chunks runs with normal autograd semantics. Chunks are watermarked independently; if SilentCipher rejects a very short terminal fragment, that fragment is yielded unwatermarked with a code comment documenting the tradeoff.

Testing

• tests/test_streaming_generation.py covers the frame-stacking and length-matching helpers (pure Python, no weights or GPU needed).
• Verified with a stubbed Generator that: concatenated stream chunks match the batch decode length for partial-chunk, exact-multiple, and empty-generation cases; inference mode does not leak to the caller between yields; and only uniform-shape decodes happen inside the streaming context.
• Not yet validated end to end on CUDA hardware (none available here). The Mimi streaming and CUDA graph behavior were verified against the moshi 0.2.2 source rather than a live run, so a smoke run of examples/stream_demo.py on a GPU box would be a useful review step.

Mimi's streaming() context manager is the same mechanism moshi's own server path uses for incremental decode; this PR reuses it rather than adding a new decode path.

[AodenTeo]

Great. I definitely want to include something like this in the repo. I'll have a look at this later. Thanks so much for contributing!