13. Streaming vocoders and chunk boundaries

What you will learn

• How GAN and inverse-STFT vocoders turn mel frames into samples, and why each output sample depends on mel frames on both sides of it.
• What a vocoder's receptive field is, and why it sets the right context a chunked vocoder must wait for.
• Measured on a CPU: what happens at the joins when a non-causal vocoder is chunked with no context, with left context, with right context, and with crossfades; and why a causal codec decoder has no joins.
• The recipes real systems ship (CosyVoice 2 and 3, FireRedTTS-1S, XTTS, F5-TTS, Orpheus), and what each costs in latency.

Before you start

• Chapters: 2 (spectrograms, the dropped phase), 10 (the pipeline), 11 (causal and non-causal codecs), 12 (the acoustic models that feed the vocoder).
• Tools: the ssh-tts-cpu environment from code/tts/README.md.

Vocoders

A vocoder turns mel frames into a waveform. Two families cover most of this book's systems.

GAN vocoders upsample the mel frames with transposed convolutions until there is one value per sample, with stacks of dilated convolutions in between, and are trained against discriminators that judge whether the result sounds real. HiFi-GAN V1 upsamples 22.05 kHz audio by 8 × 8 × 2 × 2 = 256 samples per mel frame, with residual blocks of kernel sizes 3, 7 and 11 and dilations 1, 3 and 5 (hifi-gan config_v1.json), and runs "167.9 times faster than real-time on a single V100 GPU" (Kong et al., arXiv:2010.05646, abstract). BigVGAN is a larger member of the family.

Inverse-STFT vocoders predict the spectrum's magnitude and phase per frame and let an inverse short-time Fourier transform produce the samples, which removes most of the upsampling work. Vocos is one: on the same A100 its authors measured it at 6,697 times real time against 496 for HiFi-GAN and 99 for BigVGAN, and on a CPU at 170, 5.8 and 0.40 times (Siuzdak, arXiv:2306.00814, Table 6). CosyVoice's HiFT vocoder combines both ideas: a neural source-filter model generates an excitation signal from the pitch, GAN-style upsampling (8 × 5 × 3) follows, and a small inverse STFT (hop 4) finishes, 480 samples per mel frame (cosyvoice/hifigan/generator.py, lines 378-424, with upsample_rates: [8, 5, 3] and hop_len: 4 in cosyvoice2.yaml lines 97-101; cosyvoice2.yaml lines 89-111).

All of these use convolutions padded on both sides. Every output sample depends on mel frames before and after it.

Receptive field and right context

The receptive field of a vocoder is the span of input frames that one output sample depends on. For a chunked vocoder it is the number that matters: to produce the last sample of a chunk exactly as an offline run would, the vocoder must already have the frames that sample depends on after the chunk's end. That is right context, and waiting for it is added latency.

The receptive field can be estimated from the configuration, adding (k − 1)·d/2 for each convolution of kernel k and dilation d, or measured directly: change one input frame and see which output samples move. For Vocos (24 kHz, 100 mel bins, hop 256 samples = 10.67 ms), the book's script measured it: changing frame 279 moved the output, by more than 0.1% of the largest change, from 21 frames before to 22 frames after it (code/tts/results/ch13_chunk_boundaries.json). So an exact chunk boundary for Vocos needs context on both sides: about 21 frames of left context, which costs nothing because it has already arrived, and 22 frames, 235 ms, of right context, which must be waited for. The configuration-based estimate, ±29 frames, is an upper bound; the measured number is smaller because the outermost taps contribute almost nothing.

The same estimate for HiFi-GAN V1 gives about ±148 ms, and for CosyVoice's non-causal HiFT about ±263 ms. Qwen3-TTS's paper states the figure for the BigVGAN behind its 25 Hz tokenizer: it "introduces an extra right-context look-ahead (130 ms)"; the 12 Hz tokenizer of the GPU track has no BigVGAN and a "pure left-context streaming codec decoder" (arXiv:2601.15621, p. 6).

Measured: chunking a vocoder

I decoded the same mel spectrograms (two English and two Mandarin clips) with Vocos (charactr/vocos-mel-24khz) on four CPU threads of an Intel i5-12600K, in one call as the reference, and then in chunks of 8, 16 and 32 frames under seven policies (code/tts/ch13_chunk_boundaries/chunk_boundaries.py). Each chunk can have left context (frames before it, decoded and thrown away; free, since they already exist), right context (frames after it, decoded and thrown away; they must be waited for) and a crossfade (an overlap blended linearly into the next chunk; also waited for). For each policy the script measures the error against the reference in a 5 ms window at every join, relative to the signal there: 0 dB means the error is as loud as the speech. The whole-clip SNR is the same ratio over the whole clip, signal energy over error energy: higher is better.

Policy, 16-frame chunks	Added look-ahead	Median error at joins	Worst join (mean over clips)	Whole-clip SNR
no context	0 ms	+0.5 dB	+9.5 dB	−1.1 dB
crossfade 2 frames	21 ms	+0.9 dB	+9.0 dB	−1.2 dB
left 8	0 ms	−1.9 dB	+5.0 dB	0.6 dB
left 8, right 4	43 ms	−2.5 dB	+8.0 dB	2.0 dB
left 8, right 2, crossfade 2	43 ms	−1.6 dB	+9.7 dB	2.0 dB
left 16, right 8, crossfade 4	128 ms	−15.0 dB	+3.7 dB	10.3 dB
left 32, right 32	341 ms	−113.8 dB	−97.7 dB	108.4 dB

Read it in three steps.

1. Without enough right context, nothing works. No context, a short crossfade, left context alone, or a few frames of right context all leave errors about as loud as the speech at the joins, and audible clicks. A crossfade blends two wrong signals; it cannot make them right.
2. Error falls as right context approaches the receptive field. Eight frames of right context and a 4-frame crossfade, 128 ms, bring the median join to −15 dB: much better, still not exact.
3. Past the receptive field the joins disappear. With 32 frames on each side, more than the measured 21 before and 22 after, the chunked output equals the offline one to −114 dB, rounding error. The cost is 341 ms of added look-ahead, all of it from the right side. Every policy with less right context here also had 16 frames of left context or less, so this run cannot say how much of the remaining error was missing left context.

Chunk size hardly moves the error at the joins (naive: +0.6, +0.5 and +0.4 dB at 8, 16 and 32 frames). It changes the error inside the chunks: a short chunk lies wholly within a receptive field of an edge, so its interior is wrong too (for left 16, right 8, crossfade 4: −21.5, −37.4 and −108.1 dB at 8, 16 and 32 frames). The full table is in the result file.

Causal decoders have no joins

A causal codec decoder depends only on the past, so a decoder that keeps its state across chunks produces exactly what an offline run produces, with no right context at all. On the same CPU, I decoded the same Mimi codes in one call, in chunks decoded with fresh state each time, and frame by frame inside Mimi's streaming context, which carries state across calls:

Mimi decoding (24 kHz, 80 ms frames)	Added look-ahead	Median error at joins	Whole-clip SNR
1-frame chunks, fresh state each	0 ms	−0.1 dB	2.8 dB
4-frame chunks, fresh state each	0 ms	+0.1 dB	7.1 dB
frame by frame, streaming state	0 ms	−114.9 dB	118.4 dB

The causal design is not enough by itself: chunks decoded with fresh state are as bad as an unprepared non-causal vocoder, because the decoder starts each chunk without the history it was trained with. Carry the state, and the joins vanish at zero added latency. This is why chapter 11's causal codecs, Mimi, EnCodec 24 kHz and the Qwen3-TTS 12 Hz tokenizer ("a pure left-context streaming codec decoder", arXiv:2601.15621, p. 6), are the streaming choice.

What the shipped systems do

System	Recipe	Added wait for right context at the vocoder
CosyVoice 2	8 mel frames of left cache, the excitation of the overlap reused from the previous chunk, Hamming crossfade of 3,840 samples, last 3,840 samples held back	160 ms, plus 120 ms of look-ahead tokens upstream
CosyVoice 3	causal HiFT that needs 8 mel frames of right context, re-vocodes all mel so far and slices off what was sent; no crossfade	160 ms, plus 120 ms upstream
FireRedTTS-1S	BigVGAN "pseudo-streaming": each chunk prefixed with the previous 8 frames, overlap crossfaded	8 mel frames
XTTS	re-decodes all latents every 20 tokens, linear crossfade of 1,024 samples, last 1,024 held back	42.7 ms, plus the 20-token chunk
F5-TTS	none when streaming; in non-streaming mode, a linear crossfade of 0.15 s between text batches	none within a batch
Orpheus (SNAC)	decodes 4-frame windows and keeps the second frame	170.7 ms

Sources: CosyVoice cosyvoice/cli/model.py lines 264-318 and cosyvoice/hifigan/generator.py lines 565-567 ("use cache_source to avoid glitch") and 595-725; FireRedTTS-1S (arXiv:2503.20499, Sec. 2.2.1: "the generated waveform for this chunk is overlapped with the generated waveform corresponding to the previous eight frames by fade-in/fade-out crossfading"); coqui-ai/TTS xtts.py lines 585-608; F5-TTS utils_infer.py lines 52 and 576-588; Orpheus decoder.py lines 62 and 106-108.

CosyVoice 2's 160 ms cache is shorter than its HiFT vocoder's estimated receptive field, so its joins are not exact; the excitation reuse keeps the pitch signal's phase continuous across the join, and the crossfade hides the rest. The crossfade itself has a property worth knowing. CosyVoice 2 blends with the two halves of a Hamming window (np.hamming(2 * self.source_cache_len)). For two identical signals, the halves add up to a constant gain of 1.08, +0.67 dB across the whole overlap. For two unrelated signals, their energies add up to as little as 0.58, −2.3 dB, in the middle of the overlap. A linear crossfade sums to exactly 1 for identical signals and dips by 3 dB for unrelated ones.

A real overlap is between these cases: two decodings of the same frames are similar but not identical. So a crossfade between two good decodings is close to transparent, and a crossfade between two bad ones produces a small level dip on top of their errors.

Artefacts and what fixes them

• Clicks at joins come from a step in the waveform. Context on both sides covering the receptive field removes them; a crossfade smooths a small one.
• Level dips in the overlap come from crossfading unrelated signals. They are smaller with a short overlap and vanish when the two decodings agree.
• Pitch and phase discontinuities come from an excitation or a phase that restarts each chunk. Carrying the state across chunks fixes them; CosyVoice 2's excitation reuse is a partial version.
• Prosody resets at chunk or sentence boundaries come from the acoustic model, not the vocoder: a new chunk or sentence that starts without the previous one's context restarts the intonation. No vocoder setting fixes them; the acoustic model needs the context (chapter 12).

Common misconceptions

"A crossfade fixes chunk boundaries." Crossfades with 4 frames of right context or less left the join error near 0 dB in the measurement above. Context on both sides covering the receptive field fixes joins; a crossfade only smooths what is left.

"A causal decoder streams by itself." Only if its state is carried across chunks. With fresh state per chunk, Mimi's joins were as bad as an unprepared non-causal vocoder's.

"Vocoder latency is its compute time." The right context it needs is usually larger: about 235 ms for Vocos to be exact, 160 ms in CosyVoice 2's compromise.

Try it

CPU track

1. Run python ch13_chunk_boundaries/chunk_boundaries.py from code/tts/ in the ssh-tts-cpu environment. It writes all decodings under $BOOK_DATA/runs/ch13_chunk_boundaries/ and 3-second listening clips in code/tts/results/ch13_audio/.
2. Listen to vocos_C16_naive.wav, vocos_C16_left16_right8_xfade4.wav and vocos_C16_left32_right32.wav from that folder, in that order. Count the clicks in the first.
3. Add a policy with 32 frames of left context and vary only the right context. Find the smallest right context at which the 16-frame Vocos joins fall below −60 dB, and predict it from the measured receptive field first.
4. Kokoro's iSTFTNet vocoder is part of the model and is not chunked by the pipeline. Explain from chapter 10 why Kokoro never has vocoder joins inside a sentence, and where its joins are instead.

GPU track

The GPU track's Qwen3-TTS 12 Hz tokenizer has a causal decoder with a sliding window of 72 frames (decoder_sliding_window_frames in code/tts/results/ch11_codecs.json). From its 4-token packets, compute how many packets of history the window covers, and check in the vLLM-Omni code (chapter 16) whether the server carries that state across packets.

English and Mandarin

The measurement used two clips of each language and reports their average. Find the per-clip rows in the raw output and check whether the Mandarin clips, recorded at 16 kHz and so empty above 8 kHz, show smaller join errors than the English ones. In my run they did not: Vocos's per-clip join errors overlapped between the languages, and Mimi with fresh state was worse on the Mandarin clips. Check whether yours agree.

Key terms

• vocoder: turns mel frames into a waveform; GAN-based (HiFi-GAN, BigVGAN) or inverse-STFT (Vocos).
• receptive field: the span of input frames one output sample depends on.
• left context / right context: frames before or after a chunk given to the vocoder and thrown away; right context must be waited for.
• crossfade: blending the end of one chunk into the start of the next over an overlap.
• hold-back: samples a streaming vocoder keeps until the next chunk arrives, to crossfade them.
• excitation (source) signal: the pitch-driven signal of a source-filter vocoder such as HiFT.
• prosody reset: intonation restarting at a boundary because the acoustic model lost context.

Sources

• Kong et al., "HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis", arXiv:2010.05646; jik876/hifi-gan (commit 4769534), config_v1.json.
• Siuzdak, "Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis", arXiv:2306.00814; gemelo-ai/vocos (commit eb39abf).
• FunAudioLLM/CosyVoice (commit 074ca6d): cosyvoice/cli/model.py, cosyvoice/hifigan/generator.py, cosyvoice/utils/common.py, examples/libritts/cosyvoice2/conf/cosyvoice2.yaml.
• "FireRedTTS-1S: An Upgraded Streamable Foundation Text-to-Speech System", arXiv:2503.20499.
• Hu et al., "Qwen3-TTS Technical Report", arXiv:2601.15621.
• coqui-ai/TTS (commit dbf1a08), TTS/tts/models/xtts.py; SWivid/F5-TTS (commit 45868ab), src/f5_tts/infer/utils_infer.py; canopyai/Orpheus-TTS (commit e64661f), orpheus_tts_pypi/orpheus_tts/decoder.py.
• kyutai-labs/moshi (commit e6a55d2), moshi package 0.2.13, and the Mimi weights of kyutai/moshiko-pytorch-bf16 (revision 2bfc9ae, tokenizer-e351c8d8-checkpoint125.safetensors); charactr/vocos-mel-24khz (revision 0feb3fd).