Appendix A. Unit and frame-rate conversion tables¶
Every streaming speech system runs several clocks at once. These tables convert between them. Each rate is taken from the chapter given, where its source is cited; each other entry is arithmetic on those rates.
Samples, bytes and bit rates¶
| Format | Samples per second | Bytes per second | Bit rate | Chapter |
|---|---|---|---|---|
| 8 kHz, 16-bit mono | 8,000 | 16,000 | 128 kbit/s | 1 |
| 16 kHz, 16-bit mono (recognizers) | 16,000 | 32,000 | 256 kbit/s | 1 |
| 16 kHz, float32 mono | 16,000 | 64,000 | 512 kbit/s | 1 |
| 24 kHz, 16-bit mono (most synthesizers) | 24,000 | 48,000 | 384 kbit/s | 1, 10 |
| 48 kHz, 16-bit mono (WebRTC) | 48,000 | 96,000 | 768 kbit/s | 1, 17 |
| 44.1 kHz, 16-bit stereo (CD) | 88,200 | 176,400 | 1,411.2 kbit/s | 1 |
| G.711 telephony | 8,000 | 8,000 | 64 kbit/s | 1 |
| Opus speech in WebRTC | — | 4,000 | 32 kbit/s | 1, 17 |
One 160 ms chunk at every clock¶
The running example of the book is a 160 ms chunk of 16 kHz audio.
| Clock | Unit | Units in 160 ms | Chapter |
|---|---|---|---|
| 16 kHz samples | 62.5 µs | 2,560 (5,120 bytes as int16) | 1 |
| feature frames, 10 ms hop | 10 ms | 16 | 2 |
| icefall Zipformer encoder output (4x) | 40 ms | 4 | 3, 5 |
| NeMo FastConformer encoder (8x) | 80 ms | 2 | 3, 5 |
| Whisper encoder | 20 ms | 8 | 8 |
| EnCodec 24 kHz, DAC 24 kHz codec frames | 13.3 ms | 12 | 11 |
| CosyVoice 2 and 3 speech tokens | 40 ms | 4 | 11, 12 |
| CosyVoice 2 and 3 mel frames | 20 ms | 8 | 10, 11 |
| Kokoro decoder frames | 12.5 ms | 12.8 | 10 |
| Vocos, BigVGAN-v2 24 kHz mel frames | 10.67 ms | 15 | 10, 13 |
| Mimi and Qwen3-TTS 12 Hz codec frames | 80 ms | 2 | 11 |
| Opus frames in WebRTC | 20 ms | 8 (7,680 samples at 48 kHz) | 17 |
| Silero VAD windows | 32 ms | 5 | 7 |
Feature frames from samples¶
At 16 kHz with a 25 ms (400-sample) window and a 10 ms (160-sample) hop (chapter 2):
| Convention | Frames from N samples | Frames in 1 s |
|---|---|---|
Kaldi snip_edges=true |
1 + (N − 400) // 160 | 98 |
Kaldi snip_edges=false (icefall, sherpa-onnx) |
(N + 80) // 160 | 100 |
| centered STFT (Whisper, then drops the last) | N // 160 + 1 | 101, Whisper keeps 100 |
Front-end look-ahead, how long after its hop ends a frame's window ends: 7.5 ms with snip_edges=false, 15 ms
with snip_edges=true, 2.5 ms with a centered STFT.
Streaming encoder settings in milliseconds¶
NeMo att_context_size [left, right] with 8x subsampling and 10 ms features (chapters 3 and 5):
| Setting | Chunk | Worst look-ahead | Average look-ahead | Left context |
|---|---|---|---|---|
[70,0] |
80 ms | 0 ms | 0 ms | 5.6 s |
[70,1] |
160 ms | 80 ms | 40 ms | 5.6 s |
[70,6] |
560 ms | 480 ms | 240 ms | 5.6 s |
[70,13] |
1,120 ms | 1,040 ms | 520 ms | 5.6 s |
Nemotron [56,13] |
1,120 ms | 1,040 ms | 520 ms | 4.48 s |
icefall streaming Zipformer --chunk-size, counted at 50 Hz (chapters 3 and 15):
--chunk-size |
New audio per step | Feature frames read per step | Encoder output frames | Extra look-ahead |
|---|---|---|---|---|
| 8 | 160 ms | 29 | 4 | 130 ms |
| 16 | 320 ms | 45 | 8 | 130 ms |
| 32 | 640 ms | 77 | 16 | 130 ms |
| 64 | 1,280 ms | 141 | 32 | 130 ms |
Codec bit rates and model steps¶
From chapter 11. "Steps per second" is for a large model that takes one step per codec frame.
| Codec | Frame rate | Codes per frame | Bit rate | Large-model steps per second | Budget per step |
|---|---|---|---|---|---|
| EnCodec 24 kHz at 6 kbit/s | 75 Hz | 8 | 6,000 bit/s | 75 | 13.3 ms |
| Mimi, 8 codebooks | 12.5 Hz | 8 | 1,100 bit/s | 12.5 | 80 ms |
| Mimi, 32 codebooks | 12.5 Hz | 32 | 4,400 bit/s | 12.5 | 80 ms |
| Qwen3-TTS 12 Hz tokenizer | 12.5 Hz | 16 | 2,212.5 bit/s | 12.5 | 80 ms |
| CosyVoice 2 and 3 tokens | 25 Hz | 1 | 317 bit/s | 25 | 40 ms |
| SNAC 24 kHz, flattened (Orpheus) | 82 tokens/s | 1 per token | 984 bit/s | 82 | 12.2 ms |
Smallest units and hold-backs at the output¶
From chapters 11 to 13.
| System | Smallest unit of audio | Waited for before the first sample |
|---|---|---|
| EnCodec 24 kHz | 13.3 ms | one frame |
| Mimi | 80 ms | one frame, plus 80 ms for each step of acoustic delay |
| Qwen3-TTS 12 Hz (paper) | 320 ms packet (4 frames) | the packet's tokens |
| CosyVoice 2, released code | 840 ms first chunk | 28 to 52 speech tokens (1.12 to 2.08 s of speech) |
| Orpheus (SNAC) | 85.3 ms | 28 tokens; 170.7 ms of right context per frame |
| Vocos, exact chunk joins | 10.67 ms frame | about 22 frames (235 ms) of right context |
Decibels¶
| Quantity | Value |
|---|---|
| halving the amplitude | −6.02 dB |
| one 16-bit step, relative to full scale | −90.3 dBFS |
| int16-scale samples fed as floats in a log-mel | +20.79 on every natural-log value (2 ln 32,768) |
| natural log to log10 | divide by 2.303 |
| natural log to dB (10 log10) | multiply by 4.343 |