A Hands-on Guide to Streaming Speech Recognition and Synthesis

A handbook for engineers starting out with streaming speech recognition and streaming speech synthesis: from audio as numbers, through how streaming ASR and streaming TTS models work, to measuring their accuracy, quality and latency, serving them, and the latency budget of a real-time voice agent. Every example is in English and Mandarin, and every hands-on part has a CPU track and a GPU track.

Start with the Preface. The code behind every measurement, and every result file the chapters cite, is in the code repository.

Text and figures: CC BY 4.0. Code: Apache License 2.0. A path written in a chapter as code/... is the same path in the code repository.