語音模型
VibeVoice-ASR-Streaming Combines Incremental Recognition and Speaker Attribution, Open-Sources 1.5B and 7B Weights
Microsoft Research’s streaming speech model interleaves audio chunks, brief lookahead, and the existing transcript to directly output “who said what.” The public release supports ten languages and custom hotwords, but expected speaker-attribution latency remains two seconds, and performance degrades during prolonged overlapping speech.
Microsoft Research has open-sourced the 1.5B and 7B weights for VibeVoice-ASR-Streaming, along with inference code and a vLLM serving path. Unlike cascaded systems that first produce a transcript and then perform speaker diarization, the new model interleaves fixed-size audio chunks, a small amount of future audio, and previously generated text and speaker tags within a single autoregressive sequence. This allows it to directly output “who said what” as audio continues to stream in.
The public checkpoints use 22-frame audio chunks of approximately 2.9 seconds, with an additional 4 frames, or about 0.5 seconds, reserved for lookahead. Based on this configuration, the paper calculates an expected speaker-attribution latency of 2.00 seconds. The architecture retains low-frame-rate acoustic and semantic tokenizers, while a language model maintains the history of speech, text, and speaker states. It supports ten languages, including Chinese, English, Japanese, and Korean, and accepts names or specialized terms as hotwords, making it suitable for meeting captions and multi-speaker voice agents that need to distinguish speakers in real time.
The authors compared deployed streaming services across four meeting-recording conditions and the nine-language MLC-Challenge. The 7B version achieved the lowest average WER/CER across five transcription-only evaluations. With speaker attribution enabled, it was the best or tied for best in 12 of 13 settings. Compared with Azure ConversationTranscriber, it improved cpWER/cpCER by 2.39 to 12.45 points on meeting benchmarks. However, these results were obtained by the research team using specific normalization procedures and datasets; they are not a comprehensive independent evaluation across languages, noise conditions, and hardware costs.
The model can currently only serialize overlapping speech into a single output stream, and performance degrades significantly when multiple people speak simultaneously for extended periods. Language expansion is also limited by the range supported by Qwen3 ForcedAligner. Engineering teams should next test time to first output and steady-state latency, GPU memory usage, drift during long meetings, and whether hotwords reduce accuracy on general vocabulary.