語音模型
Xiaomi-CocktailASR-1 Uses Voiceprint Prompts to Target Speech in Mixed Audio, Cutting AliMeeting Far-Field CER to 20.63%
Xiaomi has proposed an end-to-end target-speaker ASR system that feeds a one- to four-second reference utterance and mixed audio into the same audio encoder, without first performing speech separation. The model can also output an empty string when the target speaker is absent, but its training relies heavily on non-public data, and the initial report provides no verifiable model weights or code.
When multiple people speak simultaneously, conventional speech recognition systems typically produce a combined transcription of all voices. Target-speaker ASR (TS-ASR), by contrast, must use a voiceprint to transcribe only a designated person. In the Xiaomi-CocktailASR-1 technical report submitted by Xiaomi on September 10, a one- to four-second reference utterance, one second of silence, and the mixed audio to be recognized are concatenated in sequence. The reference segment serves as a voiceprint prompt, avoiding the two-stage errors introduced by first separating the target speech and then recognizing it.
The architecture begins with a 600-million-parameter audio encoder derived from Data2Vec2, which converts FBank features into 1,280-dimensional representations containing both semantic and speaker information. A linear adapter then maps the audio representations into the embedding space of a Qwen3-8B-based language model. The team trained the system in four stages—base ASR, target-speaker training, chain-of-thought data training, and final supervised fine-tuning (SFT) and reinforcement learning—using approximately 400,000 hours of multi-speaker data, 600,000 hours of single-speaker data, and 10,000 hours of negative samples in which the reference speaker is absent from the recording.
On two-speaker English LibriMix mixtures, the report gives a target-speaker word error rate (WER) of 4.11%, a 15.1% relative reduction from the listed TCP baseline of 4.84%. On two-speaker LibriSpeechMix, it reduces the previous result from 5.40% to 2.90%. For real-world far-field data, the system achieves a WER of 21.81% on AMI-SDM and a Chinese character error rate (CER) of 20.63% on AliMeeting-Far; the latter is lower than the 27.50% reported for MC-TS-ASR in the cited comparison. By training on negative samples with mismatched voiceprints, the model also learns to output an empty string directly, eliminating the need for deployers to configure a separate rejection threshold.
This unified interface is well suited to meetings, wearable devices, and smart homes, but the current evidence comes primarily from evaluations designed by the team itself. Most of the roughly one million hours of training audio is not publicly available, and the baselines do not all use comparable amounts of data or levels of task-specific optimization. The so-called chain-of-thought data is generated from templates based on the number and gender of speakers and external CAM++ similarity scores, so it should not be treated as a faithful explanation of the model’s decisions. As of the paper’s initial release, the public project page also lists neither model weights nor inference code. Real-world latency, memory requirements, and independent reproducibility therefore remain the next key tests.