語音模型
X-AuT Progressively Prunes Qwen3-ASR’s Audio Tower, Cutting In-Vehicle Encoding Latency by 21.4%
XPeng uses behavioral probes, cross-scale distillation, and LoRA to shrink the Qwen3-ASR-0.6B audio encoder from 18 layers to 14. The weights and inference code are public, but the full distillation pipeline and more than 280,000 hours of training data have not been released.
XPeng AI has released X-AuT, an effort to compress only the audio tower of a large speech model while retaining the existing text decoder. Directly removing Transformer blocks changes the audio embeddings passed to the decoder, often causing extensive omissions or premature EOS output. X-AuT therefore does not treat per-layer importance as a fixed rule; instead, it uses short-cycle “behavioral probes” to empirically test how recoverable different layer-removal configurations are.
Compression is performed in two stages, from 18→16→14 layers. At each stage, intermediate representations, bridge outputs, and logits are first aligned, followed by cross-scale distillation from a frozen Qwen3-ASR-1.7B teacher. During training, the decoding context is gradually switched from the teacher’s output to the student’s own output, before LoRA fine-tuning is applied at a lower learning rate. The 0.6B student’s language-model backbone remains frozen, with only the decoder attention adapter and selected output components participating in recovery. Under the same 16-layer configuration, the larger teacher yields a macro-average error rate of 5.55%, compared with 8.45% for self-distillation, indicating that teacher scale is important for correcting the interface shift caused by layer removal.
Across ten public Chinese and English ASR benchmarks, the 16-layer version reduces the macro-average CER/WER from 5.61% for the original model to 5.27%. The 14-layer version scores 5.75%, while reducing audio-tower parameters from approximately 186.38M to 147.79M. The latter cuts encoder latency by 21.4% on an in-vehicle PPU and 11.4% on an H800, but end-to-end latency improves by only 4.7% and 2.6%, respectively, because the unpruned 28-layer text decoder still dominates total processing time.
The release includes the complete 14-layer weights, a standalone inference script, and a simplified LoRA example, but excludes the behavioral probes, code for the first two distillation stages, and the original data pipeline. All benchmark results are also based on a single best checkpoint, with no evaluation across multiple random seeds or confidence intervals; the 14-layer model outperforms the baseline on only two benchmarks. The key questions are therefore whether the full training process can be reproduced and whether the method can deliver similar gains across languages, model architectures, and streaming batch configurations.