語音生成／推論系統
Faster IndexTTS-2 Ports Its Autoregressive Speech Pipeline to TensorRT, Delivering Up to 3.6× End-to-End Speedup
A new study restructures IndexTTS-2’s GPT, Diffusion Transformer, and vocoder using TensorRT and TensorRT-LLM, while adding streaming and batched inference. Experiments on the English and Chinese Seed-TTS benchmarks show up to a 5× speedup for GPT, but the currently available public materials remain insufficient to directly reproduce the complete optimization pipeline.
A research team submitted Faster IndexTTS-2 on July 23, targeting the main production bottlenecks in autoregressive zero-shot text-to-speech. The original IndexTTS-2 consists of an autoregressive GPT, a flow-matching Diffusion Transformer, and a final vocoder. Although it can clone a voice from reference audio and control emotion, its original inference speed only barely reaches real time, and it lacks complete streaming and batch-processing paths.
Rather than retraining an acoustic model, Faster IndexTTS-2 moves all three stages of neural network inference to NVIDIA TensorRT and TensorRT-LLM. The GPT stage benefits from execution and memory optimizations designed for large language models, while the downstream Diffusion Transformer and vocoder are also converted into GPU-optimized engines. The system additionally adds batched inference support to each component to improve GPU utilization when processing multiple speech segments concurrently, and provides streaming output so voice agents can begin playback without waiting for the waveform of an entire sentence to be generated.
In tests on the English and Chinese Seed-TTS benchmarks, the authors report speedups of up to 5× for the autoregressive GPT and up to 3.6× end to end, with only slight degradation in word error rate, speaker similarity, and naturalness. This has practical implications for real-time customer service, game characters, and voice agents: time to first audio, sustained generation speed, and concurrent throughput often have a greater impact on interaction quality than the total time required to generate an audio file offline.
However, the paper is currently only four pages long, and the results still come from the authors’ own testing. Although the original IndexTTS repository provides public models and inference code, it does not yet include a reproducible Faster IndexTTS-2 package that fully corresponds to the paper. Engineers should next examine the actual GPU model, batch size, time to first audio, VRAM usage, and TensorRT engine build time. Comparing only peak throughput figures could obscure the additional scheduling overhead in single-request or short-utterance scenarios.