SGLang-Omni Integrates MOSS-Transcribe-Diarize, Bringing Multi-Speaker Transcription to Continuous Batching

SGLang-Omni has released an official serving recipe for MOSS-Transcribe-Diarize, connecting the 0.9B multi-speaker speech model to the OpenAI-compatible `/v1/audio/transcriptions` endpoint. Applications can upload audio directly and request `verbose_json` to receive text, start and end times, and anonymous speaker labels. For long recordings, `max_new_tokens` can be increased to prevent autoregressive decoding from being cut off before all segments have been generated.

The key technical aspect of this integration is not simply another API wrapper, but the incorporation of the audio model into SGLang-Omni’s multi-stage execution pipeline. MOSS-TD first uses a 24-layer Whisper encoder to convert an 80-bin log-mel spectrogram into continuous representations, merging every four time steps before projecting them through an MLP into the text model’s embedding space. A 28-layer Qwen3 decoder then builds a KV cache and generates a structured transcript containing speaker and timestamp information token by token. The process does not require separately running ASR, alignment, and diarization models and then merging their outputs.

SGLang-Omni can therefore apply its existing CUDA Graph, asynchronous decoding, continuous batching, and KV-cache management capabilities to speech recognition workloads. The official launch example supports configuring the maximum number of concurrent requests, CUDA Graph batch size, and static memory ratio. For teams that already use an OpenAI transcription client, the migration cost is also lower than adopting a custom RPC interface.

The model itself supports more than 50 languages, a 128K context window, recordings up to approximately 90 minutes long, and domain-specific hotword prompting. According to self-reported evaluations in the model card, it outperforms the listed commercial and open baselines on most CER and cpCER metrics across AISHELL-4, Alimeeting, Podcast, and Movies. However, the `movies800times`, `aishell4_long`, and `googletime` datasets referenced in SGLang’s documentation are currently privately licensed, making it difficult for external teams to fully reproduce the serving benchmarks. Future evaluation should focus on throughput and real-time factor on public datasets, speaker-attribution errors in overlapping speech, and whether differences in output length across long-audio batches cause tail latency in scheduling.