模型發布與推論架構
Qwen3.8-Flash-Next Combines Linear Attention with Block-Sparse Retrieval, Previewing the Qwen4 Architecture
Alibaba has released the Qwen3.8-Flash-Next weights. The model uses Gated DeltaNet to compress most historical state, while Qwen Sparse Attention retrieves information from the full context. It natively supports 262K tokens, but the official million-token performance and agentic evaluations have yet to be independently reproduced.
Alibaba’s Qwen team has released Qwen3.8-Flash-Next, positioning it as an experimental preview of the next-generation Qwen4 architecture. Its language backbone has 125B parameters, with approximately 6B activated per token. It also includes a separate 51B n-gram embedding and a 4B multi-token prediction module. The model includes a vision encoder and has a native context length of 262,144 tokens, extendable to one million tokens via YaRN. The weights are released under `qwen-community-1.0`, with loading paths available for Transformers, vLLM, SGLang, and TokenSpeed.
The central change is the arrangement of its 48 layers into a repeating pattern of “three Gated DeltaNet layers followed by one Qwen Sparse Attention layer.” DeltaNet compresses history into a fixed-size recurrent state, preventing each layer’s KV cache from growing linearly with sequence length. QSA aggregates tokens into micro-blocks, estimates each block’s importance, and then selects up to 512 blocks—2,048 tokens in total—from the full context for exact attention. Compared with building a sparse index token by token, this makes the indexing cost of long contexts easier to control, but it also means that details missed by the retriever cannot enter that layer’s attention computation.
According to official figures, the QSA kernel delivers up to 7.6× faster prefill and 4.9× faster decoding than full attention. In a specific serving test with a one-million-token context and a 90% prefix-cache hit rate, prefill throughput was 8.6× that of Qwen3.7-Plus. NVIDIA separately measured peak throughput of more than 16,000 tokens/s per GPU using FP8 on a GB300 NVL72, but this is a Pareto-optimal peak recorded on rack-scale hardware and cannot be used to directly infer single-machine latency or typical concurrency costs.
The model card reports scores of 62.5 on SWE-bench Pro and 58.7 on DeepSWE 1.1. Some evaluations used in-house benchmarks, revised tasks, or the higher result from two harnesses, so these figures should not yet be treated as an independent ranking. Engineering teams should next examine the maturity of QSA kernels across frameworks, retrieval failures in long contexts, and the real-world costs of the 51B n-gram parameters in terms of GPU memory, offloading bandwidth, and quantization quality.