模型與推論系統
GLM-5.3-Flash Cuts Active Parameters to 18B with Hybrid Linear/Sparse Attention
Z.ai has released the weights for its first natively multimodal GLM-5 model. Of its 320B total parameters, only 18B are activated at a time, and it supports context lengths of up to 1 million tokens. Official measurements show 3× less attention computation and a 4.4× smaller KV cache than GLM-5.3, though the performance and cost figures remain largely vendor-reported.
Z.ai released the [GLM-5.3-Flash weights](https://huggingface.co/zai-org/GLM-5.3-Flash) on August 26. Rather than a quantized version of an existing model, it is a natively text, image, and video model pretrained from scratch. It has 320B total parameters, with MoE routing activating approximately 18B per token, while the network has been reduced from 92 layers in the GLM-4.5 series to 45. Z.ai says the training corpus comprised 30T multimodal tokens.
The main architectural change is an interleaving of linear attention and sparse global retrieval. Linear attention stores local dependencies in a fixed-size state, preventing the KV cache from growing proportionally with the context length, while sparse global retrieval uses a lightweight indexer to recover long-range information. A new IndexPool component then uses weighted pooling to compress four index keys into one, reducing indexing latency and memory usage in 1-million-token scenarios. Calculated per layer and per head, Z.ai claims that attention computation and average BF16 KV cache requirements are respectively 3.0× and 4.4× lower than those of the full GLM-5.3, although its cache remains larger than those of Kimi-K3 and DeepSeek-V4-Flash.
In terms of capabilities, the company reports that the DeepSWE v1.1 score rose from 46.2 for GLM-5.2 to 63.4, while the AutomationBench score increased from 26.2 to 48.8. The tests used context windows ranging from 400K to 1M tokens and execution budgets of up to six hours, with some tasks also evaluated by LLM judges. The table therefore should not be treated as an independent ranking under standardized cost conditions.
For deployment, the model is already supported by vLLM, SGLang, Transformers, TokenSpeed, and KTransformers, and it provides a `reasoning_effort` setting for controlling the inference budget. Z.ai’s production system also separates multimodal encoding, prefill, and decode into independent resource pools, combined with W8A8, hybrid cache quantization, and tensor parallelism. The next points to watch are whether third parties can reproduce its accuracy at 1 million tokens, real-world KV cache usage, and throughput. The 320B-weight footprint also means that “18B active” does not imply that the complete model can reside on a single GPU.