Xiaomi unveils HySparse2, reducing million-token prefill compute to about one-fifth of the baseline

On September 23, Luo Fuli of Xiaomi’s MiMo team publicly introduced HySparse2, positioning it as the core architecture of MiMo-V3. The research addresses an input workload common to agents: models generate short instructions, but tools return large volumes of documents, web pages, and execution logs, increasing the cost of prefill and caching conversation history at each turn. The paper was submitted on September 22. [Author announcement mirror](https://twstalker.com/_LuoFuli), [Paper record](https://arxiv.org/abs/2609.26368)

HySparse2 divides the model into a self-decoder and a cross-decoder. At the outer level, KV Bridging lets full-attention layers in the latter half project key-value caches from hidden states in the first half. At the inner level, subsequent sparse layers reuse the caches and selection indices from full-attention layers. This makes the information needed to build the input caches available earlier, allowing prefill to end once the self-decoder finishes. [Author’s architecture explanation](https://twstalker.com/_LuoFuli)

Another change replaces block-level selection with token-level selection and makes the inclusion of recent context mandatory in sparse attention, replacing the previously separate sliding-window branch. This allows local and distant context to share caches and reduces the waste of reading entire blocks of neighboring content to access just a few relevant positions. [Author’s technical explanation](https://twstalker.com/_LuoFuli)

The paper compares three MoE configurations, each with 80 billion total parameters and approximately 3 billion active parameters per token. At one million tokens with FP8 caches, HySparse2 requires about one-fifth of the prefill floating-point operations of Hybrid SWA, while its KV cache shrinks from 12.09 GB to 2.69 GB. However, the configuration also switches from GQA to MQA, so the cache savings cannot be attributed entirely to cross-layer sharing. [Experimental setup and analysis](https://arxiv.org/html/2609.26368v1)

The quality results also need to be considered separately: long-context evaluations after post-training extend to 256k, where HySparse2 scores 58.45 on RULER-v2, compared with 35.74 for Hybrid SWA. General reasoning and coding capabilities show both gains and declines. The million-token figures concern resource analysis; the paper does not establish full usability through quality testing at that same length. [Evaluation results](https://arxiv.org/html/2609.26368v1)

For inference systems engineers, this offers a design direction for shortening the processing path for long inputs. But the fivefold difference comes from computational analysis; actual latency still depends on kernel efficiency, data movement, and scheduling. Follow-up work should examine reproducible implementations, success rates on real-world multi-turn tasks, and time to first token. These considerations also suggest that deployment evaluations should record batch sizes and tool-response lengths and test Chinese-language documents and coding tasks spanning multiple files, rather than estimating agents’ ability to complete real tasks solely from synthetic retrieval scores.