模型架構／推論
Intern-S2-Mobius Shares FFN Memory Globally, With the 35B Model Reporting Nearly 4× End-to-End Inference Acceleration
Intern-S2-Mobius separates the knowledge storage and attention-based reasoning traditionally tied to each Transformer layer, allowing multiple Reasoners to repeatedly query the same global Memory. The open weights can be loaded with LMDeploy, vLLM, or Transformers, but the nearly 4× acceleration remains a result from the official evaluation under specific conditions.
The InternLM team has released the weights for the 35B-class Intern-S2-Mobius. Rather than adding more MoE experts, its core innovation is to reorganize knowledge and computation within the Transformer. In conventional architectures, each layer stores knowledge in its FFN and uses self-attention to operate on the current representation. Mobius-v0 instead shares a single Memory across the entire model, while multiple Reasoners repeatedly use the hidden state to query and combine the knowledge vectors it contains. Its backward residual connection also allows earlier reasoning stages to access deeper knowledge instead of passing information only in one direction through a fixed layer sequence.
According to the [technical report](https://arxiv.org/abs/2608.14290), the 7B version trained from scratch used only 62.6% as much training data as the Transformer baseline while achieving comparable downstream scores. Intern-S2-Mobius, which was continually pretrained from Qwen3.5-35B and then underwent SFT and RL, delivered nearly 4× end-to-end inference acceleration while maintaining similar overall capabilities. The team attributes the gain to dynamic latent reasoning in continuous hidden states, shorter visible reasoning outputs, and higher request throughput—not to a fourfold speedup from any single kernel.
The [model card](https://huggingface.co/internlm/Intern-S2-Mobius) provides Apache-2.0-licensed weights, lists approximately 36B parameters stored in BF16, and includes deployment instructions for LMDeploy, Transformers, and vLLM. The recommended configuration enables MTP speculative decoding with four draft tokens. vLLM additionally requires `trust_remote_code`, the Qwen3 reasoning parser, and a tool-calling parser, so the model is not currently a transparent drop-in replacement for an arbitrary 35B Transformer endpoint.
Engineering teams should next measure the speed gains attributable to the architecture, shorter outputs, and MTP separately, then rerun the evaluation using identical hardware, batch sizes, prompt lengths, and output token counts. The paper's comparisons focus primarily on the team's own models and Qwen3.5-35B; independent, cross-framework validation of throughput, time to first token, long-context quality, and memory usage is still lacking.