Intern-S2-Preview-397B Adds a Numerical Forecasting Branch, While a 4B Memory Module Raises the Average Biology Score by 3.40 Points

The InternLM team has published a full technical report for Intern-S2-Preview-397B. Rather than focusing solely on scaling the model, the work brings different representations of scientific data into a shared training pipeline. For visual pretraining, paper pages are fed directly into a frozen vision encoder; after blank regions are removed, the model learns charts, formulas, and layouts through a contrastive next-latent objective. A separate data pipeline uses OCR and layout parsing to construct interleaved text-and-image sequences spanning multiple pages. Post-training combines SFT, multi-task reinforcement learning, black-box and white-box agent RL, and on-policy distillation, with mechanisms including group entropy control and partial rollout correction to handle differences in exploration across tasks.

Time series are no longer represented solely as text or images. The model adds a dedicated encoder and numerical forecasting branch, with a Q-Former fusing semantic and numerical representations before passing them to a causal Transformer to generate future sequences. A horizon predictor determines the forecast length from the instruction. The paper reports multiple advantages over dedicated forecasting baselines on SciTS and records a zero-shot MASE of 0.785 on the general-purpose GIFT-Eval benchmark, although most results were evaluated internally by the team.

Another design with greater deployment relevance is the standalone Intern-MemDec-4B. It distills token distributions obtained through retrieval into a compact parametric memory. During inference, it runs in parallel with the frozen backbone, and a token-level router mixes the two next-token distributions. The biology version raises the average score across 21 Biology-Instructions tasks from 56.92 to 60.32, though performance declines on several subtasks. At this stage, it therefore cannot be regarded as a general-purpose, lossless specialization method. The 397B weights are released under Apache 2.0, and the model card lists deployment options using LMDeploy, vLLM, and SGLang, along with a 256K text context window. However, Hugging Face metadata indicates an actual loaded size of approximately 404B parameters, and no hosted inference provider is available. The hardware requirements, reliance on remote custom code, and the fact that the Memory Decoder has not yet been fully released remain the main adoption risks.