Nanbeige4.2-3B Compresses an Agent Model With a Looped Transformer, Supporting a 256K Context With 3B Non-Embedding Parameters

Nanbeige Lab has released Nanbeige4.2-3B under the Apache 2.0 license. The model has approximately 4B parameters in total, including 3B non-embedding parameters, and supports contexts of up to 262,144 tokens. At its core is a Looped Transformer: the same set of layers is executed repeatedly to increase effective computational depth instead of adding more independently weighted layers. This design reduces storage requirements, but it does not mean inference costs are equivalent to those of a conventional 3B model, because repeated passes through the layer stack still consume compute.

The training recipe targets both general reasoning and agentic tasks. The team says the base model was pretrained from scratch on 28T tokens. During SFT, it mixed real and synthetic executable environments, task assets, and multiple agent scaffolds, with data filtered at both the trajectory and single-turn levels. Post-training combined Think/Non-Think RLHF, reinforcement learning that constrains reasoning length, and agentic RL using both outcome and process rewards.

According to the official report, the model scored 63.6 on SWE-bench Verified, 46.9 on SWE-bench Pro, and 44.1 on Terminal-Bench 2.0. In local assistant tasks evaluated with OpenClaw, it also outperformed Qwen3.5-9B across all six tests. The model card provides integration paths for Transformers, SGLang, vLLM, llama.cpp, and Ollama, although some backends require Nanbeige-maintained forks, and loading the Transformers weights requires `trust_remote_code=True`.

Engineering teams should evaluate “fewer parameters” separately from “lower real-world latency, memory usage, and energy consumption.” Several comparisons use Nanbeige's own scaffolds or internal benchmarks, and the inference setup also retains historical thinking by default. Third-party testing across frameworks, with identical token budgets and hardware, is not yet available.