開放模型／持續訓練
Thomson-1.0-Small Strengthens Domain Expertise Through Continued Training on 200B Tokens, but Its Weight License Restricts Production Use
Building on Qwen3.6-35B-A3B, Thomson Reuters has released Thomson-1.0-Small, which has 35B total parameters and activates 3B per token. The model improves on legal, tax, and research-agent evaluations, but most results were measured by the development team, and the PolyForm Strict license is not a conventional open-source license.
Thomson Reuters, Imperial College London, DatologyAI, and Lambda have released the Thomson-1.0-Small weights and technical report, seeking to demonstrate that organizations can transform an existing open-weight model into a domain-specialized model without pretraining from scratch. It retains the Mixture-of-Experts (MoE) architecture of Qwen3.6-35B-A3B, with 35B total parameters and approximately 3B parameters activated per token. Its native context window is 262,144 tokens, and its weights use BF16.
The training was not a single narrow-domain fine-tuning run, but a three-stage continual-learning process. The team first used Constitutional DPO to realign the model’s values with the Public AI Constitution. It then selected 200B tokens from a candidate pool of more than 19T tokens for continued pretraining. The dataset consisted roughly of proprietary professional documents, synthetic rewrites, and general-capability replay, with model merging used to mitigate catastrophic forgetting. The final stage combined DPO, reinforcement learning, domain ontologies such as the legal IRAC framework, and a Deep Research harness with rewards for tool use and citations. The complete process consumed approximately 1.63×10²³ FLOP, equivalent to 35,207 B200 GPU-hours.
The unweighted overall average in the official evaluation was 74.6, compared with 71.7 for the base Qwen model. It also improved on the Harvey Legal Agent Benchmark, document processing and retrieval-augmented generation (RAG), and general-agent tasks. The gains were not universal, however: on Stanford LegalBench, coding, mathematics, and multilingual capabilities, it still trailed some comparison models, indicating that replay can mitigate—but not eliminate—capability trade-offs. Engineering teams should also note that although the weights can be loaded with Transformers, vLLM, and SGLang, the PolyForm Strict 1.0.0 license restricts commercial and production use. Key next steps include independent reproductions, data-contamination checks, and assessments of the cost-effectiveness of continued training versus RAG or smaller domain-specific models under equivalent compute budgets.