最新模型
Inkling 2 Small Opens Apache 2.0 Weights; Native NVFP4 Deployment Requires at Least 500 GB of GPU Memory
Thinking Machines made Inkling 2 Small weights available on November 12, extending a multimodal model previously limited to hosted service use to self-hosting and fine-tuning. The model uses a mixture-of-experts (MoE) architecture with 594B total parameters and 17B active parameters. Engineering teams still need to account for GPU memory, context length, and inference framework limitations.

Thinking Machines updated its official announcement on November 12, saying that Inkling 2 Small weights are available for download. The model repository on Hugging Face lists the Apache 2.0 license. The model originally launched through Tinker on November 5; the substantive change this time is that the weights are open, allowing teams to deploy, fine-tune, and integrate the model into products themselves. Official announcement
Inkling 2 Small accepts text and images and outputs text. It has 594 billion total parameters, with about 17 billion parameters active per token. The model card describes a 48-layer decoder architecture in which each token selects 8 of 384 experts, combining local and global attention. Sparse activation reduces compute requirements for deployment, but the full weights still require storage and GPU memory; hardware requirements cannot be estimated from the active parameter count alone. Model card
The company provides NVFP4 and MXFP8 checkpoints, requiring at least 500 GB and 800 GB of aggregate GPU memory, respectively. NVFP4 was used from pretraining through post-training, and the company recommends this native-precision version. The hardware thresholds listed in the model card still need to be assessed alongside actual context length and concurrent request requirements. Model card
The vLLM deployment guide further illustrates the cost of long contexts: its validated configuration supports up to 262,144 tokens on a single four-GPU tray, while a full one-million-token context uses two trays with TP8. The guide requires a dedicated image until support is included in a stable release, and provides a configuration with the DSpark speculative decoder included. This means framework version and serving parameters still directly affect available functionality after the weights are opened. vLLM deployment guide
Fine-tuning and agent integration also have limitations. The vLLM guide notes that expert-layer LoRA requires BF16 expert weights, so existing low-precision checkpoints should use attention-layer adapters. Although the tool-call parser has unit tests, end-to-end tool-calling benchmarks are not yet complete. Engineering teams should next validate success rates, latency, and memory use with their own Traditional Chinese tasks, toolchains, and traffic before choosing a deployment configuration. vLLM deployment guide