最新模型
NVIDIA Releases Dory 4B Recurrent Model Weights, Using Adjustable Loops to Increase Inference Compute Depth
On October 16, NVIDIA released Dory 1.0, which runs shared layers repeatedly so the same 4B weights can support different inference compute budgets. The model offers a final-loop KV cache option, but performance results are still based on NVIDIA’s own tests, and integration with major inference frameworks remains pending.
On October 16, NVIDIA released Nemotron Labs Dory 1.0, a 4-billion-parameter language model with a latent recurrent architecture. It lets deployers adjust how many times shared layers run, enabling tests with different compute budgets using the same weights. NVIDIA offers BF16, MXFP8, NVFP4, and Q5_K_M formats, lists support for up to a 256K-token context window, and licenses the model under OpenMDW 1.1. English is currently the only listed supported language. Official model card
Dory divides the network into an eight-layer input block, a 20-layer shared recurrent block, and an eight-layer output block, reinjecting the input at each loop. This means inference depth can be increased without loading a larger checkpoint. However, the compute required per token also increases, so speed and cost cannot be inferred from parameter count alone. Architecture description
In NVIDIA’s official BF16 evaluations, SWE-bench Verified scores were 50.3, 59.5, and 61.8 at one, two, and four loops, respectively. GPQA Diamond scores were 69.19, 72.41, and 72.16, showing that more loops do not consistently improve every task. NVIDIA recommends starting with two loops and then measuring performance on the actual workload. These figures come from NVIDIA’s evaluation setup and should not yet be treated as independently reproduced results or applied directly to quantized versions. Evaluation and deployment settings
The cache strategy is another key factor. Saving KV cache for every loop increases memory requirements as the loop count rises. The last_loop mode instead overwrites the same cache, retaining only the keys and values from the final loop. A llama.cpp support proposal cautions that this changes predictions and is affected by prompt-prefill chunk size. The proposal is still a draft, and full long-context validation has not been completed. llama.cpp support proposal
Developers should next compare task success rates, end-to-end latency, and peak memory across different loop counts and cache modes. The official installation guide still requires changes to vLLM or llama.cpp that have yet to be integrated. Chinese-language capabilities also require separate validation and cannot be inferred from English-language evaluations. Deployment guide