AI 晶片設計
Nemotron 3 Ultra Integrated with ACE-RTL Iterative Agent Achieves a 97.1% Average Pass Rate Across Nine Chip Design Task Categories
NVIDIA put Nemotron 3 Ultra into a fixed generation, simulation, and reflection loop, achieving a 97.1% average pass rate across nine categories of RTL tasks in CVDP. The results also show that the agent framework can substantially improve scores across models, although the comparison remains limited to three open models and a single evaluation pipeline.
NVIDIA announced a new evaluation of Nemotron 3 Ultra for agentic RTL programming on July 26. Rather than asking the model to generate Verilog from a prompt in a single attempt, the experiment integrated it with ACE-RTL: the generator produces or modifies RTL, the reflector interprets failure messages from simulators and checking tools, and the coordinator organizes debugging context across iterations, deciding which specifications, code, and error evidence should be retained for the next generation step. This more closely resembles the workflow of hardware engineers repeatedly compiling, simulating, linting, and fixing designs.
Across CVDP’s nine task categories, with the same ACE-RTL framework held fixed and only the underlying model changed, Nemotron 3 Ultra achieved an average pass rate of 97.1%, ahead of Kimi K2.6 at 95.2% and GLM 5.2 at 92.1%. On the `cid016` debugging and repair task alone, the three models improved from 65.7%, 68.6%, and 44.0% without the agent to 100%, 97.1%, and 94.3%, respectively, after agent integration. These differences serve as a reminder to evaluators that RTL capability is not determined solely by model weights; test feedback, history compression, and reflection prompts may also strongly influence the final success rate.
Nemotron 3 Ultra used an average of 6,629 tokens per iteration, fewer than GLM 5.2’s 9,156 and Kimi K2.6’s 22,579. The model itself is a hybrid Mamba-Attention Mixture-of-Experts model with 550B total parameters, 55B activated per token, and a one-million-token context window. Its training data also deliberately includes specification generation, editing of existing RTL, and repair examples containing injected state-machine, handshake, and timing errors. The data pipeline performs syntax checks, benchmark contamination checks, and LLM rubric-based scoring.
However, this remains a vendor-led internal comparison that does not include commercial frontier models, nor does it directly translate token counts into latency, energy consumption, or cost on identical hardware. The next step is to determine whether external teams can reproduce the ACE-RTL and CVDP results, and whether the agent-generated designs can pass formal verification, achieve timing closure, and complete a real RTL-to-GDS flow—not merely pass simulation tests.