OpenAI Publishes Jalapeño Benchmarks: 700W Inference ASIC Delivers Latency and Throughput-per-Watt Advantages Across Three Open Models

OpenAI has published the first benchmark results for Jalapeño, its first-generation in-house inference ASIC. The tests used SemiAnalysis’s InferenceX to run GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T with nominal 8K-token inputs, 1K-token outputs, and single-token prediction (STP). Rather than comparing peak compute performance per chip, the benchmark measured how much mixed-token work each system could complete per kilowatt while meeting specified latency targets.

Jalapeño has a rated power consumption of 700W, while OpenAI says sustained power draw remained below 550W across all three tests. Compared with the selected NVIDIA systems, it delivered 1.5× to 1.9× higher peak throughput per watt and 1.7× to 3.6× lower end-to-end latency. In the DeepSeek R1 test, Jalapeño achieved 19,641 mixed TPS per kilowatt, versus 11,781 for GB300. Its minimum inter-token latency was 1.43 ms, equivalent to roughly 700 tokens per second per user. On Kimi K2.5, peak performance per watt was approximately 1.5× higher, while end-to-end latency was 3.4× lower.

A key architectural focus is keeping prefill, decode, KV cache, and inter-chip communication within a single, explicitly configurable system domain, reducing the movement of model state. Its programming model describes workloads using local tensors, explicit communication, and predictable synchronization, making it easier for compilers and AI systems to handle mapping and scheduling. OpenAI says Codex, working with GPT‑Astra, completed core optimizations for the three additional models within two months. Some AI-generated implementations of GPT‑OSS attention and MoE blocks were 1.5× to 1.8× faster than existing hand-written kernels, although these figures do not represent whole-model speedups.

The results should still be interpreted cautiously. SemiAnalysis observed InferenceX running on site but did not independently execute the complete test suite. The raw data was provided by OpenAI, and the single-turn 8K/1K workload did not cover the long contexts, shared prefixes, routing, and KV-cache pressure found in real-world agent workloads. Jalapeño uses HBM4, making comparisons with the Rubin generation, which has already begun shipping, more appropriate than comparisons with Blackwell. The chip is not available for external sale, and OpenAI does not plan to begin internal deployment until later this year. Key areas to watch next include production yield, rack-level power consumption, AgentX results, and availability in production services.