推論系統與評測
AgentX Publishes Long-Context Agent Inference Results, Replacing Fixed-Length Stress Tests with Real-World Session Shapes
SemiAnalysis has published live AgentX results and a complete replay toolkit for comparing GPUs, inference engines, and cache designs using multi-turn coding-agent traces. The public corpus preserves request lengths, shared prefixes, pauses, and subagent topologies while replacing the original prompts and code with synthetic content.
SemiAnalysis’ InferenceX now features live AgentX results covering hardware including the MI355X, GB300/GB200 NVL72, B200, H200, and H100, as well as several large open models. It aims to address a blind spot in conventional inference benchmarks: fixed, single-turn requests with 8K-token inputs and 1K-token outputs cannot reflect workloads in which coding agents repeatedly expand their context, pause while waiting for tools, spawn subagents, and reuse the KV cache.
The AgentX v1.0 corpus was converted from voluntarily collected Claude Code sessions and contains 393 sessions with 135,282 requests. The median input per request is approximately 142,016 tokens, while the median output is only 444 tokens. This “extremely long prefill, extremely short decode” ratio shifts the bottleneck away from raw decoding throughput toward prefix-cache hit rates, KV capacity and offloading, conversation-aware routing, long-prompt scheduling, and bursty subagent traffic. A 256K version is also available, removing requests that exceed the context window to facilitate testing servers with shorter native context windows.
The benchmark replays traces through NVIDIA AIPerf’s `inferencex-agentx-mvp` scenario, preserving the original request timing, cache warm-up, cache busting, full response generation, and random seed. Runs that violate the rules are marked with `submission_valid: false`, while the public code, workflow artifacts, and dashboards can be traced back to individual tests. Compared with publishing a single tokens/s figure, this makes it easier to isolate issues in routers, serialization, or external KV layers.
The limitations are equally important: NVIDIA’s documentation still labels the implementation as a work-in-progress MVP, and the specification and fields may change. Because the prompt text is synthetic, the corpus preserves traffic structure rather than task semantics and cannot measure model problem-solving quality. Engineers should wait for cross-team reproductions, energy data, and sensitivity analyses across different corpus versions before translating leaderboard gaps into procurement or capacity-planning conclusions.