推論系統
Delaying the Submission of Ready Turns in Agentic Workflow Scheduling Cuts P95 Completion Time by Up to 71.4% Under Congestion
New research separates a turn becoming ready from its submission to the inference engine, using CVaR tail risk and a dynamic workload budget to determine submission order. SWE-bench trace replays achieved up to a 3.5× P95 speedup, but the results cover only fixed agent trajectories, a single randomized arrival seed, and an A100/vLLM environment.