代理執行期與推論系統
Speculative Macro Commit Pre-Executes Agent Toolchains, Cutting AppWorld Wall-Clock Time by 44.9%
New research enables a small draft model to pre-run multi-step tool operations in isolated environment snapshots, then commit the entire sequence once a large authoritative model confirms the first step. The method reduces latency on two agent benchmarks, but requires three GPUs and lowers AppWorld task completion from 41.67% to 40.48%.
Tool-using agent latency comes not only from model token generation, but also from the serial dependency of “decide, call a tool, wait for an observation, then decide again.” Speculative Macro Commit (SMC) moves the concept of speculative decoding into agent execution: a large actor retains formal decision authority, while a small drafter predicts and executes chains of future actions on isolated environment snapshots.
The system first mines successful training trajectories for recurring multi-step tool skeletons, replacing task-specific user IDs, document IDs, or timestamps with slots to form a macro library. At runtime, if the actor’s next tool call matches the first step of a drafted chain, the system can—after parameter, state, and minimum-depth checks—write the remaining pre-executed steps and their observations into the official trajectory in a single commit. The actor does not see any additional macro tools, so it does not need to learn a new action interface.
The study uses INT4 Qwen3.5-27B as the actor and Qwen3.5-4B as the drafter. Across 2,285 results on τ²-Bench Telecom, SMC matches the serial baseline’s 99.52% accuracy while reducing average latency per item from 27.60 seconds to 22.47 seconds; it is also 10.23% faster than Speculative Actions, which reuses only a single step. On AppWorld, average time falls from 355.7 seconds to 195.9 seconds, but the number of tasks achieving their goals declines from 70/168 to 68/168.
Cost and safety boundaries still limit practicality. The serial baseline uses only one GPU, whereas both speculative configurations use three GPUs in total: one for the actor, one for a speculative-request replica, and one for the drafter. Lower wall-clock time therefore does not imply lower total compute cost. The unfiltered precision of macro matching itself is only 34.6%. Although the full rule set preserves the outcomes of all actual commit events across 100 held-out tasks, this is not a general safety guarantee. Future work should reproduce the results on coding agents, APIs with irreversible side effects, different model combinations, and equal hardware budgets, while measuring erroneous commits, rollbacks, and snapshot overhead.