Success Rates Hold After Agent Context Compression, but GPT-5.5 State Rechecks Still Triple

Long-horizon agents commonly compress earlier trajectories using sliding windows, summaries, or retrieval, while evaluations often check only whether the task was completed. New research submitted on August 17 points to a cost that emerges before outright failure: after agents forget state required for execution, they must spend additional tool calls retrieving the data again, even though their final answers may still be correct.

The researchers built a deterministic planning environment that labels every tool call as either retrieval or execution and caps each run at 24 interaction rounds. The experiments compared full context, a sliding operator that retains only recent content, and summaries that preserve facts within the same token budget. An oracle condition reinjected deleted state to test whether the additional queries were indeed caused by information loss. The three models evaluated were DeepSeek V4 Flash, Qwen3.7 Plus, and GPT-5.5, with paired tests using identical seeds for each primary condition.

Retrieval calls increased across all six model–task-difficulty combinations, and five of those increases remained significant after Holm–Bonferroni correction. Execution call counts, by contrast, remained broadly unchanged. In the most pronounced case—a high-retrieval-demand setting for GPT-5.5—fivefold compression increased average retrieval calls from 21.0 to 63.9, while the change in completion rate from 80% to 85% was not statistically significant. DeepSeek likewise did not show a significant decline in completion rate until compression reached tenfold. Restoring deleted state eliminated about half of the additional queries, while fact-preserving summaries avoided most of the cost.

The findings suggest that agent platforms should not select compression methods based solely on pass rate. They should also track repeated queries, tool latency, and the number of paid API calls. Otherwise, “maintaining accuracy” may simply mean that the agent is compensating for lost information through additional interactions. However, the same sliding operator did not trigger a surge in retrieval calls in a second environment, ALFWorld, indicating that the effect depends on whether state can be retrieved again, the tool interface, and the interaction-round limit. The paper’s interaction cost also does not directly measure monetary expense or wall-clock time, so deployment teams still need to validate the findings in real-world workflows.