程式代理／推論最佳化
SWE-Pruner Pro Reads Coding Models’ Hidden States Directly, Cutting Agent Tokens by Up to 39%
SWE-Pruner Pro replaces a separate classifier with a lightweight prediction head that uses a coding agent’s internal representations to decide whether to retain each line of tool output. Tests with two open MoE models across four multi-turn benchmarks showed token savings, but the method requires inference-server modifications, and some quality gains have yet to be reproduced.
A team from Shanghai Jiao Tong University and Douyin has introduced SWE-Pruner Pro, a method targeting the large contexts that accumulate as coding agents repeatedly read files, search code, and execute commands. The original SWE-Pruner requires an additional classification model and asks the agent to generate a goal prompt on every turn. The new method instead uses the final-layer hidden states already produced during the main model’s normal prefill. A small prediction head is attached at the end of each line to directly output `keep` or `prune`. The model also adds a length-aware embedding encoding the number of lines in the tool output and uses balanced focal loss to address class imbalance between retained and pruned samples.
The researchers tested two open MoE models, MiMo-V2-Flash and Qwen3-Coder-Next, on four multi-turn benchmarks: SWE-QA, SWE-QA-Pro, Oolong, and SWE-Bench Verified. The authors report reductions of up to 39% in total prompt and completion tokens. MiMo-V2-Flash’s resolution rate on SWE-Bench Verified increased by 3.8 percentage points, while its accuracy on Oolong rose by 2.2 points. Their explanation is that removing irrelevant tool output not only reduces cost but can also prevent stale information from interfering with subsequent reasoning.
This is not a text compressor that can be placed in front of an arbitrary API. The prediction head must access the model’s internal activations, and the implementation uses a modified version of SGLang. As a result, deployments based on closed model services or exposing only standard OpenAI-compatible endpoints cannot adopt it directly. The paper estimates that the in-server prediction head increases wall-clock time for the full agent trajectory by about 15%. Whether shorter contexts offset that overhead depends on the volume of tool output, the KV cache strategy, and multi-tenant batching. Future work should assess whether the released code can reproduce the cross-model results and whether a pruner trained on specific agent trajectories might mistakenly remove critical evidence when applied to new repositories, tool formats, and task distributions.