SoL-Pi cuts agent costs with four mechanisms, retaining about 94% of baseline EdgeBench scores

Teams from NVIDIA, Nanyang Technological University and MIT released research on SoL-Pi on September 17, using automated experiments to improve the execution harness for coding agents. The system selected four mechanisms from 152 directions and 535 runnable environments, focusing on the costs of tool round trips and resending context; model weights were not updated. The researchers established capability and efficiency thresholds before testing candidate modifications, and only the retained harnesses proceeded to independent evaluation. [Paper](https://arxiv.org/abs/2609.20519)

The four mechanisms address different sources of waste. Action Fusion combines editing and subsequent validation commands into a single tool call. Online Context Compact estimates at the end of a subtask whether future savings can offset the cost of rewriting the cache. ObservationPack stores large outputs locally, passing references and excerpts in subsequent context and retrieving the original text when needed. Evidence-Preserving Reducer uses a cheaper model to first read build and test logs, then checks quotations against their sources, retaining the original output if verification fails. [Technical overview](https://nvlabs.github.io/SoL-Pi/)

This verification approach suggests that traceable quotations still do not guarantee that important information has not been omitted. Workflows that rely on long logs for diagnosis should measure how often the original text is retrieved, missed errors and retry costs together, to avoid saving on the current input only to increase subsequent investigation cycles. Particularly when error clues span multiple outputs, task outcomes must still validate the choice of content to retain.

Across 51 public EdgeBench tasks, the authors report that the full efficiency configuration reduced token traffic by 44.7% to 49.0% across two models compared with Pi, while cutting API costs by roughly one-third. These savings were not entirely lossless: GPT-5.6 Sol’s average score fell from 44.8 to 42.0, while Opus 5’s fell from 44.8 to 42.2. The traffic figures include cache reads and writes and should not be interpreted as halving generated text. Costs were also calculated using API prices from August 17. [Results tables](https://arxiv.org/html/2609.20519v1)

External tasks further highlight the trade-offs: across 63 CPU-only Terminal-Bench 4 tasks, SoL-Pi solved 15, compared with 18 for Pi. Although total spending fell, cost per successful task should still be compared alongside the completion rate. The research also retains manual steps for setting directions, reviewing results and refactoring code; continuous recursive improvement remains a future goal. [Project evaluations and limitations](https://nvlabs.github.io/SoL-Pi/)

The public release provides a Pi extension under the MIT License, with all four mechanisms disabled by default and testing conducted against Pi 0.85.1. Adopters must separately confirm whether logs may be sent to an auxiliary model and determine retention periods for local archives. The next engineering step is to enable the mechanisms individually and measure success rates, cache hits and actual bills to confirm that the savings carry over to their own workloads. [Repository](https://github.com/NVlabs/SoL-Pi)