代理系統／自動化研究
Dream-RSI Turns Agent Search Histories Into a Replay Simulator to Rewrite Exploration Strategies Offline
Research teams including Google enable a fixed coding agent to repeatedly rewrite its search schedule instead of updating model weights. Tests across eight algorithmic, mathematical, and GPU kernel tasks show that it can reduce exploration costs, although replay can evaluate only branches already traversed in the recorded history.
Google, Google DeepMind, and other institutions have introduced Dream-RSI, an effort to improve one of the most expensive parts of long-horizon program exploration: an agent must not only generate candidate solutions, but also decide which branches are worth extending, when to try alternatives in parallel, and when to stop. Rather than fine-tuning the underlying model, the system adds an executable exploration policy alongside a fixed coding agent and evaluator, then updates that policy through a three-stage loop.
During the online stage, the system records every candidate program, workspace state, parent-child branch relationship, execution result, and cost in a discovery tree. It then treats this tree as a replay simulator: a new policy can change the order in which existing nodes are visited, parallelization settings, and stopping conditions while reading stored results directly, without calling the agent or running evaluations again. Another fixed LLM generates large numbers of policy programs; after they are filtered in the replay environment, the best version is deployed for the next round of online exploration.
The [paper](https://arxiv.org/abs/2609.14858) evaluates the method on eight tasks. In the Lasso solver experiment, agent calls using Gemini 3.1 Pro fell from 550 with a fixed exploration strategy to 317. On KernelBench, the VGG16 and LayerNorm tasks reached comparable speed with 2.43× and 1.79× fewer generations, respectively, while ConvDiv achieved up to 2.09× performance under a comparable budget. Results for mathematical optimization were less consistent: performance improved on Sum–Difference, matched a strong baseline on Circle Packing, but did not outperform SimpleTES on Autocorrelation.
The engineering value lies in turning expensive trajectories into reusable test data for the control plane, making the approach suitable for kernel tuning, program search, and other tasks with objective evaluators. However, it is not a world model capable of predicting arbitrary unseen states. Replay knows only the actual outcomes of nodes that have already been explored, so it cannot reliably evaluate counterfactual policies that would leave the historical tree. The experiments also depend on Gemini, fixed evaluators, and a set of only eight tasks. Although the [project page](https://www.dream-rsi.com/) provides links to the paper and code as well as demonstrations, future work should still examine cross-model transfer, simulator bias, and the full cost after accounting for construction of the initial search tree.