AI 研究
External Memory Lets Frozen-Weight Agents Learn From Deployment Feedback, Raising τ-bench Single-Run Success by Up to 2.6×
New research distills the outcome of each agent run and post-run corrections into retrievable rules, allowing experience to accumulate without updating model weights. The results were replicated across Mistral Large and Claude Sonnet 5 on τ-bench banking tasks, though the evaluation used perfect feedback and repeated each scenario four times.
A new study proposes using external memory instead of continual fine-tuning. After an agent completes a task, the same model condenses the conversation, tool trajectory, and feedback into a single `WHEN–THEN` natural-language rule, which it proactively searches when it later encounters a relevant situation. Model weights remain frozen throughout, and writes occur only after a task ends. When a run fails and no ground-truth answer is available, the memory can record only what to avoid; the correct action may be stored only after a verified correction is obtained, reducing the risk that the agent will commit guesses to long-term knowledge.
The authors tested 97 tasks in the banking domain of τ-bench, running each task four times. For Mistral Large, the static RAG baseline using the full policy document achieved a `pass^1` of 0.064. Adding a single bit of success-or-failure feedback raised it to 0.103, while adding the correct action sequence after a failure increased it to 0.170—about 2.6 times the baseline. Of the 84 tasks that the baseline failed on all four attempts, corrective memory enabled the agent to solve 22 at least once. Claude Sonnet 5’s `pass^1` likewise rose from 0.248 to 0.397.
The researchers also swapped the read-only memory stores created by the two models. Mistral reached 0.289 when using Sonnet’s rules, while Sonnet reached 0.314 with Mistral’s rules. This suggests that some experience can transfer across models rather than merely preserving prompt shortcuts specific to one model. The experimental harness, data, and prompt excerpts have been released publicly to facilitate reproduction.
The limitations are equally clear: each condition received only one complete run, and feedback came from a deterministic grader, with no simulation of delayed, erroneous, or malicious human corrections. Repeating the same task four times also makes it easier to form reusable rules than in most production traffic. Moreover, all three Mistral conditions had a `pass^4` of 0.031, indicating that learning from some repeated scenarios does not translate into reliable success across four consecutive attempts. Future work should test noisy feedback, cross-task generalization, memory conflicts, deletion and provenance tracking, and recovery when a long-term memory store is contaminated by prompt injection.