代理記憶與評測
Agent Memory Transfers More Reliably When Broken Into Subtasks; Text Skills Lead Code by 2.9 Points on Average
A study spanning nine models and three long-horizon benchmarks finds that skills induced from complete tasks usually make agents perform worse, while those induced from subtasks deliver average gains. Natural-language process notes are also more robust than executable Python skills, though results remain highly dependent on the model and benchmark.
Accumulating “skills” from successful trajectories does not guarantee that agents will improve with use. [A new study](https://arxiv.org/abs/2608.20274) separates skill memory along two design axes: whether skills are induced after completing an entire task or after first decomposing the task and inducing skills from each subtask; and whether they are stored as natural-language operational notes or parameterized Python functions. Each configuration is compared with a memory-free version of the same agent to isolate the positive or negative transfer caused by the skills themselves.
The experiments cover 417 multi-app tasks from [AppWorld](https://github.com/StonyBrookNLP/appworld), 300 document workflows from OfficeBench, and 92 scientific data-analysis tasks from KramaBench. The evaluated models include Qwen3, GPT-OSS-120B, Nemotron-Super-120B, Gemma 3, and Gemini 3.1 Pro. Averaged across models, full-task skills reduce success rates relative to their respective memory-free baselines by 1.2 percentage points for text and 4.1 points for code. Subtask skills, by contrast, raise success rates by 1.9 and 0.5 points, respectively. At a fixed induction level, text skills outperform code skills by an average of 2.9 points for subtasks and 1.4 points for full tasks. Individual results are uneven, however: on AppWorld, for example, adding full-task code skills causes GPT-OSS-120B to fall from 27.3% to 1.0%.
The team also defines “skill utility” as the product of specificity and abstraction, in an attempt to filter out memories that are either overly specialized or overly general before execution. In controlled KramaBench comparisons, high-scoring skills only slightly outperform the low-scoring group, which is not enough to make the metric a reliable gate. For agent engineers, the findings support defining memory units as recomposable operational steps and prioritizing textual procedures that capture environmental constraints, instead of feeding entire solutions or brittle code directly back into the agent. The field must still wait for the code and skill library to be released before determining whether the results can be reproduced with different retrievers, task orderings, and continuous online updates.