代理記憶研究
TEPA Revokes Stale Agent Memories, Raising Task-Reversal Success from 21.0% to 95.0%
TEPA adds a revocable validity state to long-term memory, removing content overturned by new evidence from standard retrieval while preserving an audit trail. The method can resolve single-hop conflicts but becomes almost entirely ineffective on multi-hop tasks or with a 262K long context.
Agent memory typically answers only whether a piece of content is relevant. TEPA requires the system to also determine whether it remains valid. The research team represents observations as precedents containing a conflict key, evidence, status, and contradiction history. When sufficient counterevidence appears for the same key, the old item transitions from active to revoked. It no longer participates in standard retrieval but remains in an archive for auditing or possible future reactivation. The full version also validates candidate updates through support, counterfactual, and contamination checks, preventing a single erroneous tool result from directly overwriting long-term knowledge.
In a hidden-rule reversal experiment with 160 tasks per seed across 50 seeds, append-only and last-write-wins memory both achieved a success rate of just 0.210 during the full-reversal phase, even below the 0.309 rate of the no-memory baseline; TEPA reached 0.950. When real file I/O and tool execution were used, the first three approaches scored 0.203, 0.203, and 0.298, respectively, while TEPA still achieved 0.950. Removing the revocation mechanism reduced its score to 0.211, showing that the main gain indeed comes from moving stale precedents out of the active set rather than merely adding more content to the prompt.
Results on the external MemoryAgentBench SH-6k benchmark were more modest: TEPA achieved a substring exact match score of 0.890, merely tying a cache that directly retains the latest value for each key. Its score fell to 0.680 on SH-32k and just 0.040 on MH-6k, while it scored zero on SH-262k. TEPA addresses the lifecycle of individual facts, not multi-hop chain construction or selection over extremely long contexts. Another limitation is that the experiments used structured conflict keys; adding 20% noise to those keys reduced reversal success to 0.777. For production systems, the worthwhile idea is to store validity separately from relevance. However, reliably extracting conflict keys from open-ended memory—and the paper's lack of an independent implementation repository—remain issues that must be addressed before the results can be reproduced.