代理評測
FinEvo-Bench Tests Cross-Task Agent Evolution Across 120 Financial Tasks, With Codex Gaining 19.37 Points
FinEvo-Bench interleaves related but distinct financial cases to test whether agents can turn prior feedback into reusable memories or skills for subsequent tasks. All four agent harnesses outperformed paired controls that reset state after each task, though the results used only a single model backbone.
The newly released [FinEvo-Bench](https://arxiv.org/abs/2608.06144) attempts to turn the claim that “agents improve through work experience” into a measurable proposition. The dataset spans six financial domains, 20 business scenarios, and 120 multi-file tasks, with 775 input documents in total. Each scenario contains six cases that follow the same professional workflow but differ in facts and conclusions. The researchers randomly interleaved all tasks into three sequences, inserting unrelated work between related cases, and then compared agents that retained experience with agents whose state was cleared after every task.
All four harnesses used Qwen3.7-Max, a temperature of zero, and identical task ordering. They differed in how experience was stored: Claude Code and Codex could write skills and memories; Letta used editable memory blocks that were continuously injected into the context; and GenericAgent maintained a Markdown experience file. This is also consistent with the persistent-state design described in [Letta’s official documentation](https://docs.letta.com/v1-sdk/concepts/stateful-agents). Compared with their paired controls, the four systems improved their average scores by 9.33 to 19.37 points and reduced compliance issues by 0.12 to 0.44 per task. Letta achieved the highest final score at 91.65, while Codex recorded the largest gain at 19.37 points.
The carrier ablation results are especially noteworthy for engineering teams. When Claude Code evolved skills alone, it scored 93.71 with 0.05 compliance issues per task, outperforming configurations that used memory alone or combined skills and memory. Rubric feedback targeting omissions and violations also outperformed full reference answers by 3.95 to 7.93 points, suggesting that reusable procedures and validation rules transfer more readily than stored answers to individual cases.
However, evaluation was still performed by a Claude Opus 4.6 agent. Although its absolute-agreement ICC with a financial expert reached 0.95, the validation covered only one batch of 120 outputs. All experiments also used a single backbone, the financial domain, and non-parametric updates, so the results cannot establish which agent architecture performs best with other models or in other professional workflows. Next steps should include cross-model replications, the release of the dataset and evaluation code, and studies of whether erroneous experience contaminates performance after long-term accumulation.