JitMem tailors agent memory to new tasks, raising success rates by over 16 percentage points on two benchmarks

A Salesforce AI Research team introduced JitMem on September 23, with the authors explaining its design on Hugging Face the following day: preserve raw action trajectories from an agent’s past tasks, then generate the memory summary needed when a new task arrives. For agents that repeatedly handle similar workflows, this offers a way to rethink when memories are generated. [Authors’ introduction](https://huggingface.co/papers/2609.27334)

Common approaches distill experience into fixed reflections, skills, or workflows when a task ends. Future needs are still unknown at that point, and discarded details may be precisely what the next task requires. JitMem generates different summaries from the same raw experience depending on the query, and adds only trajectories judged successful to its memory store. Full preservation here applies only to trajectories approved for storage; it does not mean unconditionally retaining every attempt. [Research overview](https://arxiv.org/html/2609.27334v1)

The implementation uses BM25 to match task descriptions and retrieve relevant trajectories, which a memory curation model turns into a briefing for an execution model with fixed weights. The curation model is initialized from Qwen3-8B and trained for 100 steps using GRPO, receiving rewards based on the outcome of the same task. This brings memory decisions closer to verifiable outcomes, reducing the difficulty of having to wait for future tasks to learn whether a summary was useful. [Methods and configuration](https://arxiv.org/html/2609.27334v1)

The authors report that, with Qwen3-8B as the executor, the success rate on ALFWorld rose from SkillOS’s 61.2% to 77.4%, while WebShop rose from 16.5% to 32.8%—gains of 16.2 and 16.3 percentage points, respectively. These comparisons apply to specific benchmarks and model configurations. A separate evaluation on τ²-bench tested only the training-free version, so it cannot be treated as another win for the same trained model. [Results table](https://arxiv.org/html/2609.27334v1)

Costs also need to be assessed separately. The paper’s token and step comparisons count only the executor and therefore do not establish that the entire system is cheaper. Deployment assessments must also include summary generation by the curation model, success assessment, and storage of raw trajectories. [Cost accounting](https://arxiv.org/html/2609.27334v1)

Product teams can start by comparing fixed summaries with on-demand summaries on the same task stream, keeping the executor, the number of retrieved trajectories, and tool settings identical. Otherwise, differences in the summary model’s capabilities or data filtering could also affect the results. Acceptance testing should also distinguish between actual task success, the model’s own belief that it succeeded, and an evaluator’s judgment of success, to avoid repeatedly writing flawed experience back into the memory store.

This article argues that the next priority should be to re-evaluate performance when task order changes, memory starts cold, or success assessments are wrong, while comparing end-to-end latency alongside success rates. If new tasks differ substantially from past experience, teams should also check whether the curation model introduces inapplicable action recommendations before deciding whether on-demand memory curation is suitable for real-world agent services.