模型合併與多模態代理
AgentPatch Repairs Merged Multimodal Agents Without Training, Raising the Six-Benchmark Average From 54.5 to 56.6
AgentPatch first identifies weaknesses introduced by model merging, then restores only the expert models’ unique residuals and critical FFN neurons. It produces a single static checkpoint with no routing or inference-time agents, but the complete evaluation adapters and model weights have yet to be released.
Combining search, GUI operation, and visual reasoning experts into a general-purpose multimodal agent usually cannot be accomplished by simply averaging their weights. Capabilities involving longer interaction chains are more easily diluted, and if even a few actions critical to success disappear, the entire trajectory can fail. Released on August 7, AgentPatch breaks this problem down into “weak-task signal loss” and “critical behavior forgetting,” proposing a two-stage repair process that requires no gradient-based training.
The system first compares multiple existing merged models, selects a relatively stable recipient checkpoint, and identifies its weakest task. Weak-Task Unique Residual Recovery then finds parameter updates unique to the weak-task expert relative to the other experts and restores only those residuals to the recipient. In the second stage, the Diagnoser–Guardian–Compiler pipeline extracts behavioral segments that must be recovered or protected from the agent trajectories of the recipient and expert models, then locates the relevant FFN neurons based on activation values and output contributions. Guardian excludes neurons that support the recipient’s existing strengths, reducing the risk that repairing one capability will damage another.
The authors evaluated search, desktop and mobile operation, and visual processing across six benchmarks: MMSearch, FactualVQA, AndroidWorld, OSWorld, V*Bench, and HR-Bench 8K. The paper and repository report that AgentPatch increased the TSVM recipient’s overall average score from 54.5 to 56.6 and can also be applied to other merged backbones. Deployment still uses a single static checkpoint and requires no expert routing, ensemble, or additional analysis agents, so it does not add the inference branching costs associated with those approaches.
However, the GitHub repository currently contains only the core repair operators, example configurations, input contracts, and CPU tests. The complete checkpoint construction pipeline, benchmark adapters, trajectories, model weights, and directly reproducible evaluation commands are still marked for future release. Engineering teams should therefore treat AgentPatch as an inspectable algorithmic skeleton rather than a fully reproducible model release. The next question is whether the 2.1-point average gain will hold across different expert combinations and random seeds once the complete assets are published.