AI 代理研究
PILOT Lets Supervisory Agents Reroute Mid-Task and Write Execution Experience Back into Persistent Skills in Real Time
PILOT connects a supervisor and a worker agent through a bidirectional channel, allowing the former to provide guidance or terminate an erroneous path before a long-running task is complete. The authors report improved accuracy and token efficiency across three agent benchmarks, but its “self-improvement” updates only the harness, skills, and memory—not the model parameters.
Researchers from The Hong Kong Polytechnic University and other institutions have introduced PILOT, which replaces conventional post-hoc reflection with closed-loop supervision during task execution. The system consists of an independent supervisor and worker: the worker retains tool outputs, trial-and-error history, and implementation details, while the supervisor maintains a cleaner, goal-oriented view and receives progress notifications, questions, completion results, execution errors, and idle alerts. It can insert steering instructions before the worker’s next turn or directly abort branches that are no longer worth pursuing.
Another mechanism is live self-evolution. If the supervisor identifies reusable procedures, project conventions, or recurring failure patterns in a trajectory, it can immediately write them to a persistent skill repository or memory. Workers launched later in the same task or in future tasks then load the updated harness. “Self-improvement” here therefore refers to the evolution of context and the execution framework: the model weights remain frozen throughout, distinguishing the approach from online fine-tuning or recursive training.
The researchers evaluated GLM-5.1 and Kimi-K2.6, with each model serving as both supervisor and worker, on Terminal-Bench 2.0, SWE-bench Multilingual, and SWE-bench Pro. PILOT ranked first in five of the six model–benchmark combinations. Across the two models, its average pass rate on Terminal-Bench 2.0 was 71.6%, 5.3 percentage points higher than the strongest average single-agent baseline. In the cross-iteration self-improvement setting, the authors report gains of 14.6 and 12.4 percentage points for the two models, respectively, alongside reductions in average output tokens of 42.9% and 47.4%. Manual analysis also showed that supervisory interventions were concentrated on difficult tasks, while successful trajectories for simple tasks did not rely on steering.
However, using two agents introduces additional latency and inference costs. The paper evaluates only two open-weight models and three coding/terminal benchmarks, while supervising a model with the same model does not answer whether pairing an expensive supervisor with a cheaper worker is cost-effective. The publicly available paper also does not currently provide a complete, reproducible implementation. Engineering teams should therefore focus next on cost breakdowns, negative transfer caused by incorrect supervision, and rollback and auditing mechanisms for contaminated persistent skills.