代理訓練
SkillGate Separates Skill Selection from Execution Feedback, Raising 9B Agent Success Rate from 40.8% to 53.2%
SkillGate finds that when long-horizon agents are trained with outcome rewards, the small number of tokens used to select a skill receive almost no effective credit. The new method separates two gradient channels within the same GRPO update and outperforms the outcome-reward baseline across five agent benchmarks.
When an agent selects a procedural document from thousands of `SKILL.md` files, it may look like a single tool call, but that choice can determine whether the entire task succeeds. After auditing 12,800 training trajectories, the SkillGate team found that tokens specifying skill names accounted for a median of just 0.14% of each trajectory’s loss weight. About two-fifths of selections also received negative advantage because subsequent execution failed. This “selector credit starvation” worsens as trajectories grow longer: even when the correct skill is read, outcome feedback broadcast across the full sequence can still update the selection action in the wrong direction.
Rather than building a separate routing model, SkillGate uses token masks to separate credit within a single GRPO update. Task outcomes update only execution tokens, while a separate local advantage updates only skill-identification tokens. A positive signal is issued only when the full trajectory reads exactly one skill and selects the annotated oracle skill. The two channels are assigned equal loss mass, with a selector coefficient of 0.20 in the experiments. As a result, deployment still uses a standard agent, with no additional inference model or decision step.
In a setup spanning SkillsBench, SETA, SWE, Terminal-Bench 2.0, and the held-out Claw-Eval test set—with 16 candidate skills mixed into each task—Qwen3.5-9B achieved a 40.8% success rate after SFT, 47.0% with conventional outcome-reward RL, and 53.2% with SkillGate. The rate of reading the oracle skill rose from 54.3% to 83.9%, while exposure to misleading skills fell from 69.6% to 21.8%. The code, assets, evaluation pipeline, and final weights have been released publicly. However, the method requires oracle skill labels during training, and the model depends on an OpenClaw-style prompt format. Its effectiveness across models, different skill libraries, and environments without clean labels still requires independent replication.