代理開發工具
Agent Lightning 1.0.1 Packages the Agent-Tuning Workflow as a Skill, Requiring Per-Run Measurement of Cost, Latency, and Accuracy
Microsoft has introduced an Agent Lightning Skill for Claude Code, Codex, and GitHub Copilot that lets a coding agent modify another editable agent against a fixed benchmark. It provides a budget-constrained set of experimentation rules; it does not automatically perform reinforcement learning on the target agent.
Agent Lightning 1.0.1 marks the first official release of the eponymous Skill, turning the task of “improving another AI agent” into a search workflow that a coding agent can execute. After the user provides an editable target agent and a benchmark, the Skill instructs the optimizer to repeatedly measure, modify, and rescore the agent while tracking accuracy, per-run cost, latency, completion rate, and variance, rather than stopping after a single score increase. It can be installed in Claude Code, Codex, or GitHub Copilot via `gh skill install`.
The Skill’s search space includes input grounding, output schemas, prompts, tools, models, reasoning effort, failure isolation, conditional retries, routing, retrieval, and multi-candidate selection. The experimentation budget during development is handled separately from per-run cost after deployment: the optimizer must check the remaining budget, reduce randomness using fixed test cases, a validation split, or repeated runs, and retain a fully measured checkpoint. The rules also prohibit modifying the scorer, leaking held-out labels into the deployment path, or relying on metadata unavailable in production.
This release is related to the Agent Lightning reinforcement learning framework, but the two should not be conflated. The framework intercepts model calls from the actual harness through an API gateway and handles retokenization, dynamic sample counts, and rollout-level advantage. Its paper reports improving Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4% using roughly 6,000 training examples. The new Skill itself, however, primarily consists of operating procedures and search guidance. It does not promise to reproduce those RL results, nor does it include a general-purpose automatic hyperparameter-tuning algorithm. Engineering teams should next assess whether it can still find stable improvements on unseen tasks, across different models, and with different scorers, as well as whether prolonged searches could overfit the benchmark or exhaust API budgets.