AI 代理訓練
EnvACE Lets a Single Model Act as Both Agent and Environment, Raising Its Average Score Across Three Tool-Use Benchmarks to 32.91%
During training, EnvACE simulates tool responses itself, then jointly updates its action and environment roles based on task success or failure. In Qwen3-8B experiments, it outperformed the base model, but led the strongest environment-synthesis baseline by just 0.37 percentage points.
Reinforcement learning for tool-using agents typically requires executable APIs, databases, and state transitions. Building and validating these training environments is often more expensive than generating prompts. EnvACE instead has the same language model alternate between two roles: the Actor first generates a tool call, and the Rehearsal role then predicts the environment response that the call should trigger. The predicted result is written back into the trajectory for use by the next action. The two outputs share model parameters, but GRPO advantages are calculated using separate reward baselines for each role, preventing actions and simulated responses from being compared directly under the same distribution.
In the main experiment, Qwen3-8B was trained for 470 steps with four rollouts per prompt and trajectories capped at 30 turns, using 16 NVIDIA H20 GPUs. Task rewards came partly from verifiable graders and partly from a Qwen3-30B-A3B judge. EnvACE achieved an average aggregate score of 32.91% across BFCL-v4, τ²-Bench, and VitaBench, compared with 28.48% for the untrained Qwen3-8B, 31.92% for EnvScaler-8B, and 32.54% for AWM-14B. On FinMCP-Bench, its tool F1 score was 46.78%, 3.10 points higher than EnvScaler-8B, although it did not achieve the highest tool recall.
The internalized environment model was also used for test-time rehearsal: it first privately simulates candidate trajectories, then passes a summary to a single real execution. Two parallel rehearsals raised the aggregate score on the two specified benchmarks from 36.7% to 40.9%. Increasing the number to three instead reduced performance, which the authors attribute to context expansion.
The key engineering question is whether “imagining tool outputs” can remain calibrated for unfamiliar or rapidly changing APIs. The model could simultaneously generate an incorrect action and a fabricated response that accommodates it, while the current results still depend on the authors’ configurations, a limited family of models, and some LLM judges. The code has been released, but the repository was only recently created and broad independent reproduction is not yet available.