AI 研究與代理訓練
TRACE Uses Hidden Interventions to Synthesize Verifiable Rewards, Teaching a 35B Agent to Trace Data Anomalies
TRACE first injects known faults into a simulator, then has an agent use Python and SQL to infer the root cause from observational data containing noise and confounders. After SFT and reinforcement learning, Qwen3.5-35B-A3B outperformed the prompted frontier models evaluated in the paper on a held-out test set, though the results remain limited to a single synthetic advertising environment.
Math and coding are well suited to reinforcement learning with verifiable rewards (RLVR), because answers can be evaluated cheaply using calculators or test suites. Operational anomaly diagnosis, however, usually lacks immediate, unambiguous ground-truth labels. TRACE reverses the data-generation process: it first samples a hidden intervention and injects it into a controlled simulator, then generates the data observable to the agent. The intervention itself serves as the root-cause label, enabling an objective reward, while the agent must still contend with noise, confounders, and evidence distributed across multiple tables.
The research team implemented the method as a digital advertising diagnostic environment covering 12 root causes. The agent can call Python and SQL, and must not only identify the fault type but also determine the affected audience segment when applicable. Measured by FullAttr@1 across 235 held-out cases, prompted Claude Opus 5 scored 0.686. The base Qwen3.5-35B-A3B model scored just 0.159, rising to 0.637 after supervised fine-tuning and reaching 0.757 after reinforcement learning with synthetic rewards. The paper also reports that the trained model made significantly fewer tool calls than a prompted version of the same 35B base model.
The key technical contribution is not the specific advertising task, but the transformation of a diagnostic problem that is difficult to verify after the fact into a training environment that “creates the cause first, then hides it.” The approach could extend to data-pipeline failures, cloud operations, or security incidents, provided that the simulator captures the causal structure of the real world. The current results come from a single synthetic environment with only 12 root-cause categories, and they do not yet demonstrate that the model can handle unknown failures, interactions not modeled by the simulator, or distribution shifts in real enterprise data. Engineering teams should next watch whether the environment generator is open-sourced, whether the reward can be gamed, and how much performance degrades when transferring across systems.