ToolGrad Reverses the Tool-Use Data Generation Process, Raising Gemma 3 12B’s BFCL Score to 83.1 with 500 Synthetic Examples

Google Research published a full technical explanation of ToolGrad on September 10, reversing the order of tool-use data creation from “write the question first, then search for a solution” to “verify the solution first, then generate the question.” Traditional ToolBench-style pipelines have agents repeatedly try APIs using DFS; the longer the path, the more likely an incorrect parameter or tool selection at any step will invalidate the entire annotation. ToolGrad instead incrementally assembles a successful workflow from more than 16,000 APIs, then has the model produce a user instruction and answer consistent with that workflow.

Each iteration of the system includes four modules: the API Proposer narrows the set of candidate tools, parallel Executors invoke the tools, the Selector chooses the next step based on execution reports, and the Updater synchronously revises the instruction and response. The natural-language feedback produced by the Selector is called a “textual gradient.” It is not a differentiable gradient; rather, it tells the next iteration which tools and parameters are worth retaining. The official report says the data-generation pass rate reached 99.8%, meaning the primary savings come from reducing failed trajectories rather than lowering the cost of each model inference. [Google Research](https://research.google/blog/toolgrad-efficient-tool-use-dataset-generation-with-textual-gradients/)

The team fine-tuned the 1B, 4B, and 12B versions of Gemma 3 using 500 generated examples. On the single-turn portion of the Berkeley Function Calling Leaderboard, which uses a different set of tools, the three model sizes improved by 8.1, 8.0, and 6.3 points over their respective base models. The 12B version scored 83.1, close to the 83.2 reported for Gemini 2.5 Pro in the paper. Gains were larger on the synthetic non-live subset than on the subset of real-user queries, suggesting that the model may also have learned structural preferences specific to the data generator or benchmark. [Paper](https://arxiv.org/abs/2508.04086)

For agent teams, this method could help create cold-start training datasets for internal enterprise APIs, particularly for read-only tools that can be replayed safely and whose outputs can be verified automatically. However, the current evidence comes mainly from single-turn function calling. Long-running state changes, irreversible operations, permission errors, and cross-tool side effects have yet to be tested thoroughly. The next step is to determine whether the same method can maintain its success rate in real-world sandboxes and whether its cost advantage persists after human review.