Back Home

推論系統

Gemini can load tool specifications on demand; whether it pays off depends on search costs

Large tool libraries can let the model read tool descriptions first and load parameters only when needed. But server-side mode still requires resending the full declaration, and each search adds billable input.

Asoundd · CC BY-SA 4.0 · Image source
zh-Hant

When agents connect to dozens or hundreds of tools, developers can now have Gemini read tool names and descriptions first, then load the full parameter specifications only when needed. On November 19, Google introduced the Tool Retrieval preview, bringing tool search, specification loading, and function-call generation into a single Interactions API request. For large tool libraries, the practical change is that unused parameter specifications no longer need to occupy the model’s context from the start. Official announcement

Model reading is deferred; declarations still have to be sent

Parameter specifications describe the fields a tool accepts, their data types, and which ones are required. For example, a meeting-scheduling tool might require attendees, a date, and a time, allowing the model to turn a natural-language request into parameters a program can execute. Function calling guide

To enable server-side retrieval, developers add tool_search and set defer_loading: true for less frequently used tools. The model first sees tool names and short descriptions, then loads the full specifications after selecting a tool. The application still executes ordinary custom functions and returns their results; completing search and generating a call in the same request does not mean the external operation has been completed. Remote MCP servers also support deferring an entire set of tools. MCP is a protocol for connecting models to external tools. Workflow and examples

So the question of whether fewer tool specifications can be sent needs some nuance: server-side mode reduces what the model initially reads, but the application must still include the full tool declaration in every request, including follow-up requests that return execution results. Tool Retrieval documentation

No separate retrieval fee doesn’t mean search is free

The official documentation says unloaded parameter specifications are not billed yet. However, each server-side search triggers an additional model call that includes the full context and the specifications loaded so far. These inputs are recorded in total_tool_use_tokens and billed at the input rate. Loaded tools also remain in the conversation context and cannot be unloaded. Billing and limitations

Google therefore identifies large tool libraries where only a few tools are used per request as a scenario more likely to save tokens. If there are only a few tools, or a long conversation searches on every turn, preloading all specifications may instead be cheaper. This is the cost assessment in Google’s documentation, not a result measured by this article. When it applies

The announcement claims this design can reduce latency and input costs, but provides no quantified comparison. Developers should not assume that enabling the feature will lower bills or waiting times by a fixed percentage. Official announcement

Build your own retrieval: one extra round trip for dynamic tool selection

If you already have a tool index or a workflow that filters tools by user permissions, you can also use client-side retrieval: the application searches and then returns the matching tool specifications, without declaring every tool up front. The trade-off is one additional API round trip per search. Client-side mode

Tool Retrieval is currently in preview, requires v1beta, may change before general availability, and is not supported by generateContent. Feature status The Interactions API itself is generally available, and Google recommends it for new projects; the existing generateContent API remains supported. API overview

For teams preparing to adopt it, the key factors are the size of the tool specifications, how many tools are actually used per turn, and how much context has accumulated by the time a search runs. Those factors determine whether the specification input saved outweighs the added search cost.

Sources

  1. Introducing Tool Retrieval in the Gemini API
  2. Tool retrieval
  3. Function calling with the Gemini API
  4. Interactions API