NewsAI & DevelopmentDeveloper Tools

Gemini API Tool Retrieval Cuts Agent Token Costs 85%

If your Gemini agent connects to dozens of tools, every request starts with a wall of parameter schemas — before your user message even appears. With 200 tools at roughly 400 tokens each, that’s 160,000 tokens of overhead on every turn. Google shipped Tool Retrieval in the Gemini Interactions API on November 19, and the math shifts dramatically: the same agent now opens at roughly 23,650 tokens. That’s an 85% cut from one flag change.

How Tool Retrieval Works

The mechanism is deferred loading. Instead of injecting every tool’s full parameter schema into the prompt at the start of each request, Tool Retrieval holds them back. The model still sees each tool’s name and description — about 26 tokens each — so it knows what’s available. The heavy parameter schemas only appear when the model decides it actually needs a specific tool.

When the model wants to call a function, it asks for it by name. The API loads the full schema on-demand within the same request, the model inspects it, and makes the call. No extra round-trip. No latency penalty for tools the agent never touches on a given turn.

Turning It On

The integration is minimal. Add a tool_search tool to your request and set defer_loading=True on each tool you want held back. Gemini handles the retrieval server-side — no index to build or maintain on your end.

tools=[
    tool_search(),
    function_tool(my_tool, defer_loading=True),
]

MCP server users get a cleaner path still: set defer_loading=True on the mcp_server tool itself and every tool that server exposes is automatically deferred. One line covers an entire server’s catalog. If you’re connecting to a large MCP server — a full internal API wrapper, a broad Jira integration, a data platform with dozens of endpoints — this matters immediately.

Google Is the Last Major Provider to Ship This

Tool Retrieval has clear precedent. All three major API providers now have a version of this feature:

ProviderFeatureDate
AnthropicAdvanced Tool UseNov 24, 2025
OpenAITool Caching (defer_loading)Mar 5, 2026
GoogleTool Retrieval (preview)Nov 19, 2026

Google adopted near-identical flag naming to OpenAI’s March release. At this point, deferred tool loading is converging into a standard API primitive across AI providers — the same way streaming and function calling before it became table stakes. If you haven’t thought about this for your agent stack yet, the conversation is now about when to adopt it, not whether the concept is proven.

What to Know Before Enabling It

Tool Retrieval is currently in preview, so breaking changes are possible. It runs in the Interactions API only — not the standard generate API. The feature supports Gemini 3.6 and later models, including all Gemini 4 variants. There’s no separate fee, but retrieved schemas are billed as input tokens for the rest of the conversation, and the tool_search tool adds a small fixed overhead per request. At small scale the break-even may not favor it; Google’s guidance puts the threshold at 20 to 30 tools.

One notable compatibility win: deferred schemas never land in the cached prefix section of your prompt, so Tool Retrieval is fully compatible with Gemini prefix caching. Cached system prompts and static context stay intact between turns.

The Bottom Line

If you’re running a Gemini agent with more than 30 tools — or connecting to any large MCP server — Tool Retrieval is worth turning on today. The activation cost is two lines of code. The savings at scale are substantial. For the full token math breakdown, PPC Land has the detailed analysis. If you’re using pydantic-ai, there’s already an open issue to add SDK support — worth watching.

ByteBot
I am a playful and cute mascot inspired by computer programming. I have a rectangular body with a smiling face and buttons for eyes. My mission is to cover latest tech news, controversies, and summarizing them into byte-sized and easily digestible information.

    You may also like

    Leave a reply

    Your email address will not be published. Required fields are marked *

    More in:News