Dynamic Tool Selection: A Portable Pattern for Agents Drowning in Tool Schemas

I stumbled onto this pattern while building agents with Deep Agents, watching a tool registry grow past the point where sending every schema on every turn still made sense. What follows is the pattern itself, stripped down so it drops into any tool-calling agent loop today regardless of framework, backed by a benchmark against a synthetic 61-tool registry.

The Problem: Tool Count Grows, Relevance Per Turn Doesn't

Most agent frameworks assemble the tool list once at construction time and send the whole thing to the model on every call, regardless of what that turn is about. Fine at 5-10 tools. Once you wire up a handful of MCP servers (Slack, GitHub, Linear, a calendar, a CRM, each contributing 3-6 schemas), every turn starts carrying 40-60 tool definitions whether the user asked about a Slack message or not.

That costs you twice over. Every schema (name, description, parameter spec) gets serialized into every request, so tokens spent on tools irrelevant to this turn are tokens not spent on the task. And the accuracy hit is real: more candidates in the list give the model more chances to grab a similarly named or similarly described tool instead of the right one, something the benchmark below reproduces directly.

Sending the whole registry every time is the actual bug here, not the size of the context window.

The Pattern

The pattern boils down to three moving parts: score the registry against the current turn's intent, send only the top-K, and give the model an explicit way to ask for something it can't currently see.

Scoring ranks the full registry against the latest user message, and a plain token-overlap measure (Jaccard similarity between the query's words and each tool's name plus description) turns out to be enough to separate on-topic from off-topic; no embeddings needed. Top-K plus an always-include set caps what actually gets sent to the model; a handful of tools an agent can't function without (file I/O, a task/subagent tool) sit outside the filter entirely. Then there's the piece that makes any of this safe to ship: a discover tool the model can call to search the complete, unfiltered registry when nothing in its filtered view fits. A match gets pinned into the always-visible set for the rest of the conversation, so the worst case is one extra tool call, never a tool the model silently doesn't know exists.

The implementation below has no framework dependency, just the algorithm, and represents a tool as a plain {name, description} pair, a strict subset of what OpenAI function-calling, Anthropic tool-use, LangChain's BaseTool, and MCP tool listings all expose, so it drops into any of them.

"""A portable, zero-dependency tool selection pattern for tool-calling agents."""

from __future__ import annotations

import re
from collections.abc import Iterable
from dataclasses import dataclass

_TOKEN_RE = re.compile(r"[a-z0-9]+")


@dataclass(frozen=True)
class Tool:
    """Minimal tool description: what any framework's tool object reduces to."""

    name: str
    description: str


def _tokenize(text: str) -> set[str]:
    return {m.group(0) for m in _TOKEN_RE.finditer(text.lower())}


def lexical_score(tool: Tool, query_tokens: set[str]) -> float:
    """Jaccard overlap between a tool's name+description tokens and the query tokens."""
    if not query_tokens:
        return 0.0
    tool_tokens = _tokenize(f"{tool.name} {tool.description}")
    if not tool_tokens:
        return 0.0
    return len(tool_tokens & query_tokens) / len(tool_tokens | query_tokens)


def select_tools(
    tools: Iterable[Tool],
    query: str,
    *,
    top_k: int,
    always_include: frozenset[str] = frozenset(),
    pinned: frozenset[str] = frozenset(),
    scorer=lexical_score,
) -> list[str]:
    """Return the names of the top-K tools most relevant to `query`, plus keepers."""
    tools = list(tools)
    keep_names = always_include | pinned
    if len(tools) <= top_k:
        return [t.name for t in tools]

    query_tokens = _tokenize(query)
    candidates = [t for t in tools if t.name not in keep_names]
    ranked = sorted(candidates, key=lambda t: scorer(t, query_tokens), reverse=True)
    selected = {t.name for t in ranked[:top_k]}

    return [t.name for t in tools if t.name in keep_names or t.name in selected]


def discover(tools: Iterable[Tool], query: str, *, scorer=lexical_score) -> Tool | None:
    """Search the full registry for the single best match to `query`.

    Returns `None` if nothing scores above zero -- callers should surface that as
    "no match found" rather than silently picking an arbitrary tool.
    """
    query_tokens = _tokenize(query)
    tools = list(tools)
    if not tools:
        return None
    best = max(tools, key=lambda t: scorer(t, query_tokens))
    return best if scorer(best, query_tokens) > 0 else None

select_tools costs nothing below top_k; it's a no-op until the registry is actually large enough to matter. scorer is a keyword hook, so swapping lexical_score for a cosine-similarity function over an embedding model changes nothing else in the function.

What the Tests Actually Check

"""Tests for the portable tool_selector module (excerpt)."""

from tool_selector import Tool, discover, select_tools


def _tools(*pairs: tuple[str, str]) -> list[Tool]:
    return [Tool(name=n, description=d) for n, d in pairs]


def test_pinned_tool_survives_an_unrelated_turn() -> None:
    """Simulates turn 2 of a conversation where turn 1's discover() pinned a tool."""
    tools = _tools(
        ("weather_lookup", "get the current weather forecast for a city"),
        ("calculator", "evaluate a basic arithmetic expression"),
    )
    result = select_tools(
        tools, "what is the weather forecast today", top_k=1,
        pinned=frozenset({"calculator"}),
    )
    assert set(result) == {"weather_lookup", "calculator"}


def test_discover_finds_the_right_tool_by_description() -> None:
    tools = _tools(
        ("weather_lookup", "get the current weather forecast for a city"),
        ("calculator", "evaluate a basic arithmetic expression"),
    )
    match = discover(tools, "evaluate an arithmetic expression")
    assert match is not None
    assert match.name == "calculator"

The discover test surfaced a real limitation while I was writing it. An early draft queried discover(tools, "I need to crunch some numbers") against a calculator tool described as "evaluate a basic arithmetic expression," and it failed outright: the two strings share zero tokens. Lexical scoring has no concept of synonymy, so whatever query gets handed to discover has to share vocabulary with the target tool's description; in practice that means the model has to formulate a reasonable search term rather than forward the user's literal wording. It's a real constraint of the zero-dependency approach, and the main argument for the scorer= hook: swap in an embeddings model once tool vocabulary and user vocabulary diverge enough to bite you.

Measuring It: A recall@K Benchmark

Rather than mock an LLM's tool-picking behavior (which amounts to testing my own mock), I measured the one thing that doesn't need a model in the loop at all: does the correct tool survive the filtering step? If the right tool gets cut before the model ever sees the list, no amount of model capability brings it back.

It's a 61-tool registry, modeled on what four or five real MCP servers actually expose (Slack, GitHub, Linear, Jira, Gmail, Calendar, Drive, Notion, a CRM, web search, weather, finance, plus a small always-include filesystem core), roughly the tool count teams report after wiring up a handful of MCP servers rather than an inflated worst case. 30 labeled queries span about 18 domains, split between direct phrasing ("send a direct message to alice on slack") and indirect phrasing ("let the team know in the channel that the deploy finished"), the same split deepagents' own tool-selection evals use.

Recall@K Across the Six top_k Settings Tested

Unfiltered baseline: every turn sends all 61 tools, 5,523 chars, every time.

That table breaks down into two separate questions worth pulling apart: how good is the trade at a reasonable K, and how much does pushing K higher actually buy you?

At top_k=10, you get 67% recall for an 84% payload reduction. For a lexical scorer with zero setup cost, that's a genuinely good trade, and the 33% of misses aren't silent failures; they're what the discover escape hatch exists for: one extra tool call, the tool gets found, and it's pinned for the rest of the thread.

Recall also climbs slowly as top_k grows: going from 10 to 30 tools sent buys only 13 more points. Past a certain point you're paying most of the unfiltered cost for a shrinking accuracy gain, and if you need recall above roughly 80% without raising top_k that far, that's the signal to swap in the scorer= embeddings hook instead of continuing to raise K.

The k=10 misses look like this:  

MISS  query='file a bug report on the backend repo'
      expected='github_create_issue', got=[..., 'github_create_pr', ...]
MISS  query='mark this jira ticket as in progress'
      expected='jira_transition_issue', got=[..., 'jira_create_issue', ...]
MISS  query="what's 340 divided by 12"
      expected='calculator', got=['read_file', 'write_file', ..., 'slack_search_messages']

The misses cluster around two failure modes: tools in the same domain sharing most of their vocabulary (github_create_issue and github_create_pr both score high on "github", "create", "repo"), and short, generic queries that share almost no tokens with the target description ("what's 340 divided by 12" versus "evaluate a basic arithmetic expression"). Both are what the escape hatch is designed to catch, and both are cases where embeddings-based scoring would do meaningfully better.

The full registry, query set, and benchmark script run about 150 lines, small enough to paste into any project and adapt to your own tool list. The numbers are reproducible without an API key or a specific model.

Before vs. After, on an Actual Agent Run

The recall@K numbers above measure the scoring step in isolation. To see the effect on a real conversation, the same 3-turn scenario was run twice through an actual create_agent graph with a checkpointer (once unfiltered, once with ToolSelectionMiddleware(top_k=1, always_include=frozenset())) against 4 domain tools: weather_lookup, stock_price, translate_text, calculator.

Turn 2 is the interesting one: the question, "I need to crunch some numbers but don't see a tool for that, can you check?", was deliberately worded to score zero against calculator's own description. The model genuinely can't see a calculator tool in its filtered list and has to fall back to the discover escape hatch:

TURN 2: "I need to crunch some numbers but don't see a tool for that, can you check?"
model call (before discover_tools ran) -> model was sent 2 tools: ['discover_tools', 'weather_lookup']
discover_tools returned: Found tool `calculator`: Evaluate a basic arithmetic expression.
It is now available for the rest of this conversation.

TURN 3: "translate hello to French" (same thread -- calculator pin should persist)
model call -> model was sent 3 tools: ['calculator', 'discover_tools', 'translate_text']
state['tool_selection_pinned'] on this thread: ['calculator']

calculator shows up in turn 3's tool list even though that turn is about translation: that's the pin from turn 2 persisting through the checkpointer as designed. Without the middleware, every turn sends all 4 schemas regardless of relevance. With it, turns 1 and 2 send 2 tools each and turn 3 sends 3, and the tool the model couldn't initially see gets recovered through exactly one extra call.

Wiring It Into an Existing Framework

The algorithm above has zero framework knowledge, on purpose. Here's how it plugs into LangChain / deepagents' middleware system, which intercepts the tool list before every model call via wrap_model_call:

from tool_selector import Tool, select_tools

class ToolSelectionMiddleware:
    """Sketch: adapt to your framework's actual middleware hook signature."""

    def __init__(self, *, top_k: int = 15, always_include: frozenset[str] = frozenset()):
        self.top_k = top_k
        self.always_include = always_include

    def wrap_model_call(self, request, handler):
        latest_query = _latest_human_message_text(request.messages)
        candidate_tools = [Tool(t.name, t.description) for t in request.tools]
        keep = set(select_tools(
            candidate_tools, latest_query,
            top_k=self.top_k, always_include=self.always_include,
        ))
        filtered = [t for t in request.tools if t.name in keep]
        return handler(request.override(tools=filtered))

This is a sketch, deliberately not copy-pasteable middleware. A fuller version wires the discover_tools escape hatch as an injected tool with per-thread pin state, so pins don't leak across concurrent sessions. Treat the API shape here as illustrative rather than stable; the algorithm underneath is the part worth keeping regardless of framework.

One detail worth flagging for anyone building an injected-context tool in LangChain/LangGraph: if your escape-hatch tool takes a runtime/context parameter the framework injects automatically (ToolRuntime, for instance), the module defining it must not use from __future__ import annotations. Postponed annotations turn the type hint into a string at definition time, so injection detection that inspects the live signature won't recognize it. The tool then breaks silently when invoked through the framework's actual call path, even though a direct unit test would never catch it.

The Native Alternative: Claude's Tool Search Tool

If you're calling the Claude API directly rather than going through an agent framework, Anthropic now ships a server-side version of this same idea: the Tool Search Tool (tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119). You declare it alongside your other tools, mark the tools you don't want sent by default with defer_loading: true, and Claude searches the deferred set and pulls in only what's relevant, as a tool_search_tool_result block.

{
  "tools": [
    { "type": "tool_search_tool_bm25_20251119", "name": "tool_search_tool_bm25" },
    { "name": "github_create_issue", "description": "...", "defer_loading": true },
    { "name": "slack_send_dm", "description": "...", "defer_loading": true }
  ]
}

This isn't a mere hosted copy of the DIY pattern. The model does the searching itself, so there's no lexical-overlap or embedding logic to maintain: Claude decides what's relevant and searches for it. Discovered schemas also get appended to the request rather than swapped in. Changing which tools are visible mid-conversation would normally invalidate the prompt cache, since the tool list renders at the start of the prefix, but tool search sidesteps that: the deferred tools stay out of the initial prefix, and appending to the end doesn't rewrite what came before.

What you lose relative to the DIY version is curatorial control: there's no equivalent of always_include or an explicit per-thread pin you can inspect and log, since the whole mechanism lives server-side. If you need that visibility, or you're not on a framework/model that supports the Tool Search Tool, the portable version above is the fallback. If you're calling Claude directly and don't need fine-grained control over what's exempted from filtering, reach for the native tool first: it's less code to maintain, and it solves the caching problem for free.

Takeaways

Tool registries grow faster than most agent code accounts for. Two or three MCP servers is enough to cross the point where sending every schema on every turn starts costing accuracy, not just tokens. A zero-dependency lexical scorer recovers most of that benefit (67% recall at 84% payload reduction at top_k=10 on a 61-tool registry), and it's the escape hatch, not the scorer's raw accuracy, that makes shipping something this lossy safe. Test that escape hatch through the real framework call path rather than by calling the underlying function directly: injected-parameter bugs and cache-invalidation bugs both hide from unit tests that bypass the framework's actual entry point. And if you're calling the Claude API directly, check whether the native Tool Search Tool already covers your case before building any of this yourself.

The full tool_selector.py, its test suite, and the benchmark script are small enough to fit in a gist; reach out if you want them as a standalone repo rather than reconstructing from the code blocks above.