How to Implement Tool Search with Embeddings for Dynamic Tool Selection in Claude

You can enable dynamic tool discovery by exposing a single tool_search meta‑tool that performs cosine‑similarity search over pre‑computed embeddings of your tool library, allowing Claude to retrieve relevant capabilities on‑demand without exhausting the context window.

When building Claude applications with large tool catalogs, sending every tool definition with each API request overwhelms the model’s context limit and drives up latency. The semantic tool‑search pattern—implemented in anthropics/claude-cookbooks—solves this by embedding tool descriptions locally and letting Claude query them dynamically, keeping prompts lightweight while supporting hundreds or thousands of available capabilities.

The Problem with Static Tool Catalogs

Claude’s tool‑use system traditionally requires the full JSON schema of every available tool to be included in the system prompt. As the catalog grows beyond a few dozen entries, this approach becomes unsustainable:

• Context exhaustion: Large tool definitions consume precious token budget needed for reasoning.
• Higher latency: Every additional tool increases request payload size.
• Increased costs: More input tokens directly raise API expenses.

The solution is to send Claude only a single meta‑tool (tool_search) that can retrieve specific tools when needed.

The Semantic Tool Search Architecture

The pattern implemented in tool_use/tool_search_with_embeddings.ipynb relies on three core components working together:

1. Static tool library – A Python list of dictionaries containing name, description, and JSON schema for every available tool.
2. Dense vector index – Pre‑computed embeddings of each tool generated by a local sentence‑transformer model (e.g., all‑MiniLM‑L6‑v2), cached in memory and reused across queries.
3. Meta‑tool handler – A function that embeds Claude’s natural‑language query, computes cosine similarity against the tool vectors, and returns lightweight tool_reference objects that Claude can immediately invoke.

Because embeddings are computed locally, search latency remains under a few milliseconds regardless of catalog size.

Step‑by‑Step Implementation

Initialize the Embedding Model

First, load a lightweight sentence‑transformer model to generate 384‑dimensional embeddings. According to the cookbook source, this initialization occurs once at startup and the model persists in memory for all subsequent searches.

import os
import json
import numpy as np
from sentence_transformers import SentenceTransformer
import anthropic

# Initialize Claude client

claude_client = anthropic.Anthropic()

# Load embedding model (runs once)

print("Loading SentenceTransformer model…")
embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

Define the Static Tool Library

Create a comprehensive registry where each tool is a dictionary with name, description, and input_schema keys. The example in the cookbook defines 8 demo tools spanning weather and finance domains.

TOOL_LIBRARY = [
    {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "The unit of temperature"
                },
            },
            "required": ["location"],
        },
    },
    # Additional tools (get_stock_price, calculate_mortgage, etc.)

]

Convert Tools to Searchable Text

To enable semantic search, flatten each tool’s structured definition into a human‑readable string that captures its purpose and parameters. The tool_to_text function in the notebook concatenates the tool name, description, and parameter documentation.

def tool_to_text(tool: dict) -> str:
    """Flatten tool dictionary into searchable text."""
    parts = [
        f"Tool: {tool['name']}",
        f"Description: {tool['description']}"
    ]
    
    if "input_schema" in tool and "properties" in tool["input_schema"]:
        params = tool["input_schema"]["properties"]
        param_desc = [
            f"{name} ({info.get('type','')}) : {info.get('description','')}"
            for name, info in params.items()
        ]
        if param_desc:
            parts.append("Parameters: " + ", ".join(param_desc))
    
    return "\n".join(parts)

# Generate searchable corpus

tool_texts = [tool_to_text(t) for t in TOOL_LIBRARY]

Pre‑compute Tool Embeddings

Encode the entire library into dense vectors and store them in a NumPy array. Since the embeddings are L2‑normalized by the model, you can use dot product for cosine similarity calculation.

# Shape: (num_tools, 384)

tool_embeddings = embedding_model.encode(tool_texts, convert_to_numpy=True)
print(f"Created embeddings with shape: {tool_embeddings.shape}")

Implement the Semantic Search Function

When Claude needs a tool, embed the query and compute similarities against the cached tool vectors. Return the top‑k matches ranked by similarity score.

def search_tools(query: str, top_k: int = 5) -> list[dict]:
    """Return the top_k most semantically similar tools."""
    query_emb = embedding_model.encode(query, convert_to_numpy=True)
    
    # Dot product works because embeddings are L2-normalized

    similarities = np.dot(tool_embeddings, query_emb)
    top_idxs = np.argsort(similarities)[-top_k:][::-1]
    
    return [
        {
            "tool": TOOL_LIBRARY[i],
            "similarity_score": float(similarities[i])
        }
        for i in top_idxs
    ]

# Example usage

results = search_tools("I need to check the weather", top_k=3)

Define the Meta‑Tool Schema

Expose a single tool_search definition to Claude. This is the only tool definition Claude needs in its context to access the entire library dynamically.

TOOL_SEARCH_DEFINITION = {
    "name": "tool_search",
    "description": (
        "Search for available tools that can help with a task. Returns tool "
        "definitions for matching tools. Use this when you need a tool but "
        "don't have it available yet."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "What you need (e.g. 'weather info')"
            },
            "top_k": {
                "type": "number",
                "description": "Number of tools to return (default 5)"
            },
        },
        "required": ["query"],
    },
}

Handle Tool Search Requests

When Claude invokes tool_search, execute the semantic search and return tool_reference objects—lightweight pointers that indicate which tools are now available for invocation.

def handle_tool_search(query: str, top_k: int = 5) -> list[dict]:
    """Process tool_search request and return tool references."""
    results = search_tools(query, top_k=top_k)
    
    # Convert to tool_reference blocks that Claude expects

    tool_refs = [
        {"type": "tool_reference", "tool_name": r["tool"]["name"]}
        for r in results
    ]
    
    # Optional: log results for debugging

    print(f"\n🔍 Tool search: '{query}'")
    for i, r in enumerate(results, 1):
        print(f"   {i}. {r['tool']['name']} (similarity: {r['similarity_score']:.3f})")
    
    return tool_refs

Integrate with Claude's Tool Loop

In your application logic, include only TOOL_SEARCH_DEFINITION (plus any critical always‑available tools) in the initial request. When Claude responds with a tool_use block naming tool_search, invoke handle_tool_search, append the returned references to the conversation history as a user message, and let Claude issue the concrete tool calls in its next turn.

# Initial request includes only the meta-tool

initial_tools = [TOOL_SEARCH_DEFINITION]

# In your tool-use loop:

if tool_use["name"] == "tool_search":
    refs = handle_tool_search(
        tool_use["input"]["query"],
        tool_use["input"].get("top_k", 5)
    )
    # Append refs to messages and continue conversation

Why This Approach Scales

Semantic similarity over keywords: By embedding the full tool description and parameter documentation, the system retrieves get_stock_price even when Claude asks for "latest equity prices," capturing intent beyond exact string matching.

Constant context size: Regardless of whether you have 50 or 5,000 tools, Claude only receives the compact tool_search definition initially. Tool payloads are fetched lazily, keeping prompts within the model’s context limits.

Millisecond‑latency retrieval: Because the embedding model runs locally and the vector index lives in memory, tool discovery adds negligible overhead compared to round‑trips to the Anthropic API.

Summary

• Problem: Large tool catalogs overwhelm Claude’s context window and increase costs when sent statically.
• Solution: Implement a tool_search meta‑tool backed by local sentence‑transformer embeddings.
• Mechanism: Pre‑embed tool descriptions using all‑MiniLM‑L6‑v2, compute cosine similarity with NumPy dot products, and return tool_reference objects.
• Source: Full reference implementation available in tool_use/tool_search_with_embeddings.ipynb within the anthropics/claude-cookbooks repository.

Frequently Asked Questions

How does semantic tool search reduce token usage?

By sending Claude only the tool_search definition (approximately 20–30 lines of JSON) rather than the full schema for every available tool, you reduce the system prompt size by orders of magnitude. The actual tool definitions are transmitted only when Claude explicitly requests them via the search function, minimizing per‑request token consumption.

Which embedding model works best for tool search?

The cookbook implementation uses sentence-transformers/all-MiniLM-L6-v2 because it provides an optimal balance of speed, memory usage (384 dimensions), and semantic quality for short technical descriptions. Since the model runs locally, there are no API costs for embedding generation, and latency remains under 10ms for catalogs with thousands of entries.

How do I return search results to Claude?

After invoking handle_tool_search, return the results as a list of tool_reference objects in a user message. These references inform Claude which tools are now available to call. Claude will then automatically issue the appropriate tool_use requests for the specific capabilities it needs to complete the task.

Can I update the tool library without restarting the application?

Yes. You can append new tools to TOOL_LIBRARY and incrementally generate embeddings for the new entries, or regenerate the entire embedding matrix if needed. Since the embedding model and vector index reside in memory, updates take effect immediately for subsequent queries without requiring a server restart or model reload.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}