# How to Implement Tool Search with Embeddings for Dynamic Tool Selection in Claude

> Implement tool search with embeddings for dynamic tool selection in Claude. Enable on demand retrieval of relevant capabilities, saving context window space with this advanced technique.

- Repository: [Anthropic/claude-cookbooks](https://github.com/anthropics/claude-cookbooks)
- Tags: how-to-guide
- Published: 2026-04-14

---

**You can enable dynamic tool discovery by exposing a single `tool_search` meta‑tool that performs cosine‑similarity search over pre‑computed embeddings of your tool library, allowing Claude to retrieve relevant capabilities on‑demand without exhausting the context window.**

When building Claude applications with large tool catalogs, sending every tool definition with each API request overwhelms the model’s context limit and drives up latency. The **semantic tool‑search pattern**—implemented in `anthropics/claude-cookbooks`—solves this by embedding tool descriptions locally and letting Claude query them dynamically, keeping prompts lightweight while supporting hundreds or thousands of available capabilities.

## The Problem with Static Tool Catalogs

Claude’s tool‑use system traditionally requires the full JSON schema of every available tool to be included in the system prompt. As the catalog grows beyond a few dozen entries, this approach becomes unsustainable:

- **Context exhaustion**: Large tool definitions consume precious token budget needed for reasoning.
- **Higher latency**: Every additional tool increases request payload size.
- **Increased costs**: More input tokens directly raise API expenses.

The solution is to send Claude only a **single meta‑tool** (`tool_search`) that can retrieve specific tools when needed.

## The Semantic Tool Search Architecture

The pattern implemented in `tool_use/tool_search_with_embeddings.ipynb` relies on three core components working together:

1. **Static tool library** – A Python list of dictionaries containing name, description, and JSON schema for every available tool.
2. **Dense vector index** – Pre‑computed embeddings of each tool generated by a local sentence‑transformer model (e.g., `all‑MiniLM‑L6‑v2`), cached in memory and reused across queries.
3. **Meta‑tool handler** – A function that embeds Claude’s natural‑language query, computes cosine similarity against the tool vectors, and returns lightweight `tool_reference` objects that Claude can immediately invoke.

Because embeddings are computed locally, search latency remains under a few milliseconds regardless of catalog size.

## Step‑by‑Step Implementation

### Initialize the Embedding Model

First, load a lightweight sentence‑transformer model to generate 384‑dimensional embeddings. According to the cookbook source, this initialization occurs once at startup and the model persists in memory for all subsequent searches.

```python
import os
import json
import numpy as np
from sentence_transformers import SentenceTransformer
import anthropic

# Initialize Claude client

claude_client = anthropic.Anthropic()

# Load embedding model (runs once)

print("Loading SentenceTransformer model…")
embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

```

### Define the Static Tool Library

Create a comprehensive registry where each tool is a dictionary with `name`, `description`, and `input_schema` keys. The example in the cookbook defines 8 demo tools spanning weather and finance domains.

```python
TOOL_LIBRARY = [
    {
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "The unit of temperature"
                },
            },
            "required": ["location"],
        },
    },
    # Additional tools (get_stock_price, calculate_mortgage, etc.)

]

```

### Convert Tools to Searchable Text

To enable semantic search, flatten each tool’s structured definition into a human‑readable string that captures its purpose and parameters. The `tool_to_text` function in the notebook concatenates the tool name, description, and parameter documentation.

```python
def tool_to_text(tool: dict) -> str:
    """Flatten tool dictionary into searchable text."""
    parts = [
        f"Tool: {tool['name']}",
        f"Description: {tool['description']}"
    ]
    
    if "input_schema" in tool and "properties" in tool["input_schema"]:
        params = tool["input_schema"]["properties"]
        param_desc = [
            f"{name} ({info.get('type','')}) : {info.get('description','')}"
            for name, info in params.items()
        ]
        if param_desc:
            parts.append("Parameters: " + ", ".join(param_desc))
    
    return "\n".join(parts)

# Generate searchable corpus

tool_texts = [tool_to_text(t) for t in TOOL_LIBRARY]

```

### Pre‑compute Tool Embeddings

Encode the entire library into dense vectors and store them in a NumPy array. Since the embeddings are L2‑normalized by the model, you can use dot product for cosine similarity calculation.

```python

# Shape: (num_tools, 384)

tool_embeddings = embedding_model.encode(tool_texts, convert_to_numpy=True)
print(f"Created embeddings with shape: {tool_embeddings.shape}")

```

### Implement the Semantic Search Function

When Claude needs a tool, embed the query and compute similarities against the cached tool vectors. Return the top‑k matches ranked by similarity score.

```python
def search_tools(query: str, top_k: int = 5) -> list[dict]:
    """Return the top_k most semantically similar tools."""
    query_emb = embedding_model.encode(query, convert_to_numpy=True)
    
    # Dot product works because embeddings are L2-normalized

    similarities = np.dot(tool_embeddings, query_emb)
    top_idxs = np.argsort(similarities)[-top_k:][::-1]
    
    return [
        {
            "tool": TOOL_LIBRARY[i],
            "similarity_score": float(similarities[i])
        }
        for i in top_idxs
    ]

# Example usage

results = search_tools("I need to check the weather", top_k=3)

```

### Define the Meta‑Tool Schema

Expose a single `tool_search` definition to Claude. This is the only tool definition Claude needs in its context to access the entire library dynamically.

```python
TOOL_SEARCH_DEFINITION = {
    "name": "tool_search",
    "description": (
        "Search for available tools that can help with a task. Returns tool "
        "definitions for matching tools. Use this when you need a tool but "
        "don't have it available yet."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "What you need (e.g. 'weather info')"
            },
            "top_k": {
                "type": "number",
                "description": "Number of tools to return (default 5)"
            },
        },
        "required": ["query"],
    },
}

```

### Handle Tool Search Requests

When Claude invokes `tool_search`, execute the semantic search and return `tool_reference` objects—lightweight pointers that indicate which tools are now available for invocation.

```python
def handle_tool_search(query: str, top_k: int = 5) -> list[dict]:
    """Process tool_search request and return tool references."""
    results = search_tools(query, top_k=top_k)
    
    # Convert to tool_reference blocks that Claude expects

    tool_refs = [
        {"type": "tool_reference", "tool_name": r["tool"]["name"]}
        for r in results
    ]
    
    # Optional: log results for debugging

    print(f"\n🔍 Tool search: '{query}'")
    for i, r in enumerate(results, 1):
        print(f"   {i}. {r['tool']['name']} (similarity: {r['similarity_score']:.3f})")
    
    return tool_refs

```

### Integrate with Claude's Tool Loop

In your application logic, include only `TOOL_SEARCH_DEFINITION` (plus any critical always‑available tools) in the initial request. When Claude responds with a `tool_use` block naming `tool_search`, invoke `handle_tool_search`, append the returned references to the conversation history as a user message, and let Claude issue the concrete tool calls in its next turn.

```python

# Initial request includes only the meta-tool

initial_tools = [TOOL_SEARCH_DEFINITION]

# In your tool-use loop:

if tool_use["name"] == "tool_search":
    refs = handle_tool_search(
        tool_use["input"]["query"],
        tool_use["input"].get("top_k", 5)
    )
    # Append refs to messages and continue conversation

```

## Why This Approach Scales

**Semantic similarity over keywords**: By embedding the full tool description and parameter documentation, the system retrieves `get_stock_price` even when Claude asks for "latest equity prices," capturing intent beyond exact string matching.

**Constant context size**: Regardless of whether you have 50 or 5,000 tools, Claude only receives the compact `tool_search` definition initially. Tool payloads are fetched lazily, keeping prompts within the model’s context limits.

**Millisecond‑latency retrieval**: Because the embedding model runs locally and the vector index lives in memory, tool discovery adds negligible overhead compared to round‑trips to the Anthropic API.

## Summary

- **Problem**: Large tool catalogs overwhelm Claude’s context window and increase costs when sent statically.
- **Solution**: Implement a `tool_search` meta‑tool backed by local sentence‑transformer embeddings.
- **Mechanism**: Pre‑embed tool descriptions using `all‑MiniLM‑L6‑v2`, compute cosine similarity with NumPy dot products, and return `tool_reference` objects.
- **Source**: Full reference implementation available in `tool_use/tool_search_with_embeddings.ipynb` within the `anthropics/claude-cookbooks` repository.

## Frequently Asked Questions

### How does semantic tool search reduce token usage?

By sending Claude only the `tool_search` definition (approximately 20–30 lines of JSON) rather than the full schema for every available tool, you reduce the system prompt size by orders of magnitude. The actual tool definitions are transmitted only when Claude explicitly requests them via the search function, minimizing per‑request token consumption.

### Which embedding model works best for tool search?

The cookbook implementation uses `sentence-transformers/all-MiniLM-L6-v2` because it provides an optimal balance of speed, memory usage (384 dimensions), and semantic quality for short technical descriptions. Since the model runs locally, there are no API costs for embedding generation, and latency remains under 10ms for catalogs with thousands of entries.

### How do I return search results to Claude?

After invoking `handle_tool_search`, return the results as a list of `tool_reference` objects in a user message. These references inform Claude which tools are now available to call. Claude will then automatically issue the appropriate `tool_use` requests for the specific capabilities it needs to complete the task.

### Can I update the tool library without restarting the application?

Yes. You can append new tools to `TOOL_LIBRARY` and incrementally generate embeddings for the new entries, or regenerate the entire embedding matrix if needed. Since the embedding model and vector index reside in memory, updates take effect immediately for subsequent queries without requiring a server restart or model reload.