# How to Implement Scalable Tool Discovery for AI Agents Managing Hundreds of Tools

> Implement scalable tool discovery for AI agents. Learn to manage hundreds of tools efficiently using JSON libraries, keyword search, and sandboxed execution. Optimize agent performance today.

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: how-to-guide
- Published: 2026-07-30

---

**Implement scalable tool discovery for AI agents by combining a lightweight JSON-based ToolLibrary with keyword search, lazy loading, and sandboxed execution to safely manage hundreds of tools without pre-loading everything into memory.**

The `bojieli/ai-agent-book` repository demonstrates a production-ready approach to tool discovery for AI agents that scales effortlessly to hundreds of registered capabilities. Instead of loading all tools into the context window or memory, the system uses a self-evolving file-based architecture that lets agents discover, create, validate, and reuse tools dynamically. This approach eliminates context limitations while maintaining fast lookup and safe execution.

## Core Architecture of the Tool Discovery System

### The ToolLibrary Metadata Store

At the heart of the system is the `ToolLibrary` class defined in [[`tool_manager.py`](https://github.com/bojieli/ai-agent-book/blob/main/tool_manager.py)](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/self-evolving-tools/tool_manager.py#L43-L53) (lines 43-53). Each tool is stored as a lightweight JSON file containing its metadata and executable Python code. This file-system approach avoids database dependencies and keeps the architecture portable across environments.

### Tool Creation and Schema Normalization

The `create_tool` method (lines 50-99) wraps user-provided functions as OpenAI-compatible tools. It enforces quality through a two-stage validation process: first, a syntax compile check, then an optional sandbox execution via `_run_record`. The method also calls `normalize_schema` to ensure every tool conforms to OpenAI's expected JSON schema, preventing costly runtime 400 errors when scaling to many tools.

### Keyword-Based Tool Search

For discovery, the `search_tools` method (lines 105-124) performs simple keyword tokenization over tool names and descriptions. While this implements an O(N) linear scan, the constant factor remains extremely low because JSON files are tiny and disk I/O stays cheap even when managing hundreds of tools.

### Sandboxed Execution

The `execute_tool` method (lines 142-152) runs stored `run(**kwargs)` functions in isolated subprocesses using `_run_python_source`. This sandbox approach injects a custom `PYTHONPATH` and temporary execution environment, ensuring that misbehaving tools cannot crash the agent process—a critical safety feature when executing hundreds of tools concurrently.

## Scaling Mechanisms for Hundreds of Tools

The repository implements several specific strategies to handle scale efficiently:

- **Metadata Indexing**: Uses simple keyword tokenization (`query → terms`) with linear scan over JSON files. This avoids the complexity of vector databases while maintaining acceptable performance for hundreds of small metadata files.

- **Lazy Loading**: Tools are read from disk only when needed via `get_tool`, not pre-loaded into memory. This keeps the memory footprint constant regardless of how many tools are registered in the library.

- **Pre-validation Gate**: The syntax compile and sandbox run before persisting ensures bad tools never pollute the library. This saves storage space and prevents cascading failures when many tools are available.

- **Unlocking Mechanism**: After `search_tools` finds a tool, its name is added to the `_unlocked` set, allowing immediate subsequent calls. This avoids unnecessary re-searches during multi-step agent workflows.

## The Self-Evolving Agent Loop

The `SelfEvolvingAgent` class in [[`agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/agent.py)](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/self-evolving-tools/agent.py#L92-L101) (lines 92-101) orchestrates the complete discovery workflow:

1. **Search**: The agent first calls `search_tools` to check for existing capabilities.
2. **Generate**: When no matching tool exists, it uses `web_search`, `code_interpreter`, and LLM generation to create new code.
3. **Validate**: The new tool undergoes validation via `create_tool` before registration.
4. **Execute**: Once registered, subsequent tasks reuse the tool directly via `execute_tool` without recreation.

The demonstration in [[`demo.py`](https://github.com/bojieli/ai-agent-book/blob/main/demo.py)](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/self-evolving-tools/demo.py#L57-L72) (lines 57-72) showcases this evolution-reuse cycle: the first task creates the tool, while the second task discovers and reuses it immediately.

## Implementation Examples

Create a new tool with automatic validation:

```python
from tool_manager import ToolLibrary
lib = ToolLibrary()

code = """
def run(start, end):
    from datetime import date
    s = date.fromisoformat(start)
    e = date.fromisoformat(end)
    return {"days": (e - s).days}
"""

params = {
    "type": "object",
    "properties": {
        "start": {"type": "string", "description": "ISO start date"},
        "end":   {"type": "string", "description": "ISO end date"},
    },
    "required": ["start", "end"],
}

# Optional test_args trigger the pre-validation sandbox run

result = lib.create_tool(
    name="days_between",
    description="Calculate days between two ISO dates",
    parameters=params,
    code=code,
    test_args={"start": "2020-01-01", "end": "2020-01-10"},
)
print(result)  # → {"success": True, ...}

```

Search for existing tools by keyword:

```python
search = lib.search_tools("date days between")
print(search["tools"])  # → [{"name": "days_between", ...}]

```

Execute the registered tool in isolation:

```python
out = lib.execute_tool("days_between", {"start": "2021-03-01", "end": "2021-03-15"})
print(out["result"])  # → {"days": 14}

```

Agent-level automatic discovery and creation:

```python
from agent import SelfEvolvingAgent
agent = SelfEvolvingAgent(allow_create=True, model="gpt-4")
answer = agent.run("How many days between 2022-05-01 and 2022-05-20?")

# Agent discovers, creates, and reuses the tool automatically

print(answer)

```

## Summary

- **File-based storage** enables O(N) keyword search with minimal overhead, avoiding database complexity while scaling to hundreds of tools.
- **Pre-validation gates** using syntax checks and sandbox execution prevent invalid tools from entering the library, ensuring system stability.
- **Lazy loading** maintains constant memory footprint by reading tools from disk only when requested via `get_tool`.
- **Subprocess sandboxing** isolates execution environments to prevent tool failures from crashing the agent process.
- **Unlock tracking** eliminates redundant searches by caching discovered tool availability in the `_unlocked` set.

## Frequently Asked Questions

### How does keyword search scale to hundreds of tools without vector databases?

The `search_tools` implementation uses simple keyword tokenization and linear scanning over JSON files. While technically O(N), the approach remains performant because each tool metadata file is only a few kilobytes, making disk I/O negligible for hundreds of files. The repository prioritizes simplicity and low constant factors over the added complexity of vector search infrastructure for this scale.

### What prevents malicious or buggy tools from crashing the agent?

The system implements defense in depth through the `execute_tool` method's subprocess isolation. Each tool runs in a separate Python process with controlled `PYTHONPATH` via `_run_python_source`, ensuring memory leaks or exceptions in one tool cannot affect the agent's main process. Additionally, the `create_tool` validation gate catches syntax errors and runtime exceptions during the sandbox test phase before the tool ever enters the library.

### Can the ToolLibrary handle concurrent tool creation and execution?

Yes, because each tool is stored as an independent JSON file on the filesystem. The `ToolLibrary` does not maintain an in-memory index that requires locking, and tool execution occurs in separate subprocesses. However, concurrent writes to the same tool file should be managed at the application level, as the base implementation focuses on single-agent usage patterns.

### How does the unlocking mechanism improve performance?

After the agent successfully calls `search_tools` and finds a matching tool, the tool name is added to the `_unlocked` set within the `SelfEvolvingAgent` class. On subsequent operations, the agent checks this set first before searching, effectively creating a hot-cache of available capabilities. This prevents repeated linear scans when the same tool is needed multiple times during a conversation, reducing the search overhead to O(1) for frequently used tools.