# Active Tool Discovery in Large-Scale LLM Agent Systems: 3 Strategies Explained

> Discover 3 effective strategies for active tool discovery in large-scale LLM agent systems. Reduce token usage and maintain 100% recall with dynamic tool requests.

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: deep-dive
- Published: 2026-07-25

---

**Active tool discovery enables LLM agents to dynamically request only the tools they need during task execution, reducing context token usage from thousands to hundreds while maintaining 100% recall in large-scale tool catalogs.**

Large-scale LLM agents face a critical scalability challenge known as the **tool-explosion problem**, where hundreds of available tool schemas overwhelm the model's context window and degrade performance. The `bojieli/ai-agent-book` repository demonstrates how **active tool discovery** strategies allow agents to request capabilities on-demand rather than loading entire tool catalogs upfront, dramatically improving token efficiency and task accuracy in production systems.

## The Tool-Explosion Problem

When an agent ecosystem contains dozens or hundreds of possible tools, injecting every tool schema into the prompt overwhelms the model's context window and forces the agent into a passive "pick-one-of-many" mode. This linear growth in token consumption makes it impossible to scale beyond small tool sets without degrading response quality or incurring prohibitive API costs.

## Three Strategies for Active Tool Discovery

The repository implements three complementary strategies for managing tool discovery at different scales:

| Strategy | Implementation | Token Impact | Best For |
|----------|---------------|--------------|----------|
| `all-tools` (passive) | All tool schemas injected at start | Linear growth with catalog size—thousands of tokens for 200+ tools | Tiny tool sets (< 10) where every tool will be used |
| `retrieval` (semantic top-k) | Single semantic retrieval selects *k* most relevant tools before first LLM call | Token cost stays flat—only top-k schemas sent | Medium-size catalogs where static top-k suffices |
| `active` (MCP-Zero) | Agent analyzes task, emits structured `<tool_request>`, loads matching tools on-demand | Tokens grow only with actually used tools; recall stays 100% while keeping schema cost low | Very large catalogs (hundreds+ tools) and multi-turn tasks where requirements evolve |

Quantitative benchmarks from the repository show token savings from **3,857 → 540 tokens** when moving from `all-tools` to `retrieval`, with active discovery achieving similar efficiency gains while preserving full recall.

## Core Architectural Elements of Active Discovery

### Agent Execution Loop

The `ActiveToolAgent.execute_task` method in [`chapter4/active-tool-selection/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/agent.py) implements a progressive disclosure pattern. The agent iterates up to `MAX_TOOL_REQUESTS` (configurable), alternating between LLM calls and tool-request handling. This allows the agent to start with no tools, discover only the needed ones, and incrementally build a toolchain that mirrors the evolving understanding of the task.

### Structured Tool Requests

The system prompt defined in `agent._create_system_message` establishes a strict XML-like syntax for capability requests:

```text
<tool_request>
server: …  
tool: …  
</tool_request>

```

When the LLM detects a missing capability, it fills in the `server` and `tool` fields, triggering the discovery mechanism. This structured format eliminates ambiguity in intent parsing and enables deterministic routing.

### Hierarchical Semantic Router

Implemented in [`chapter4/active-tool-selection/semantic_router.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/semantic_router.py), the **hierarchical semantic router** executes a two-stage reduction. It first matches the request to a **server** (e.g., GitHub, Filesystem, Database) and then to a specific **tool** within that server using **TF-IDF vector similarity** via `router.route_request`. This approach cuts the search space from *O(N)* to *O(log N)*, making discovery feasible even with hundreds of available tools.

### Tool Knowledge Base Design

The `tool_knowledge_base.create_tool_knowledge_base` function in [`chapter4/active-tool-selection/tool_knowledge_base.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/tool_knowledge_base.py) defines **8 domain servers**, each exposing 4-5 concrete tools (total 35 tools, optionally padded with synthetic distractors). This data structure is intentionally modular—new servers or tools can be appended without modifying the core agent logic, supporting extensible domain coverage.

### Metrics and Benchmarking

Each agent records **token usage**, **number of tool requests**, and **which tools were actually called**. The offline deterministic benchmark in [`chapter4/active-tool-selection/benchmark.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/benchmark.py) provides a labeled set of 10 tasks and measures **recall@k** and **schema token cost** without any API calls, guaranteeing reproducible performance comparisons across strategies.

## Practical Implementation Examples

### Basic Active Discovery

Execute a task requiring GitHub search capabilities:

```python
from agent import ActiveToolAgent

agent = ActiveToolAgent()
result = agent.execute_task("Search for Python ML repositories on GitHub")
print("Tools loaded:", result["tools_loaded"])      # e.g. ['github_search_repos']

print("Response:", result["response"])

```

### Multi-Domain Task Execution

Complex workflows spanning multiple domains trigger sequential tool discovery:

```python
from agent import ActiveToolAgent

task = """
1. Query the database for recent user sign‑ups.
2. Compute the mean and median of the signup timestamps.
3. Email the summary report to the analytics team.
"""
agent = ActiveToolAgent()
out = agent.execute_task(task)

print("Loaded tools:", out["tools_loaded"])

# Expected output: ['db_query', 'analytics_summarize', 'comm_send_email']

print("Metrics:", out["metrics"])

```

### Strategy Comparison (Offline)

Run deterministic benchmarks without API calls using [`demo_comparison.py`](https://github.com/bojieli/ai-agent-book/blob/main/demo_comparison.py):

```python
from demo_comparison import demo_comparison

# Offline run – no API key required

demo_comparison.main(["--offline"])

# Output includes:

# | Strategy | Tools in context | Schema tokens | Recall |

# |----------|------------------|---------------|--------|

# | all‑tools| 35               | 3 857         | 100 % |

# | retrieval| 5                | 551           | 100 % |

```

### Extending the Knowledge Base

Add new tools by modifying [`tool_knowledge_base.py`](https://github.com/bojieli/ai-agent-book/blob/main/tool_knowledge_base.py):

```python

# In tool_knowledge_base.py

new_tool = ToolDefinition(
    name="cloud_restart_vm",
    description="Restart a virtual machine in the cloud",
    parameters={
        "type": "object",
        "properties": {
            "instance_id": {"type": "string", "description": "ID of the VM"},
            "region": {"type": "string", "description": "Cloud region"}
        },
        "required": ["instance_id"]
    },
    server="cloud"
)

# Append to the cloud server's tool list and re‑run the agent.

```

## Summary

- **Active tool discovery** solves the tool-explosion problem by loading tool schemas on-demand rather than at initialization, reducing context window pressure from thousands to hundreds of tokens.
- The **MCP-Zero strategy** combines structured XML requests with hierarchical semantic routing to achieve *O(log N)* discovery complexity while maintaining 100% recall.
- **Key implementation files** include [`chapter4/active-tool-selection/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/agent.py) for the agent loop, [`semantic_router.py`](https://github.com/bojieli/ai-agent-book/blob/main/semantic_router.py) for TF-IDF routing, and [`tool_knowledge_base.py`](https://github.com/bojieli/ai-agent-book/blob/main/tool_knowledge_base.py) for extensible catalog management.
- **Offline benchmarking** via [`benchmark.py`](https://github.com/bojieli/ai-agent-book/blob/main/benchmark.py) enables reproducible measurement of recall@k and token cost without consuming API credits.
- The architecture supports **progressive disclosure**, allowing agents to build toolchains incrementally as task requirements evolve across multiple turns.

## Frequently Asked Questions

### What is the tool-explosion problem in LLM agents?

The tool-explosion problem occurs when an agent's available tool catalog grows to dozens or hundreds of options, making it impossible to inject all tool schemas into the context window without overwhelming the model or exceeding token limits. This forces the agent into a passive selection mode where it must choose from hundreds of irrelevant options, degrading accuracy and response quality.

### How does active tool discovery reduce token costs?

Active tool discovery reduces token costs by implementing **progressive disclosure**—the agent starts with zero or minimal tool definitions and emits structured `<tool_request>` XML tags when it identifies capability gaps. Only the requested tool schemas are loaded into context, keeping the token count proportional to the number of actually used tools rather than the total catalog size, as measured in the reduction from **3,857 to 540 tokens** in benchmark tests.

### What is MCP-Zero in the context of active tool discovery?

**MCP-Zero** refers to the active discovery strategy where the agent begins with zero pre-loaded tools and dynamically requests capabilities using structured XML syntax. Unlike passive baseline strategies or static top-k retrieval, MCP-Zero allows the agent to discover tools across multiple turns as task requirements evolve, using a hierarchical semantic router to locate specific capabilities within large-scale catalogs.

### How do I add custom tools to the active discovery system?

Custom tools are added by extending the **Tool Knowledge Base** in [`chapter4/active-tool-selection/tool_knowledge_base.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-selection/tool_knowledge_base.py). Create a new `ToolDefinition` object with the tool name, description, and JSON schema parameters, then assign it to an existing server or create a new server category. The hierarchical semantic router in [`semantic_router.py`](https://github.com/bojieli/ai-agent-book/blob/main/semantic_router.py) automatically indexes new tools using TF-IDF vectorization, requiring no changes to the core agent execution logic.