How Agents Discover and Select Optimal Tool Combinations Dynamically

Agents dynamically discover and select optimal tool combinations by detecting capability gaps, retrieving relevant specialists via semantic embedding search, and following strict protocols that prioritize domain-specific tools over generic fallbacks.

The bojieli/ai-agent-book repository demonstrates how modern AI agents can move beyond static toolsets to dynamically discover and select optimal tool combinations during task execution. Instead of relying on pre-defined tool lists or falling back to generic utilities like web search, these agents use semantic discovery and protocol-driven selection to assemble the precise capabilities needed for each sub-task. This article examines the implementation details from Chapter 4's active tool discovery system, revealing how agents balance discovery overhead with execution accuracy.

The Three-Pillar Architecture

The dynamic tool selection system rests on three integrated components: a comprehensive typed catalog, a semantic retrieval engine, and a strict selection protocol.

Rich Typed Tool Catalog

In chapter4/active-tool-discovery/tools_library.py, the system defines over 120 OpenAI-compatible function schemas organized by domain. The library exposes categorization helpers including ALL_TOOLS, BASE_TOOL_NAMES, and GENERIC_TOOL_NAMES to distinguish between foundational utilities and specialized capabilities.

The catalog provides the substrate for discovery. When the agent begins execution, it only has access to base tools—specifically calculator and get_current_time—plus the discovery mechanism itself. All other tools remain latent until semantic relevance triggers their injection.

Semantic Discovery Engine

When the agent encounters a task requiring unavailable capabilities, it invokes the discover_tools action. This triggers LocalEmbeddingIndex.search() in chapter4/active-tool-discovery/run_exact_experiment.py, which encodes every tool schema using sentence-transformers/all-MiniLM-L6-v2 embeddings.

The index performs dense retrieval against the agent's stated need (e.g., "real-time stock quote"), returning the top-k most relevant schemas. These schemas are then rendered and injected into the conversation context, effectively expanding the agent's available toolset mid-session.

Rule-Driven Selection Protocol

The _AGENT_PROTOCOL defined in run_exact_experiment.py enforces strict behavioral constraints through lines 78-86. The protocol mandates that agents:

• Call discover_tools only when a needed capability is absent from the current context
• Prefer specialists from the freshly-injected schema set over generic alternatives
• Avoid generic fallbacks like web_search or code_interpreter unless explicitly permitted

This rule system prevents lazy tool selection and forces the agent to leverage domain-specific specialists discovered through the semantic index.

The Dynamic Discovery Workflow

The runtime workflow follows a precise sequence from gap detection to specialist execution, with each step recorded in the conversation history.

Detecting Capability Gaps

When the model determines it lacks a required capability, it emits a JSON action:

{
  "action": "discover_tools",
  "need": "real-time stock quote for a ticker"
}

The parse_action() function (lines 95-105) intercepts this signal and routes it to the embedding index. This represents the agent's explicit recognition that its current toolset is insufficient for the task at hand.

Semantic Retrieval and Schema Injection

Upon receiving a discovery request, the system queries LocalEmbeddingIndex.search() (lines 64-72) to retrieve matching schemas. The injection process in append_history() (lines 125-136) adds these schemas to the conversation while updating the available tool set.

Each injection includes a SHA-256 proof of the schema block (lines 166-176), creating a cryptographic receipt that ensures tool integrity and prevents tampering with the discovered capabilities.

Specialist Tool Execution

With new schemas available, the agent issues precise calls to the discovered specialists:

{
  "action": "call_tool",
  "tool": "get_stock_price",
  "query": "AAPL",
  "options": {}
}

The _call_real_tool() function (lines 140-158) executes the underlying MCP tool, while mcp_receipt() records the execution trace. This strict separation between discovery (discover_tools) and execution (call_tool) ensures that the agent only invokes tools that have been explicitly vetted and injected into the context.

Verification and Accuracy

Measuring Selection Quality

The system validates tool selection through grade_plan() (lines 27-34). This grading function verifies two critical conditions:

• Slot coverage: Every required capability slot in the task plan must be satisfied by at least one specialist tool call
• Generic avoidance: The agent receives penalties for incorrectly using generic tools when specialists were available

This objective measurement ensures that the semantic discovery system actually improves task performance rather than adding unnecessary overhead.

Practical Implementation Example

The following pattern from run_exact_experiment.py demonstrates a multi-domain workflow combining finance and news retrieval:

def run_finance_news_task(session, index):
    # Step 1: Discover stock price capability

    discovery = agent.send({
        "action": "discover_tools",
        "need": "stock price data"
    })
    
    # Step 2: Call the discovered specialist

    price = agent.send({
        "action": "call_tool",
        "tool": "get_stock_price",
        "query": "AAPL",
        "options": {}
    })
    
    # Step 3: Discover news capability for the same ticker

    discovery2 = agent.send({
        "action": "discover_tools",
        "need": "latest news for AAPL"
    })
    
    # Step 4: Use specialist search_news (not generic web_search)

    news = agent.send({
        "action": "call_tool",
        "tool": "search_news",
        "query": "AAPL",
        "options": {}
    })
    
    # Step 5: Combine results

    return agent.send({
        "action": "finish",
        "answer": f"Stock: ${price['price']}; Headline: {news[0]['title']}"
    })

This implementation demonstrates how agents iteratively assemble optimal tool combinations by discovering specialists for each sub-task domain rather than relying on monolithic generic utilities.

Summary

• Dynamic discovery enables agents to start with minimal toolsets and expand capabilities on-demand through semantic search against a catalog of 120+ typed schemas.

• Semantic retrieval uses LocalEmbeddingIndex with all-MiniLM-L6-v2 embeddings to match capability descriptions with relevant tool schemas, ensuring high relevance in top-k results.

• Protocol enforcement through _AGENT_PROTOCOL prevents generic tool fallback and mandates specialist discovery, verified by grade_plan() checking for slot coverage and generic misuse.

• Integrity verification via SHA-256 proofs in append_history() ensures that discovered tool schemas remain tamper-evident throughout the conversation.

Frequently Asked Questions

How does semantic tool discovery work?

When an agent detects a capability gap, it emits a discover_tools action describing its need. The LocalEmbeddingIndex.search() function encodes this description and all tool schemas using all-MiniLM-L6-v2 embeddings, performing dense retrieval to return the most semantically similar tool definitions. These schemas are then injected into the conversation context, making them available for subsequent calls.

What prevents agents from using generic tools?

The _AGENT_PROTOCOL explicitly forbids agents from using generic utilities like web_search or code_interpreter unless no specialist exists for the required capability. This protocol is enforced through the system prompt and validated by grade_plan(), which penalizes generic tool usage when domain-specific alternatives were available.

How is tool selection accuracy measured?

The grade_plan() function in run_exact_experiment.py evaluates execution traces against task requirements. It verifies that every required capability slot is filled by an appropriate specialist tool and flags any instances where generic tools were used incorrectly. This provides an objective metric for whether the semantic discovery system successfully identified optimal tool combinations.

Can this system handle multi-domain queries?

Yes. The iterative discovery pattern allows agents to sequentially discover tools for distinct domains. For example, an agent can first discover get_stock_price for financial data, then discover search_news for information retrieval, assembling the optimal combination of specialists needed for complex cross-domain tasks without loading unnecessary tools upfront.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}