Developing Agents for Specific Domains Like Healthcare or Finance: A Modular Blueprint

Build domain-specific AI agents by curating specialized tool libraries, restricting the LLM's available functions through system prompts, and evaluating tool selection precision to prevent generic fallbacks like web search.

The ai-agent-book repository provides a production-ready framework for developing agents for specific domains like healthcare or finance without rebuilding core architecture. The codebase implements a tool-centric separation of concerns where domain expertise lives in modular tool libraries rather than monolithic agents, allowing the same generic agent backbone to serve financial analysis, healthcare diagnostics, or e-commerce operations through configuration rather than code changes.

Core Architecture for Domain-Specific Agents

Domain specialization in this architecture is achieved through five interconnected components that isolate vertical knowledge from general reasoning:

Component	Role	Implementation Location
LLM + Prompt Engine	Generates responses and decides tool invocation strategy.	chapter2/system-hint/agent.py contains the generic Agent class with a run loop that wraps any LLM backend.
Tool Registry	Stores OpenAI-compatible function schemas organized by vertical.	chapter4/active-tool-discovery/tools_library.py defines 126 tools across finance, news, web, crypto, and healthcare categories.
Tool Selector	Dynamically prunes the toolbox to match task requirements.	The select_tools function (lines 107-123) filters available functions based on task size and required slots.
Domain Prompt	Injects domain-specific tool lists and restrictions into the system message.	chapter4/active-tool-discovery/demo.py constructs prompts that enumerate allowed tools and explicitly forbid generic fallbacks.
Evaluation Harness	Grades whether the agent used precise domain tools versus generic substitutes.	The grade function (lines 49-70) checks for missed required slots and unauthorized generic tool usage.

How Domain-Specificity Is Achieved

The repository implements four specific mechanisms to ensure agents utilize domain expertise rather than generic reasoning:

Tool Set Pruning via select_tools

Before the LLM receives any prompt, the select_tools function builds a constrained toolbox that always includes base utilities (calculator, get_current_time) but injects only the tools required by the chosen domain task. For a finance query, this means loading get_stock_price and get_company_financials while excluding unnecessary healthcare or travel functions. This mechanical restriction prevents the model from drifting toward generic web_search when specialized APIs exist.

System Prompt Engineering with Tool Enumeration

The system prompt explicitly lists allowed functions and domain boundaries:

You are a finance assistant. You may call the following tools:
- get_stock_price(symbol): retrieve latest price
- get_company_financials(symbol): fetch quarterly statements
Do NOT use web_search for stock data.

This domain-aware instruction conditions the LLM to recognize that web_search is explicitly out of scope for financial data queries, steering the model toward specialized functions defined in tools_library.py.

Grading and Feedback via grade Function

After execution, the grade(task, called_tools) function (implemented in chapter4/active-tool-discovery/tools_library.py) verifies whether every required slot was filled by the precise domain tool. It returns a precise flag indicating whether the agent correctly used specialized APIs or incorrectly fell back to generic substitutes like quick_answer. This feedback loop allows developers to detect when agents hallucinate tool availability or bypass domain expertise.

Domain-Specific Test Data Generation

The generate_test_cases.py script in chapter3/user-memory-evaluation/ synthesizes long-form conversation scenarios (up to 55 rounds) for specific verticals including healthcare and finance. These YAML specifications simulate realistic user queries that force the agent to retrieve domain-relevant facts, creating a validation dataset that drives evaluation of tool selection accuracy.

Extending to a New Domain

Adding healthcare or any new vertical requires three concrete steps using the existing modular infrastructure:

1. Define Domain Tools in chapter4/active-tool-discovery/tools_library.py using the _tool helper function and append them to the _DEFS list under a new section header.
2. Create Test Specifications in chapter3/user-memory-evaluation/ by extending generate_test_cases.py with a new scenario entry (e.g., "healthcare") that produces domain-specific utterances and required tool slots.
3. Execute Domain Runs using the demo script with task filtering: python chapter4/active-tool-discovery/demo.py --tasks healthcare.

Code Examples

Loading a Finance-Only Agent

This example demonstrates how to initialize an agent restricted to financial tools using the select_tools function and the generic Agent class from chapter2/system-hint/agent.py:

from chapter4.active_tool_discovery.tools_library import select_tools, TOOLS_BY_NAME
from chapter2.system_hint.agent import Agent

# Select a toolbox of size 30 containing only finance and news tools

tool_defs = select_tools(size=30, tasks=[{"id": "finance+news"}])

# Build a restrictive system prompt enumerating allowed functions

system_prompt = "You are a finance assistant. You may call the following tools:\n"
for t in tool_defs:
    name = t["function"]["name"]
    desc = t["function"]["description"]
    system_prompt += f"- {name}: {desc}\n"
system_prompt += "Do NOT use web_search for stock data."

# Initialize the agent with filtered tools

agent = Agent(
    llm_name="gpt-4o",
    system_prompt=system_prompt,
    tools={name: TOOLS_BY_NAME[name] for name in [t["function"]["name"] for t in tool_defs]},
)

# Execute a query requiring the specialized get_stock_price tool

response = agent.run("What is the latest price of AAPL?")
print(response)

The LLM receives the constrained context, invokes get_stock_price instead of web_search, and returns the formatted result using domain-specific data structures defined in the tool library.

Evaluating Precise Tool Selection

Use the grade function to verify whether your domain agent avoided generic fallbacks:

from chapter4.active_tool_discovery.tools_library import TASKS, grade

# Load a finance task definition

task = next(t for t in TASKS if t["id"] == "finance+news")

# Simulate an agent that incorrectly used web_search

called_tools = ["get_stock_price", "web_search"]
result = grade(task, called_tools)

print(f"Precise execution: {result['precise']}")
print(f"Missed required slots: {result['missed_slots']}")
print(f"Generic substitutes used: {result['used_generic_substitute']}")

The output flags the execution as imprecise because the agent utilized web_search as a substitute for the required financial API, triggering a failure in the evaluation harness.

Adding a New Healthcare Tool

Extend the tool library with EMR integration capabilities by appending to _DEFS in tools_library.py:

# In chapter4/active-tool-discovery/tools_library.py

_DEFS += [
    _tool(
        "get_patient_lab_results",
        "Retrieve a patient's latest lab test results from the EMR system.",
        {"patient_id": _s("EMR patient identifier")}
    ),
]

Once defined, select_tools() can include get_patient_lab_results for healthcare tasks, and the grading harness will treat it as a required slot when you add corresponding entries to the TASKS configuration.

Summary

• Developing agents for specific domains like healthcare or finance relies on tool library curation rather than agent reimplementation, using the modular architecture in ai-agent-book.
• The select_tools function mechanically restricts the LLM's available functions to domain-specific APIs, preventing generic fallback behaviors.
• System prompts must explicitly enumerate allowed tools and forbid general-purpose alternatives like web_search to steer the model toward specialized reasoning.
• The grade evaluation function in tools_library.py provides automated verification that agents utilize precise domain tools rather than generic substitutes.
• New verticals require only three steps: define tools using the _tool schema, generate test cases via generate_test_cases.py, and execute through the filtered demo pipeline.

Frequently Asked Questions

How do I prevent my domain agent from falling back to generic web search?

Mechanically restrict the tool registry using select_tools to exclude web_search and quick_answer from the available functions, then explicitly forbid these tools in the system prompt. According to the ai-agent-book source code, this combination of tool pruning and prompt engineering prevents the LLM from invoking generic fallbacks when specialized APIs exist for the domain.

What is the role of the select_tools function in domain specialization?

The select_tools function (lines 107-123 in chapter4/active-tool-discovery/tools_library.py) dynamically builds a domain-specific toolbox by loading only the tools required for a given task ID plus essential base utilities. This ensures that healthcare agents cannot accidentally call financial APIs and vice versa, creating strict boundaries for tool selection without modifying the core agent logic.

How does the evaluation harness verify domain expertise?

The grade function checks whether the agent called the exact tools specified in the task's required slots and flags any usage of generic substitutes like web_search. As implemented in lines 49-70 of tools_library.py, it returns a precise boolean indicating whether the agent correctly utilized domain-specific APIs or incorrectly defaulted to general-purpose search, providing quantitative feedback on vertical specialization.

Can I add custom tools for a niche industry not covered in the repository?

Yes. Define new tools using the _tool(name, description, params) helper pattern in chapter4/active-tool-discovery/tools_library.py, append them to the _DEFS list under a descriptive comment block (e.g., # --- manufacturing ---), and update the TASKS configuration to include your vertical. The existing Agent class in chapter2/system-hint/agent.py will automatically recognize and execute these tools without requiring changes to the agent's core implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}