# Developing Agents for Specific Domains Like Healthcare or Finance: A Modular Blueprint

> Develop AI agents for healthcare or finance. Learn a modular blueprint to curate tool libraries, restrict LLM functions, and boost tool selection precision for specialized AI.

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: modular-blueprint
- Published: 2026-07-30

---

**Build domain-specific AI agents by curating specialized tool libraries, restricting the LLM's available functions through system prompts, and evaluating tool selection precision to prevent generic fallbacks like web search.**

The **ai-agent-book** repository provides a production-ready framework for developing agents for specific domains like healthcare or finance without rebuilding core architecture. The codebase implements a tool-centric separation of concerns where domain expertise lives in modular tool libraries rather than monolithic agents, allowing the same generic agent backbone to serve financial analysis, healthcare diagnostics, or e-commerce operations through configuration rather than code changes.

## Core Architecture for Domain-Specific Agents

Domain specialization in this architecture is achieved through five interconnected components that isolate vertical knowledge from general reasoning:

| Component | Role | Implementation Location |
|---|---|---|
| **LLM + Prompt Engine** | Generates responses and decides tool invocation strategy. | [`chapter2/system-hint/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter2/system-hint/agent.py) contains the generic `Agent` class with a `run` loop that wraps any LLM backend. |
| **Tool Registry** | Stores OpenAI-compatible function schemas organized by vertical. | [`chapter4/active-tool-discovery/tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/tools_library.py) defines 126 tools across finance, news, web, crypto, and healthcare categories. |
| **Tool Selector** | Dynamically prunes the toolbox to match task requirements. | The `select_tools` function (lines 107-123) filters available functions based on task size and required slots. |
| **Domain Prompt** | Injects domain-specific tool lists and restrictions into the system message. | [`chapter4/active-tool-discovery/demo.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/demo.py) constructs prompts that enumerate allowed tools and explicitly forbid generic fallbacks. |
| **Evaluation Harness** | Grades whether the agent used precise domain tools versus generic substitutes. | The `grade` function (lines 49-70) checks for missed required slots and unauthorized generic tool usage. |

## How Domain-Specificity Is Achieved

The repository implements four specific mechanisms to ensure agents utilize domain expertise rather than generic reasoning:

**Tool Set Pruning via `select_tools`**

Before the LLM receives any prompt, the `select_tools` function builds a constrained toolbox that always includes base utilities (`calculator`, `get_current_time`) but **injects only the tools required by the chosen domain task**. For a finance query, this means loading `get_stock_price` and `get_company_financials` while excluding unnecessary healthcare or travel functions. This mechanical restriction prevents the model from drifting toward generic `web_search` when specialized APIs exist.

**System Prompt Engineering with Tool Enumeration**

The system prompt explicitly lists allowed functions and domain boundaries:

```text
You are a finance assistant. You may call the following tools:
- get_stock_price(symbol): retrieve latest price
- get_company_financials(symbol): fetch quarterly statements
Do NOT use web_search for stock data.

```

This domain-aware instruction conditions the LLM to recognize that `web_search` is explicitly out of scope for financial data queries, steering the model toward specialized functions defined in [`tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/tools_library.py).

**Grading and Feedback via `grade` Function**

After execution, the `grade(task, called_tools)` function (implemented in [`chapter4/active-tool-discovery/tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/tools_library.py)) verifies whether every **required slot** was filled by the precise domain tool. It returns a `precise` flag indicating whether the agent correctly used specialized APIs or incorrectly fell back to generic substitutes like `quick_answer`. This feedback loop allows developers to detect when agents hallucinate tool availability or bypass domain expertise.

**Domain-Specific Test Data Generation**

The [`generate_test_cases.py`](https://github.com/bojieli/ai-agent-book/blob/main/generate_test_cases.py) script in `chapter3/user-memory-evaluation/` synthesizes long-form conversation scenarios (up to 55 rounds) for specific verticals including healthcare and finance. These YAML specifications simulate realistic user queries that force the agent to retrieve domain-relevant facts, creating a validation dataset that drives evaluation of tool selection accuracy.

## Extending to a New Domain

Adding healthcare or any new vertical requires three concrete steps using the existing modular infrastructure:

1. **Define Domain Tools** in [`chapter4/active-tool-discovery/tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/tools_library.py) using the `_tool` helper function and append them to the `_DEFS` list under a new section header.
2. **Create Test Specifications** in `chapter3/user-memory-evaluation/` by extending [`generate_test_cases.py`](https://github.com/bojieli/ai-agent-book/blob/main/generate_test_cases.py) with a new scenario entry (e.g., `"healthcare"`) that produces domain-specific utterances and required tool slots.
3. **Execute Domain Runs** using the demo script with task filtering: `python chapter4/active-tool-discovery/demo.py --tasks healthcare`.

## Code Examples

### Loading a Finance-Only Agent

This example demonstrates how to initialize an agent restricted to financial tools using the `select_tools` function and the generic `Agent` class from [`chapter2/system-hint/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter2/system-hint/agent.py):

```python
from chapter4.active_tool_discovery.tools_library import select_tools, TOOLS_BY_NAME
from chapter2.system_hint.agent import Agent

# Select a toolbox of size 30 containing only finance and news tools

tool_defs = select_tools(size=30, tasks=[{"id": "finance+news"}])

# Build a restrictive system prompt enumerating allowed functions

system_prompt = "You are a finance assistant. You may call the following tools:\n"
for t in tool_defs:
    name = t["function"]["name"]
    desc = t["function"]["description"]
    system_prompt += f"- {name}: {desc}\n"
system_prompt += "Do NOT use web_search for stock data."

# Initialize the agent with filtered tools

agent = Agent(
    llm_name="gpt-4o",
    system_prompt=system_prompt,
    tools={name: TOOLS_BY_NAME[name] for name in [t["function"]["name"] for t in tool_defs]},
)

# Execute a query requiring the specialized get_stock_price tool

response = agent.run("What is the latest price of AAPL?")
print(response)

```

The LLM receives the constrained context, invokes `get_stock_price` instead of `web_search`, and returns the formatted result using domain-specific data structures defined in the tool library.

### Evaluating Precise Tool Selection

Use the `grade` function to verify whether your domain agent avoided generic fallbacks:

```python
from chapter4.active_tool_discovery.tools_library import TASKS, grade

# Load a finance task definition

task = next(t for t in TASKS if t["id"] == "finance+news")

# Simulate an agent that incorrectly used web_search

called_tools = ["get_stock_price", "web_search"]
result = grade(task, called_tools)

print(f"Precise execution: {result['precise']}")
print(f"Missed required slots: {result['missed_slots']}")
print(f"Generic substitutes used: {result['used_generic_substitute']}")

```

The output flags the execution as **imprecise** because the agent utilized `web_search` as a substitute for the required financial API, triggering a failure in the evaluation harness.

### Adding a New Healthcare Tool

Extend the tool library with EMR integration capabilities by appending to `_DEFS` in [`tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/tools_library.py):

```python

# In chapter4/active-tool-discovery/tools_library.py

_DEFS += [
    _tool(
        "get_patient_lab_results",
        "Retrieve a patient's latest lab test results from the EMR system.",
        {"patient_id": _s("EMR patient identifier")}
    ),
]

```

Once defined, `select_tools()` can include `get_patient_lab_results` for healthcare tasks, and the grading harness will treat it as a required slot when you add corresponding entries to the `TASKS` configuration.

## Summary

- **Developing agents for specific domains like healthcare or finance** relies on tool library curation rather than agent reimplementation, using the modular architecture in `ai-agent-book`.
- The `select_tools` function mechanically restricts the LLM's available functions to domain-specific APIs, preventing generic fallback behaviors.
- System prompts must explicitly enumerate allowed tools and forbid general-purpose alternatives like `web_search` to steer the model toward specialized reasoning.
- The `grade` evaluation function in [`tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/tools_library.py) provides automated verification that agents utilize precise domain tools rather than generic substitutes.
- New verticals require only three steps: define tools using the `_tool` schema, generate test cases via [`generate_test_cases.py`](https://github.com/bojieli/ai-agent-book/blob/main/generate_test_cases.py), and execute through the filtered demo pipeline.

## Frequently Asked Questions

### How do I prevent my domain agent from falling back to generic web search?

Mechanically restrict the tool registry using `select_tools` to exclude `web_search` and `quick_answer` from the available functions, then explicitly forbid these tools in the system prompt. According to the `ai-agent-book` source code, this combination of tool pruning and prompt engineering prevents the LLM from invoking generic fallbacks when specialized APIs exist for the domain.

### What is the role of the `select_tools` function in domain specialization?

The `select_tools` function (lines 107-123 in [`chapter4/active-tool-discovery/tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/tools_library.py)) dynamically builds a domain-specific toolbox by loading only the tools required for a given task ID plus essential base utilities. This ensures that healthcare agents cannot accidentally call financial APIs and vice versa, creating strict boundaries for tool selection without modifying the core agent logic.

### How does the evaluation harness verify domain expertise?

The `grade` function checks whether the agent called the exact tools specified in the task's **required slots** and flags any usage of **generic substitutes** like `web_search`. As implemented in lines 49-70 of [`tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/tools_library.py), it returns a `precise` boolean indicating whether the agent correctly utilized domain-specific APIs or incorrectly defaulted to general-purpose search, providing quantitative feedback on vertical specialization.

### Can I add custom tools for a niche industry not covered in the repository?

Yes. Define new tools using the `_tool(name, description, params)` helper pattern in [`chapter4/active-tool-discovery/tools_library.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/active-tool-discovery/tools_library.py), append them to the `_DEFS` list under a descriptive comment block (e.g., `# --- manufacturing ---`), and update the `TASKS` configuration to include your vertical. The existing `Agent` class in [`chapter2/system-hint/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter2/system-hint/agent.py) will automatically recognize and execute these tools without requiring changes to the agent's core implementation.