How to Implement Dispatch Handlers for Fanning Out Work to Multiple Sessions

Dispatch handlers in the Anthropic CWC workshops map tool names to Python functions via a central dispatch router, enabling orchestrators like StockPilot to fan out complex tasks to specialized sub-agents running in independent API sessions.

The anthropics/cwc-workshops repository provides a reference implementation for building dispatch-based agent systems that distribute work across multiple sessions. This architecture allows a single orchestrating agent to remain lightweight while delegating complex operations to specialized sub-agents through separate Anthropic Messages API calls.

Architecture of the Dispatch System

The dispatch mechanism relies on three tightly integrated components defined in agent-decomposition/agents/before/tools.py.

Tool implementations are pure Python functions that perform single operations such as reading CSV files or calling sub-agents. These live in agent-decomposition/agents/before/tools.py.

TOOL_DEFS contains JSON-Schema descriptions of each tool passed to the Anthropic messages.create call. This schema teaches the model which functions are available and their required parameters.

TOOL_IMPLS and the dispatch(name, args) function form the runtime router. The dispatcher looks up function names in TOOL_IMPLS and executes them, returning JSON-encoded results or errors.

When the orchestrator receives a tool_use block from the LLM, it calls dispatch (implemented in the StockPilot loop). The dispatcher executes the appropriate implementation, potentially invoking another sub-agent such as subagents.forecasting_subagent. Because each sub-agent initiates a separate Messages-API call, the system fans out work across multiple independent sessions while the orchestrator remains single-threaded.

The Dispatch Flow

The orchestration cycle follows five distinct steps:

1. Prompt ingestion: The user prompt enters stockpilot.run, which sends the system prompt, tool schema (TOOL_DEFS), and conversation history to the LLM.

2. Tool selection: When the model requires external data or side effects, it returns a tool_use block containing a name (tool identifier) and input (arguments).

3. Handler execution: The dispatch function looks up the tool name in TOOL_IMPLS and invokes the implementation. This may read local files, write JSONL records, or delegate to a sub-agent.

4. Result wrapping: The function returns a JSON-encoded string wrapped as a tool_result block. The orchestrator appends this to the message list and continues the loop.

5. Session fan-out: Functions like forecast_demand, compare_supplier_quotes, and draft_email_to_supplier call subagents.<name> helpers, each performing a separate API request. Every sub-agent runs in its own request/response cycle with independent token accounting via subagents.token_counter.

Implementing a Basic Dispatch Handler

To add a new handler that executes locally without fanning out, modify agent-decomposition/agents/before/tools.py in three locations.

Step 1: Write the Tool Function

Create a pure function accepting typed arguments and returning a JSON-serializable string:

# agents/before/tools.py

def get_inventory_summary(warehouse: str) -> str:
    rows = _latest_stock()
    summary = {
        "warehouse": warehouse,
        "total_skus": len({r["sku"] for r in rows if r["warehouse"] == warehouse}),
        "total_on_hand": sum(int(r["on_hand"]) for r in rows if r["warehouse"] == warehouse),
    }
    return json.dumps(summary)

Source reference: Implementation appears at tools.py line 43-48.

Step 2: Register the Implementation

Add the function to the TOOL_IMPLS dictionary mapping tool names to callables:

# agents/before/tools.py – TOOL_IMPLS

TOOL_IMPLS["get_inventory_summary"] = get_inventory_summary   # line ≈ 59

Source reference: Router entry at tools.py line 59.

Step 3: Define the JSON Schema

Append a schema entry to TOOL_DEFS using the helper function _t:

# agents/before/tools.py – TOOL_DEFS

_t(
    "get_inventory_summary",
    "Return a high‑level inventory summary for a warehouse.",
    {"warehouse": _S},
    ["warehouse"],
)

Source reference: Schema definition at tools.py line 86-88.

Fanning Out to Multiple Sessions via Sub-Agents

For computationally intensive tasks requiring dedicated reasoning contexts, dispatch handlers delegate to sub-agents running in separate sessions.

Delegating to Specialized Agents

Instead of executing logic directly, the tool function calls a sub-agent helper:

# agents/before/tools.py

def run_complex_analysis(sku: str, metric: str) -> str:
    # Delegates to a separate sub‑agent that performs heavy statistical work.

    return subagents.complex_analysis_subagent(sku=sku, metric=metric)

The corresponding sub-agent in subagents.py creates an isolated Messages-API session:

def complex_analysis_subagent(sku: str, metric: str) -> str:
    system = f"You are a data‑science analyst. Compute `{metric}` for SKU `{sku}`."
    user = f"Provide a concise numeric answer and a short justification."
    return _call(system, user, max_tokens=500)

Source references: Dispatcher call at stockpilot.py line 40-45; Sub-agent helper at subagents.py line 28-38.

Independent Token Accounting

Each sub-agent invocation runs as a distinct API session. The subagents.token_counter tracks token usage separately from the orchestrator's main loop, enabling precise cost attribution across the fan-out architecture.

Complete Working Example

The following client code demonstrates the end-to-end flow through the dispatch system:

from agents.before.stockpilot import run

prompt = """I need to know the latest on‑hand quantity for SKU 12345 in warehouse A,
and then get a 30‑day demand forecast for that SKU."""
result = run(prompt)

print("Final answer:")
print(result.final_text)
print("\nTurns taken:", result.turns)
print("Tokens (inc. sub‑agents):", result.tokens_in, "/", result.tokens_out)

Source reference: Orchestrator entry point at stockpilot.py line 13-20.

Key Files and Their Roles

Understanding the file structure is essential for extending the dispatch system:

• agent-decomposition/agents/before/tools.py: Central registry containing tool implementations, JSON schema definitions (TOOL_DEFS), the TOOL_IMPLS mapping, and the dispatch router function.

• agent-decomposition/agents/before/stockpilot.py: Orchestrator loop that transmits prompts to the LLM, processes incoming tool_use blocks, and invokes dispatch to route execution.

• agent-decomposition/agents/before/subagents.py: Helper module creating independent Messages-API sessions for specialized reasoning tasks such as forecasting, procurement analysis, and content generation.

• agent-decomposition/agents/common.py: Shared constants including the MODEL identifier and the AgentResult dataclass used by both orchestrators and sub-agents.

Summary

• Dispatch handlers route LLM tool calls to Python functions via a centralized dispatch(name, args) router in tools.py.
• TOOL_DEFS provides JSON-Schema descriptions while TOOL_IMPLS maps names to functions, enabling the orchestrator to dynamically invoke capabilities.
• Fan-out architecture allows single tools to spawn separate sub-agent sessions via subagents.py, distributing complex work across multiple API calls with independent token accounting.
• Zero core-loop changes are required when adding new tools; simply register the function in TOOL_IMPLS and add its schema to TOOL_DEFS.

Frequently Asked Questions

What is the difference between TOOL_DEFS and TOOL_IMPLS?

TOOL_DEFS is a list of JSON-Schema objects describing tool names, purposes, and argument specifications sent to the Anthropic Messages API. TOOL_IMPLS is a Python dictionary mapping those same tool names to actual callable functions. The schema informs the LLM what tools exist, while the implementation dictionary provides the runtime execution logic.

How does the orchestrator handle tool results from sub-agents?

The orchestrator treats sub-agent results identically to local tool results. When dispatch invokes a sub-agent (e.g., subagents.forecasting_subagent), the sub-agent returns a JSON-encoded string that dispatch wraps as a tool_result block. StockPilot appends this result to the conversation history and continues the main loop, unaware whether the execution occurred locally or in a separate session.

Can dispatch handlers maintain state across multiple sessions?

No, dispatch handlers and sub-agents are stateless by design. Each invocation receives arguments via the tool_use block and returns JSON-encoded results. Persistent state must be managed externally through file I/O (CSV/JSONL operations shown in the stock tools) or database calls, as the Messages API does not maintain server-side state between independent sessions.

How do I add error handling to dispatch functions?

Wrap tool implementations in standard Python try-except blocks and return JSON-encoded error objects. The dispatch function in tools.py executes the looked-up function directly, so unhandled exceptions will propagate to the orchestrator. Best practice involves catching expected errors (file not found, API timeouts) and returning structured error messages as strings, allowing the LLM to retry or respond appropriately.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}