How to Implement Structured Output with ToolMode Configurations in Agent Framework

Agent Framework lets you force specific tool calls while enforcing strict JSON schemas by combining the tool_choice option with response_format, returning validated Pydantic objects via response.value.

The microsoft/agent-framework enables developers to build reliable LLM agents that both invoke specific functions and return structured data. By configuring ToolMode options alongside structured output schemas, you can constrain model behavior to call required tools and parse responses into typed Python objects.

Define a Response Schema for Structured Output

Structured output ensures the model returns JSON matching a predefined schema rather than free-form text. Define your schema using Pydantic and pass it to the agent via the response_format parameter.

from pydantic import BaseModel

class WeatherReport(BaseModel):
    location: str
    temperature_c: float
    condition: str

When the agent finishes execution, AgentResponse.value contains an instance of your model (or None if the LLM failed to obey the format). Internally, Agent._prepare_response_and_text_format stores this schema in python/packages/openai/agent_framework_openai/_chat_client.py, where the client later parses the raw JSON into the Pydantic object.

Configure ToolMode to Control Tool Invocation

ToolMode dictates whether and which functions the model must call. The framework supports three modes:

• "auto": The model decides when to call a tool.
• "required": The model must call a tool; optionally restricted to a specific function via required_function_name.
• "none": Tool calling is disabled.

Pass these as either a raw string or a dictionary:

# Simple mode

options = {"tool_choice": "auto"}

# Require a specific function

options = {
    "tool_choice": {
        "mode": "required",
        "required_function_name": "get_weather"
    }
}

Validate ToolMode with validate_tool_mode

Before sending requests to the provider, the framework sanitizes inputs using validate_tool_mode in python/packages/core/agent_framework/_types.py (lines 3353–3366). This function ensures the dictionary contains valid keys (mode, required_function_name) and raises ContentError for invalid configurations.

Merge Runtime Options with merge_chat_options

When an agent runs, default options merge with runtime overrides via merge_chat_options in python/packages/core/agent_framework/_types.py (lines 3370–3395). This function preserves the higher-priority tool_choice from runtime options over agent defaults:

from agent_framework import merge_chat_options

merged = merge_chat_options(agent_defaults, run_options)

# run_options.tool_choice takes precedence

Provider-Specific Payload Translation

Each LLM provider translates the generic ToolMode into its native API format. For OpenAI, the client in python/packages/openai/agent_framework_openai/_chat_client.py (lines 1195–1208) converts the configuration:

tool_mode = validate_tool_mode(tool_choice)
if tool_mode is not None:
    if (mode := tool_mode.get("mode")) == "required" and (
        func_name := tool_mode.get("required_function_name")
    ) is not None:
        run_options["tool_choice"] = {"type": "function", "name": func_name}
    else:
        run_options["tool_choice"] = mode

If no tools are registered, the client automatically strips tool_choice and parallel_tool_calls to avoid API errors.

Complete Working Example

The following implementation combines a required tool call with structured output parsing:

from agent_framework import Agent, FunctionTool, Message, Content
from pydantic import BaseModel

# 1. Define structured output schema

class WeatherReport(BaseModel):
    location: str
    temperature_c: float
    condition: str

# 2. Create the tool function

def get_weather(location: str) -> dict:
    """Retrieve weather for a specific location."""
    return {"location": location, "temperature_c": 22.5, "condition": "Sunny"}

weather_tool = FunctionTool(
    name="get_weather",
    description="Get current weather for a location.",
    func=get_weather
)

# 3. Initialize agent with defaults

agent = Agent(
    client="openai",
    model="gpt-4o-mini",
    tools=[weather_tool],
    response_format=WeatherReport,  # Enforce JSON schema

    tool_choice="auto"              # Default behavior

)

# 4. Override at runtime to require specific tool

run_options = {
    "tool_choice": {
        "mode": "required",
        "required_function_name": "get_weather"
    }
}

# 5. Execute

message = Message(role="user", contents=[Content.from_text("Weather in Paris?")])
response = agent.run(message, options=run_options)

# 6. Access parsed output

if response.value:
    report: WeatherReport = response.value
    print(f"{report.location}: {report.temperature_c}°C, {report.condition}")

Execution flow:

1. merge_chat_options combines agent defaults with run_options.
2. validate_tool_mode confirms the required function configuration.
3. The OpenAI client translates the setting into {"type": "function", "name": "get_weather"}.
4. The model is forced to call get_weather, and the final assistant message is parsed into WeatherReport and stored in response.value.

Troubleshooting Common Integration Issues

Symptom	Root Cause	Solution
tool_choice is ignored	Empty tools list passed to agent	Verify at least one FunctionTool or MCPTool is registered.
response.value is None	Model returned plain text or schema mismatch	Simplify the Pydantic schema and explicitly instruct the model to output JSON.
ContentError: Invalid tool choice	Typo in mode string or extraneous dictionary keys	Test values with validate_tool_mode before passing to the agent.
API error: tool_choice not supported	Provider (e.g., Ollama) only supports "auto"	Use "auto" or rely on defaults; unsupported values are stripped automatically.

Summary

• Structured output requires a Pydantic model passed to response_format, with parsed results available in response.value.
• ToolMode configurations (auto, required, none) control tool invocation behavior via the tool_choice option.
• validate_tool_mode in _types.py enforces schema compliance, while merge_chat_options preserves runtime overrides.
• Provider-specific clients (e.g., OpenAI in _chat_client.py) translate generic ToolMode dictionaries into native API payloads.
• Always register at least one tool when using required mode, or the framework will strip the constraint to prevent API errors.

Frequently Asked Questions

What happens if I set tool_choice to "required" but provide no tools?

The provider-specific client detects the empty tools list and removes the tool_choice parameter before sending the request. According to the OpenAI client implementation in python/packages/openai/agent_framework_openai/_chat_client.py, this prevents API validation errors that would otherwise terminate the request.

How does response.value get populated with structured output?

During agent initialization, _prepare_response_and_text_format stores your Pydantic schema. After the LLM returns the final message, the client attempts to parse the JSON content against that schema. If validation succeeds, the framework instantiates the model and assigns it to response.value; otherwise, the value remains None.

Can I use ToolMode configurations with Anthropic or Azure OpenAI?

Yes. The core validation and merging logic in python/packages/core/agent_framework/_types.py is provider-agnostic. Each provider package (e.g., agent_framework_anthropic) contains its own translation layer similar to the OpenAI example, converting the standardized ToolMode dict into provider-specific request parameters.

Why is my tool_choice setting being ignored at runtime?

Check your call to agent.run(). If you pass options that do not explicitly include tool_choice, the agent uses its default configuration. Additionally, if you define agent.defaults with a tool_choice and pass runtime options without one, merge_chat_options retains the agent default rather than removing it. Ensure runtime options explicitly specify the desired mode to guarantee override behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}