# How to Implement Structured Output with ToolMode Configurations in Agent Framework

> Implement structured output in Agent Framework using ToolMode. Force tool calls and enforce JSON schemas for validated Pydantic objects with response_format and response_format.

- Repository: [Microsoft/agent-framework](https://github.com/microsoft/agent-framework)
- Tags: how-to-guide
- Published: 2026-04-05

---

**Agent Framework lets you force specific tool calls while enforcing strict JSON schemas by combining the `tool_choice` option with `response_format`, returning validated Pydantic objects via `response.value`.**

The `microsoft/agent-framework` enables developers to build reliable LLM agents that both invoke specific functions and return structured data. By configuring **ToolMode** options alongside **structured output** schemas, you can constrain model behavior to call required tools and parse responses into typed Python objects.

## Define a Response Schema for Structured Output

**Structured output** ensures the model returns JSON matching a predefined schema rather than free-form text. Define your schema using **Pydantic** and pass it to the agent via the `response_format` parameter.

```python
from pydantic import BaseModel

class WeatherReport(BaseModel):
    location: str
    temperature_c: float
    condition: str

```

When the agent finishes execution, `AgentResponse.value` contains an instance of your model (or `None` if the LLM failed to obey the format). Internally, `Agent._prepare_response_and_text_format` stores this schema in [`python/packages/openai/agent_framework_openai/_chat_client.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/openai/agent_framework_openai/_chat_client.py), where the client later parses the raw JSON into the Pydantic object.

## Configure ToolMode to Control Tool Invocation

**ToolMode** dictates whether and which functions the model must call. The framework supports three modes:

- **`"auto"`**: The model decides when to call a tool.
- **`"required"`**: The model must call a tool; optionally restricted to a specific function via `required_function_name`.
- **`"none"`**: Tool calling is disabled.

Pass these as either a raw string or a dictionary:

```python

# Simple mode

options = {"tool_choice": "auto"}

# Require a specific function

options = {
    "tool_choice": {
        "mode": "required",
        "required_function_name": "get_weather"
    }
}

```

### Validate ToolMode with validate_tool_mode

Before sending requests to the provider, the framework sanitizes inputs using **`validate_tool_mode`** in [`python/packages/core/agent_framework/_types.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/core/agent_framework/_types.py) (lines 3353–3366). This function ensures the dictionary contains valid keys (`mode`, `required_function_name`) and raises `ContentError` for invalid configurations.

### Merge Runtime Options with merge_chat_options

When an agent runs, default options merge with runtime overrides via **`merge_chat_options`** in [`python/packages/core/agent_framework/_types.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/core/agent_framework/_types.py) (lines 3370–3395). This function preserves the higher-priority `tool_choice` from runtime options over agent defaults:

```python
from agent_framework import merge_chat_options

merged = merge_chat_options(agent_defaults, run_options)

# run_options.tool_choice takes precedence

```

## Provider-Specific Payload Translation

Each LLM provider translates the generic `ToolMode` into its native API format. For **OpenAI**, the client in [`python/packages/openai/agent_framework_openai/_chat_client.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/openai/agent_framework_openai/_chat_client.py) (lines 1195–1208) converts the configuration:

```python
tool_mode = validate_tool_mode(tool_choice)
if tool_mode is not None:
    if (mode := tool_mode.get("mode")) == "required" and (
        func_name := tool_mode.get("required_function_name")
    ) is not None:
        run_options["tool_choice"] = {"type": "function", "name": func_name}
    else:
        run_options["tool_choice"] = mode

```

If no tools are registered, the client automatically strips `tool_choice` and `parallel_tool_calls` to avoid API errors.

## Complete Working Example

The following implementation combines a **required tool call** with **structured output parsing**:

```python
from agent_framework import Agent, FunctionTool, Message, Content
from pydantic import BaseModel

# 1. Define structured output schema

class WeatherReport(BaseModel):
    location: str
    temperature_c: float
    condition: str

# 2. Create the tool function

def get_weather(location: str) -> dict:
    """Retrieve weather for a specific location."""
    return {"location": location, "temperature_c": 22.5, "condition": "Sunny"}

weather_tool = FunctionTool(
    name="get_weather",
    description="Get current weather for a location.",
    func=get_weather
)

# 3. Initialize agent with defaults

agent = Agent(
    client="openai",
    model="gpt-4o-mini",
    tools=[weather_tool],
    response_format=WeatherReport,  # Enforce JSON schema

    tool_choice="auto"              # Default behavior

)

# 4. Override at runtime to require specific tool

run_options = {
    "tool_choice": {
        "mode": "required",
        "required_function_name": "get_weather"
    }
}

# 5. Execute

message = Message(role="user", contents=[Content.from_text("Weather in Paris?")])
response = agent.run(message, options=run_options)

# 6. Access parsed output

if response.value:
    report: WeatherReport = response.value
    print(f"{report.location}: {report.temperature_c}°C, {report.condition}")

```

**Execution flow:**
1. `merge_chat_options` combines agent defaults with `run_options`.
2. `validate_tool_mode` confirms the required function configuration.
3. The OpenAI client translates the setting into `{"type": "function", "name": "get_weather"}`.
4. The model is forced to call `get_weather`, and the final assistant message is parsed into `WeatherReport` and stored in `response.value`.

## Troubleshooting Common Integration Issues

| Symptom | Root Cause | Solution |
|---------|------------|----------|
| `tool_choice` is ignored | Empty `tools` list passed to agent | Verify at least one `FunctionTool` or `MCPTool` is registered. |
| `response.value` is `None` | Model returned plain text or schema mismatch | Simplify the Pydantic schema and explicitly instruct the model to output JSON. |
| `ContentError: Invalid tool choice` | Typo in mode string or extraneous dictionary keys | Test values with `validate_tool_mode` before passing to the agent. |
| API error: `tool_choice` not supported | Provider (e.g., Ollama) only supports `"auto"` | Use `"auto"` or rely on defaults; unsupported values are stripped automatically. |

## Summary

- **Structured output** requires a Pydantic model passed to `response_format`, with parsed results available in `response.value`.
- **ToolMode** configurations (`auto`, `required`, `none`) control tool invocation behavior via the `tool_choice` option.
- **`validate_tool_mode`** in [`_types.py`](https://github.com/microsoft/agent-framework/blob/main/_types.py) enforces schema compliance, while **`merge_chat_options`** preserves runtime overrides.
- **Provider-specific clients** (e.g., OpenAI in [`_chat_client.py`](https://github.com/microsoft/agent-framework/blob/main/_chat_client.py)) translate generic ToolMode dictionaries into native API payloads.
- Always register at least one tool when using `required` mode, or the framework will strip the constraint to prevent API errors.

## Frequently Asked Questions

### What happens if I set `tool_choice` to `"required"` but provide no tools?

The provider-specific client detects the empty tools list and removes the `tool_choice` parameter before sending the request. According to the OpenAI client implementation in [`python/packages/openai/agent_framework_openai/_chat_client.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/openai/agent_framework_openai/_chat_client.py), this prevents API validation errors that would otherwise terminate the request.

### How does `response.value` get populated with structured output?

During agent initialization, `_prepare_response_and_text_format` stores your Pydantic schema. After the LLM returns the final message, the client attempts to parse the JSON content against that schema. If validation succeeds, the framework instantiates the model and assigns it to `response.value`; otherwise, the value remains `None`.

### Can I use ToolMode configurations with Anthropic or Azure OpenAI?

Yes. The core validation and merging logic in [`python/packages/core/agent_framework/_types.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/core/agent_framework/_types.py) is provider-agnostic. Each provider package (e.g., `agent_framework_anthropic`) contains its own translation layer similar to the OpenAI example, converting the standardized `ToolMode` dict into provider-specific request parameters.

### Why is my `tool_choice` setting being ignored at runtime?

Check your call to `agent.run()`. If you pass `options` that do not explicitly include `tool_choice`, the agent uses its default configuration. Additionally, if you define `agent.defaults` with a `tool_choice` and pass runtime options without one, `merge_chat_options` retains the agent default rather than removing it. Ensure runtime options explicitly specify the desired mode to guarantee override behavior.