When to Use Function Calling vs Structured Outputs vs JSON Mode for LLMs

Use function calling when you need provider-enforced schema validation for a fixed toolbox, structured outputs when you require custom validation across diverse providers, and JSON mode for quick prototypes with single payloads.

Large language models offer multiple ways to produce machine-readable output, and selecting the right method determines your system's reliability and portability. This guide draws from the humanlayer/12-factor-agents repository to explain when to use function calling vs structured outputs vs JSON mode for LLMs, with concrete implementation patterns from production agent architectures.

The Three Output Patterns

Function Calling (Native Tool Support)

Function calling APIs—such as OpenAI's function_call parameter—provide a structured way for models to invoke deterministic actions. In this pattern, you define a list of function specifications with JSON schemas describing parameters. When the model decides a tool is needed, it returns a function_call object containing the function name and arguments that exactly match one of the declared schemas.

This approach excels when you have a fixed set of deterministic actions like payment creation or issue tracking. The primary advantage is strong validation from the provider: OpenAI will reject or constrain calls that don't match the defined schema, reducing malformed output errors.

According to content/factor-01-natural-language-to-tool-calls.md, this represents the foundational pattern of converting natural language into structured objects that deterministic functions can execute.

Structured Outputs (Schema-First Validation)

Structured outputs shift validation responsibility to your application layer. Instead of relying on the provider's native function support, you prompt the model to emit JSON that conforms to a schema you enforce—typically using Pydantic models or TypeScript types. Your code parses the result and dispatches the appropriate logic.

This method is optimal when your toolset is dynamic or extensible, when you need rich type-checking beyond provider offerings, or when building portable agents that work across providers like Anthropic or Llama-Index without vendor-specific APIs.

The content/factor-04-tools-are-structured-outputs.md file in the humanlayer/12-factor-agents repository emphasizes that "tools are just structured outputs," advocating for this approach when you cannot rely on a provider's built-in function-call validation. This file links to external comparisons reinforcing that structured outputs offer maximum flexibility for complex agent architectures.

JSON Mode (Raw JSON Enforcement)

JSON mode—enabled via flags like OpenAI's response_format: {type: "json_object"}—forces the model to return raw JSON without the function_call wrapper. This creates a single top-level JSON object suitable for simple payloads or configuration objects.

Use JSON mode for quick prototyping or constrained environments where defining formal functions is overkill. However, you retain full responsibility for parsing and validation, and the single-object limitation can become cumbersome for workflows requiring multiple distinct actions.

Implementation Examples

Function Calling with OpenAI SDK

The following example from the repository demonstrates defining a payment creation tool with strict schema enforcement:

import openai
import json

functions = [
    {
        "name": "create_payment_link",
        "description": "Create a Stripe payment link",
        "parameters": {
            "type": "object",
            "properties": {
                "amount": {"type": "integer"},
                "customer": {"type": "string"},
                "product": {"type": "string"},
                "price": {"type": "string"},
                "quantity": {"type": "integer"},
                "memo": {"type": "string"},
            },
            "required": ["amount", "customer", "product", "price"],
        },
    }
]

response = openai.ChatCompletion.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Create a payment link for $750 to Terri"}],
    functions=functions,
    function_call="auto",
)

call = response["choices"][0]["message"]["function_call"]
payload = json.loads(call["arguments"])

# deterministic code runs here

stripe.paymentlinks.create(**payload)

Structured Outputs with Pydantic Validation

For scenarios requiring custom validation across providers, implement Pydantic models as shown in packages/create-12-factor-agent/template/README.md patterns:

from pydantic import BaseModel, ValidationError
import json

class Issue(BaseModel):
    title: str
    description: str
    team_id: str
    assignee_id: str

class CreateIssue(BaseModel):
    intent: str = "create_issue"
    issue: Issue

def parse_tool_output(text: str) -> CreateIssue:
    data = json.loads(text)
    return CreateIssue(**data)

# LLM is prompted to output JSON matching CreateIssue

raw_output = llm.generate("""\
Return JSON for creating an issue about "Fix login bug".
""")
try:
    tool_call = parse_tool_output(raw_output)
    # deterministic handling

    github.create_issue(**tool_call.issue.dict())
except ValidationError as e:
    # fallback handling

    logger.error(e)

JSON Mode for Simple Payloads

When you need a single payload without function overhead, use JSON mode:

response = openai.ChatCompletion.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize the meeting notes"}],
    response_format={"type": "json_object"},
)

summary = json.loads(response["choices"][0]["message"]["content"])
print(summary["title"])
print(summary["highlights"])

Summary

• Function calling is ideal when you have a small, stable toolbox and want the LLM provider to enforce schemas automatically via the native function_call wrapper, as demonstrated in content/factor-01-natural-language-to-tool-calls.md.
• Structured outputs provide maximum portability and custom validation, making them suitable for dynamic toolsets or multi-provider deployments where you control parsing with libraries like Pydantic, aligning with the principles in content/factor-04-tools-are-structured-outputs.md.
• JSON mode works best for single-payload scenarios or rapid prototyping where defining formal functions is unnecessary, though you retain responsibility for validation and error handling.

Frequently Asked Questions

When should I choose function calling over structured outputs?

Function calling is preferable when you have a fixed, stable set of actions and want the provider to handle schema validation automatically. According to the humanlayer/12-factor-agents methodology, this approach reduces boilerplate validation code but limits you to provider-supported platforms. Use structured outputs when you need to support diverse providers or require validation logic that exceeds native capabilities.

Can I combine function calling and structured outputs in the same agent architecture?

Yes. Production agents often use function calling for the initial intent classification—selecting which tool to invoke—then use structured outputs for parameter extraction within that tool's execution path. The packages/create-12-factor-agent/template/README.md scaffold suggests this hybrid approach for complex workflows requiring both provider validation and custom type checking.

Is JSON mode less reliable than function calling or structured outputs?

JSON mode offers the same underlying model capabilities but provides no schema enforcement from the provider. While function calling rejects malformed schemas at the API level, and structured outputs allow custom validation with Pydantic, JSON mode returns raw JSON that may be malformed. You must implement robust error handling and retry logic when using JSON mode for production workloads.

How does the 12-Factor Agents methodology recommend handling validation?

The repository emphasizes that tools are fundamentally structured outputs, advocating for validation at the application layer when using structured outputs or JSON mode. As noted in content/factor-04-tools-are-structured-outputs.md, treating tool calls as plain JSON objects with Pydantic validation gives you portability across LLM providers and protects against provider-specific API changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}