LiteLLM Tool Calling and Function Calling: Universal Capability Detection Across Providers

LiteLLM unifies function calling support across all providers through a centralized detection system that checks model metadata before transforming requests, automatically stripping unsupported tool parameters to prevent API errors.

LiteLLM enables tool calling and function calling for over 100 LLM providers through a sophisticated capability detection layer. According to the BerriAI/litellm source code, the library maintains a unified abstraction that automatically determines whether a specific model supports tool use, then conditionally modifies request payloads to match the provider's actual capabilities.

How LiteLLM Detects Function Calling Capabilities

LiteLLM centralizes capability detection in litellm/utils.py through three public helper functions that query a comprehensive model metadata catalog.

Core Detection Helpers

The detection logic resides in lines 2496-2524 of litellm/utils.py:

• supports_function_calling (lines 2496-2516): Returns True if the model advertises function-calling support
• supports_parallel_function_calling (lines 2483-2492): Detects parallel tool call capabilities
• supports_tool_choice (lines 2519-2524): Checks if the model accepts the tool_choice argument

All three helpers delegate to a private _supports_factory that normalizes model names and checks the model-info catalog before falling back to provider-specific metadata.

The Detection Algorithm

The _supports_factory executes three steps:

1. Normalizes the model name via litellm.get_llm_provider, handling aliases like openai/gpt-4o
2. Queries the model entry in model_prices_and_context_window.json for capability flags
3. Falls back to provider metadata (get_provider_info) if the catalog entry lacks the specific flag

If neither source contains the capability flag, the function returns False and logs a debug message for troubleshooting.

Provider-Specific Request Transformations

Once capabilities are detected, LiteLLM applies provider-specific transformations to prevent unsupported parameters from reaching remote APIs.

Dynamic Parameter Stripping

In litellm/llms/openai_like/dynamic_config.py (lines 100-124), the system removes tool-related parameters for models that lack function calling support:

if not _supports_fc:
    for param in ["tools", "tool_choice", "function_call", "functions", "parallel_tool_calls"]:
        if param in supported_params:
            supported_params.remove(param)

This ensures that requests to older models (like anthropic.claude-2) never include incompatible tools arrays that would trigger provider errors.

Conditional Parameter Addition

Conversely, providers like Fireworks AI explicitly add tool support only when flags are positive. In litellm/llms/fireworks_ai/chat/transformation.py (lines 112-118):

if supports_function_calling(model=model, custom_llm_provider="fireworks_ai"):
    supported_params.append("tools")
if supports_tool_choice(model=model, custom_llm_provider="fireworks_ai"):
    supported_params.append("tool_choice")

The Model Information Catalog

All capability metadata lives in model_prices_and_context_window.json, a centralized JSON file that maps model identifiers to their supported features.

Each entry defines boolean flags for tool support:

{
  "gpt-4o": {
    "supports_function_calling": true,
    "supports_tool_choice": true,
    "supports_parallel_function_calling": true
  },
  "claude-3-sonnet-20240229": {
    "supports_function_calling": true,
    "supports_tool_choice": true
  }
}

When providers release new models, maintainers update this catalog without modifying transformation logic, enabling immediate capability detection across the entire LiteLLM ecosystem.

End-to-End Function Calling Flow

The complete request pipeline operates as follows:

1. User invocation: Code calls litellm.completion() with a tools parameter
2. Provider resolution: litellm/router.py resolves the provider and canonical model name via get_llm_provider
3. Capability check: supports_function_calling queries the model-info catalog
4. Payload transformation: Provider-specific classes either add or strip tool parameters based on the capability flag
5. Request transmission: The sanitized payload reaches the remote API
6. Response normalization: Tool call responses are post-processed uniformly regardless of the original provider

This architecture guarantees that function calling works for any model advertising the capability while preventing accidental misuse on unsupported models.

Practical Implementation Examples

Example 1: Basic Function Calling with OpenAI

import litellm

response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get current weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
)

Behind the scenes, LiteLLM confirms supports_function_calling returns True for gpt-4o-mini and forwards the tools array to OpenAI's /chat/completions endpoint.

Example 2: Automatic Parameter Stripping for Unsupported Models

import litellm

# Claude 3 Opus supports tools - parameter is sent

resp = litellm.completion(
    model="anthropic.claude-3-opus-20240229",
    messages=[{"role": "user", "content": "Summarize the text."}],
    tools=[{"type": "function", "function": {"name": "summarize"}}]
)

# Claude 2 does NOT support tools - parameter is automatically removed

resp2 = litellm.completion(
    model="anthropic.claude-2",
    messages=[{"role": "user", "content": "Summarize the text."}],
    tools=[{"type": "function", "function": {"name": "summarize"}}]
)

The second request succeeds because LiteLLM strips the tools key before transmission, preventing the Bedrock API from returning a validation error.

Example 3: Programmatic Capability Checking

from litellm.utils import (
    supports_function_calling,
    supports_tool_choice,
    supports_parallel_function_calling,
)

model = "fireworks_ai/qwen2.5-72b-instruct"
print("Function calling:", supports_function_calling(model, "fireworks_ai"))
print("Tool choice:", supports_tool_choice(model, "fireworks_ai"))
print("Parallel calls:", supports_parallel_function_calling(model, "fireworks_ai"))

Use this pattern in UI layers to conditionally display tool configuration options based on the selected model's actual capabilities.

Summary

• Unified detection lives in litellm/utils.py through supports_function_calling, supports_tool_choice, and supports_parallel_function_calling
• Model metadata is centralized in model_prices_and_context_window.json, enabling updates without code changes
• Automatic sanitization occurs in provider transformation classes (e.g., litellm/llms/openai_like/dynamic_config.py) that strip unsupported parameters before API transmission
• Provider flexibility allows Fireworks AI and other providers to conditionally add tool support only when capability flags are positive
• Zero-config operation means function calling works automatically for supported models while preventing errors on legacy models

Frequently Asked Questions

How does LiteLLM determine if a specific model supports function calling?

LiteLLM checks the supports_function_calling helper in litellm/utils.py (lines 2496-2516), which queries the model's entry in model_prices_and_context_window.json for the capability flag. If the flag is missing, it falls back to provider-specific metadata via get_provider_info before defaulting to False.

What happens if I send tool parameters to a model that doesn't support them?

LiteLLM automatically strips unsupported tool parameters. In litellm/llms/openai_like/dynamic_config.py (lines 100-124), the transformation layer removes tools, tool_choice, function_call, functions, and parallel_tool_calls from the request payload when supports_function_calling returns False, preventing API errors while allowing the request to proceed.

Where is the capability data stored in LiteLLM?

All model capability flags reside in model_prices_and_context_window.json in the repository root. This JSON file contains boolean fields like supports_function_calling, supports_tool_choice, and supports_parallel_function_calling for each model identifier, serving as the single source of truth for the entire detection system.

Can I check function calling support programmatically before making a request?

Yes. Import the detection helpers from litellm.utils and call them with your model string and provider. This returns a boolean immediately without making a network request, allowing your application to adjust UI elements or routing logic before invoking litellm.completion().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}