Implementing Function Calling for LLMs: A Complete Technical Guide

Function calling enables LLMs to request external code execution by emitting structured JSON tool calls that the runtime deserializes, executes, and injects back into the conversation context until a final answer is generated.

The rohitg00/ai-engineering-from-scratch repository provides a production-grade curriculum for implementing function calling for LLMs, transforming static language models into autonomous agents capable of interacting with external APIs, databases, and search engines. This mechanism relies on a strict JSON-Schema contract between the model and executable Python functions, implemented through a reusable runtime loop.

The Architecture of Function Calling

The repository structures function calling across five conceptual layers, each handling a specific concern from schema definition to production deployment.

Layer	Description	Repository Location
Tool Interface	Defines the JSON-Schema contract (name, parameters, description) that governs how the LLM structures its requests.	phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md
Provider Adapters	Translates generic tool contracts into provider-specific payloads for OpenAI, Anthropic, or Gemini.	phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md
Execution Loop	The runtime driver that detects tool calls, dispatches to Python functions, and manages conversation state.	phases/11-llm-engineering/09-function-calling/code/function_calling.py
Parallel & Streaming	Support for issuing multiple tool calls in a single model turn and streaming results as they arrive.	phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md
Production Patterns	Decision frameworks, error handling, security guards, and logging conventions for real-world deployments.	phases/11-llm-engineering/09-function-calling/outputs/skill-function-calling-patterns.md

The Tool Interface

At the core of implementing function calling for LLMs lies the tool interface—a JSON-Schema contract that defines each available function. This schema specifies the function name, description, and parameter properties with types and constraints. According to the curriculum in phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md, the LLM emits a structured call that conforms strictly to this schema, enabling type-safe deserialization and execution.

Provider-Specific Adapters

While the conceptual loop remains identical across providers, the API payload structures differ significantly:

• OpenAI: Uses a tool_calls array where each object contains name and arguments fields. Parallel execution is enabled by default but can be disabled with parallel_tool_calls: false. Strict-mode validation is enforced via the strict flag combined with response_format: { type: "json_object" }.
• Anthropic: Implements tool_use objects, typically limited to one per turn, requiring sequential rather than parallel execution. Schema validation is built into the API.
• Gemini: Employs functionDeclarations paired with functionCall objects and supports id-correlated parallel calls. Validation is enforced through the responseSchema parameter.

These nuances are documented in phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md.

The Function-Calling Loop Implementation

The function_calling.py file in phases/11-llm-engineering/09-function-calling/code/ implements the six-step execution loop that drives tool interactions:

1. Prompt Assembly: The system prompt registers the available tool catalog (JSON-Schema list) and instructs the model to emit structured calls when needed.
2. Model Invocation: The LLM returns either a text response or a tool_calls block containing the function name and serialized arguments.
3. Dispatch: The runtime matches the function name to its Python implementation (e.g., web_search, calculate_sum).
4. Execution: The function runs in a sandboxed environment; exceptions are caught and converted to structured error objects.
5. Result Injection: The function output (or error) is formatted as a new message appended to the conversation history.
6. Final Completion: The model receives the enriched context and generates the user-facing answer.

This loop transforms the LLM from a static knowledge base into an agent capable of acting on the external world while retaining core reasoning capabilities inside the model.

Practical Python Implementation

The curriculum provides concrete patterns for implementing function calling for LLMs using the FunctionCallingLoop class.

Define a tool using standard JSON-Schema syntax:

weather_tool = {
    "name": "get_current_weather",
    "description": "Retrieve the current temperature for a city.",
    "parameters": {
        "type": "object",
        "properties": {
            "city": {"type": "string", "description": "Name of the city"}
        },
        "required": ["city"]
    },
}

Register the tool with the runtime and execute a conversation turn:

from function_calling import FunctionCallingLoop

loop = FunctionCallingLoop(tools=[weather_tool])

user_msg = "What's the temperature in Paris right now?"
final_answer = loop.run(user_msg)
print(final_answer)   # → "It’s currently 12 °C in Paris."

For OpenAI-style parallel execution, configure the loop with multiple tools:

parallel_tools = [
    {
        "name": "stock_price",
        "description": "Fetch the latest price for a ticker symbol.",
        "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]},
    },
    {
        "name": "currency_rate",
        "description": "Get USD‑to‑EUR exchange rate.",
        "parameters": {"type": "object", "properties": {}, "required": []},
    },
]

loop = FunctionCallingLoop(tools=parallel_tools, allow_parallel=True)
answer = loop.run("Give me AAPL stock price and the current EUR‑USD rate.")

Advanced Patterns: Parallelism, Streaming, and Production Safety

As detailed in phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md, advanced implementations support issuing several tool calls in a single model turn and streaming results back as they arrive.

For production deployments, the skill-function-calling-patterns.md document establishes critical guardrails:

• Retry on Transient Failures: Wrap external calls in exponential back-off logic.
• Timeout Guardrails: Abort tool execution after a configurable deadline to prevent hangs.
• Infinite-Loop Detection: Limit consecutive tool calls to a maximum (e.g., 5) per turn and surface errors if exceeded.
• Security Checks: Sanitize arguments, enforce a whitelist of permissible functions, and log every invocation for auditability.

Testing and Validation Strategies

The repository emphasizes a three-tier testing strategy for function calling implementations:

• Unit Tests: Validate individual tool functions in isolation (e.g., tests/test_<tool>.py).
• Integration Tests: Simulate full chat turns, asserting that model outputs containing tool calls trigger correct function execution and produce valid final answers.
• Property-Based Tests: Verify JSON-Schema compliance between the tool definition and the deserialized arguments passed to Python functions.

Capstone Integration: Building Production-Grade LLM Services

The final lesson in phases/11-llm-engineering/13-production-app/docs/en.md demonstrates how to wire function calling into a complete system alongside retrieval-augmented generation (RAG), response caching, input guardrails, and observability tooling. This capstone project shows how the function-calling loop serves as the orchestration backbone for real-world LLM services that combine external data retrieval with generative capabilities.

Summary

• Implementing function calling for LLMs requires a strict JSON-Schema tool interface that defines available functions and their parameters.
• The execution loop involves six discrete steps: prompt assembly, model invocation, dispatch, execution, result injection, and final completion.
• Provider implementations differ in payload structure—OpenAI uses tool_calls arrays with native parallel support, Anthropic uses sequential tool_use objects, and Gemini uses functionDeclarations with id-correlated parallelism.
• Production safety requires infinite-loop detection (limiting consecutive calls to approximately 5), timeout guardrails, argument sanitization, and comprehensive audit logging.
• The FunctionCallingLoop class in phases/11-llm-engineering/09-function-calling/code/function_calling.py provides a reusable Python implementation of these patterns.

Frequently Asked Questions

What is the difference between function calling and tool use across LLM providers?

OpenAI emits calls as a tool_calls array supporting parallel execution by default, Anthropic structures them as tool_use objects typically limited to one per turn, and Gemini uses functionDeclarations paired with functionCall objects that support id-correlated parallel calls. While the payload shapes differ, the underlying six-step execution loop remains identical across all providers according to the source implementation.

How do you prevent infinite loops when implementing function calling?

The curriculum recommends implementing an infinite-loop detection mechanism that limits the maximum number of consecutive tool calls per turn to approximately five. If this threshold is exceeded, the system should surface a friendly error message and halt execution. Additionally, timeout guardrails should abort long-running external calls, and retry logic should use exponential back-off for transient failures only.

Can LLMs execute multiple function calls simultaneously?

Parallel tool calls are supported natively by OpenAI (enabled by default, disabled via parallel_tool_calls: false) and Gemini (using id-correlated calls), allowing a single model turn to request multiple independent function executions. Anthropic's API requires sequential tool_use calls, necessitating manual orchestration if parallel execution is required. The FunctionCallingLoop class supports this through its allow_parallel=True configuration parameter.

What testing approach validates that function calling works end-to-end?

The recommended strategy includes integration tests that simulate complete chat turns, asserting that model outputs containing tool calls lead to correct function execution and valid final answers. Additionally, property-based tests verify that deserialized arguments comply with the JSON-Schema defined in the tool interface, while unit tests isolate individual tool function logic in files like tests/test_<tool>.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}