# Implementing Function Calling for LLMs: A Complete Technical Guide

> Master LLM function calling by learning to emit structured JSON tool calls for external code execution. Get a complete technical guide to inject results back into conversations.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: how-to-guide
- Published: 2026-06-15

---

**Function calling enables LLMs to request external code execution by emitting structured JSON tool calls that the runtime deserializes, executes, and injects back into the conversation context until a final answer is generated.**

The `rohitg00/ai-engineering-from-scratch` repository provides a production-grade curriculum for implementing function calling for LLMs, transforming static language models into autonomous agents capable of interacting with external APIs, databases, and search engines. This mechanism relies on a strict **JSON-Schema contract** between the model and executable Python functions, implemented through a reusable runtime loop.

## The Architecture of Function Calling

The repository structures function calling across five conceptual layers, each handling a specific concern from schema definition to production deployment.

| Layer | Description | Repository Location |
|-------|-------------|----------------------|
| **Tool Interface** | Defines the JSON-Schema contract (name, parameters, description) that governs how the LLM structures its requests. | [`phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md) |
| **Provider Adapters** | Translates generic tool contracts into provider-specific payloads for OpenAI, Anthropic, or Gemini. | [`phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md) |
| **Execution Loop** | The runtime driver that detects tool calls, dispatches to Python functions, and manages conversation state. | [`phases/11-llm-engineering/09-function-calling/code/function_calling.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/09-function-calling/code/function_calling.py) |
| **Parallel & Streaming** | Support for issuing multiple tool calls in a single model turn and streaming results as they arrive. | [`phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md) |
| **Production Patterns** | Decision frameworks, error handling, security guards, and logging conventions for real-world deployments. | [`phases/11-llm-engineering/09-function-calling/outputs/skill-function-calling-patterns.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/09-function-calling/outputs/skill-function-calling-patterns.md) |

### The Tool Interface

At the core of implementing function calling for LLMs lies the **tool interface**—a JSON-Schema contract that defines each available function. This schema specifies the function name, description, and parameter properties with types and constraints. According to the curriculum in [`phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/01-the-tool-interface/docs/en.md), the LLM emits a structured call that conforms strictly to this schema, enabling type-safe deserialization and execution.

### Provider-Specific Adapters

While the conceptual loop remains identical across providers, the API payload structures differ significantly:

- **OpenAI**: Uses a `tool_calls` array where each object contains `name` and `arguments` fields. Parallel execution is enabled by default but can be disabled with `parallel_tool_calls: false`. Strict-mode validation is enforced via the `strict` flag combined with `response_format: { type: "json_object" }`.
- **Anthropic**: Implements `tool_use` objects, typically limited to one per turn, requiring sequential rather than parallel execution. Schema validation is built into the API.
- **Gemini**: Employs `functionDeclarations` paired with `functionCall` objects and supports *id-correlated* parallel calls. Validation is enforced through the `responseSchema` parameter.

These nuances are documented in [`phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/02-function-calling-deep-dive/docs/en.md).

## The Function-Calling Loop Implementation

The [`function_calling.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/function_calling.py) file in `phases/11-llm-engineering/09-function-calling/code/` implements the six-step execution loop that drives tool interactions:

1. **Prompt Assembly**: The system prompt registers the available tool catalog (JSON-Schema list) and instructs the model to emit structured calls when needed.
2. **Model Invocation**: The LLM returns either a text response or a `tool_calls` block containing the function name and serialized arguments.
3. **Dispatch**: The runtime matches the function name to its Python implementation (e.g., `web_search`, `calculate_sum`).
4. **Execution**: The function runs in a sandboxed environment; exceptions are caught and converted to structured error objects.
5. **Result Injection**: The function output (or error) is formatted as a new message appended to the conversation history.
6. **Final Completion**: The model receives the enriched context and generates the user-facing answer.

This loop transforms the LLM from a static knowledge base into an agent capable of acting on the external world while retaining core reasoning capabilities inside the model.

## Practical Python Implementation

The curriculum provides concrete patterns for implementing function calling for LLMs using the `FunctionCallingLoop` class.

Define a tool using standard JSON-Schema syntax:

```python
weather_tool = {
    "name": "get_current_weather",
    "description": "Retrieve the current temperature for a city.",
    "parameters": {
        "type": "object",
        "properties": {
            "city": {"type": "string", "description": "Name of the city"}
        },
        "required": ["city"]
    },
}

```

Register the tool with the runtime and execute a conversation turn:

```python
from function_calling import FunctionCallingLoop

loop = FunctionCallingLoop(tools=[weather_tool])

user_msg = "What's the temperature in Paris right now?"
final_answer = loop.run(user_msg)
print(final_answer)   # → "It’s currently 12 °C in Paris."

```

For OpenAI-style parallel execution, configure the loop with multiple tools:

```python
parallel_tools = [
    {
        "name": "stock_price",
        "description": "Fetch the latest price for a ticker symbol.",
        "parameters": {"type": "object", "properties": {"symbol": {"type": "string"}}, "required": ["symbol"]},
    },
    {
        "name": "currency_rate",
        "description": "Get USD‑to‑EUR exchange rate.",
        "parameters": {"type": "object", "properties": {}, "required": []},
    },
]

loop = FunctionCallingLoop(tools=parallel_tools, allow_parallel=True)
answer = loop.run("Give me AAPL stock price and the current EUR‑USD rate.")

```

## Advanced Patterns: Parallelism, Streaming, and Production Safety

As detailed in [`phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/13-tools-and-protocols/03-parallel-and-streaming-tool-calls/docs/en.md), advanced implementations support issuing several tool calls in a single model turn and streaming results back as they arrive.

For production deployments, the [`skill-function-calling-patterns.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/skill-function-calling-patterns.md) document establishes critical guardrails:

- **Retry on Transient Failures**: Wrap external calls in exponential back-off logic.
- **Timeout Guardrails**: Abort tool execution after a configurable deadline to prevent hangs.
- **Infinite-Loop Detection**: Limit consecutive tool calls to a maximum (e.g., 5) per turn and surface errors if exceeded.
- **Security Checks**: Sanitize arguments, enforce a whitelist of permissible functions, and log every invocation for auditability.

## Testing and Validation Strategies

The repository emphasizes a three-tier testing strategy for function calling implementations:

- **Unit Tests**: Validate individual tool functions in isolation (e.g., `tests/test_<tool>.py`).
- **Integration Tests**: Simulate full chat turns, asserting that model outputs containing tool calls trigger correct function execution and produce valid final answers.
- **Property-Based Tests**: Verify JSON-Schema compliance between the tool definition and the deserialized arguments passed to Python functions.

## Capstone Integration: Building Production-Grade LLM Services

The final lesson in [`phases/11-llm-engineering/13-production-app/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/13-production-app/docs/en.md) demonstrates how to wire function calling into a complete system alongside **retrieval-augmented generation (RAG)**, response caching, input guardrails, and observability tooling. This capstone project shows how the function-calling loop serves as the orchestration backbone for real-world LLM services that combine external data retrieval with generative capabilities.

## Summary

- **Implementing function calling for LLMs** requires a strict JSON-Schema tool interface that defines available functions and their parameters.
- The execution loop involves six discrete steps: prompt assembly, model invocation, dispatch, execution, result injection, and final completion.
- Provider implementations differ in payload structure—OpenAI uses `tool_calls` arrays with native parallel support, Anthropic uses sequential `tool_use` objects, and Gemini uses `functionDeclarations` with id-correlated parallelism.
- Production safety requires infinite-loop detection (limiting consecutive calls to approximately 5), timeout guardrails, argument sanitization, and comprehensive audit logging.
- The `FunctionCallingLoop` class in [`phases/11-llm-engineering/09-function-calling/code/function_calling.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/11-llm-engineering/09-function-calling/code/function_calling.py) provides a reusable Python implementation of these patterns.

## Frequently Asked Questions

### What is the difference between function calling and tool use across LLM providers?

OpenAI emits calls as a `tool_calls` array supporting parallel execution by default, Anthropic structures them as `tool_use` objects typically limited to one per turn, and Gemini uses `functionDeclarations` paired with `functionCall` objects that support id-correlated parallel calls. While the payload shapes differ, the underlying six-step execution loop remains identical across all providers according to the source implementation.

### How do you prevent infinite loops when implementing function calling?

The curriculum recommends implementing an **infinite-loop detection** mechanism that limits the maximum number of consecutive tool calls per turn to approximately five. If this threshold is exceeded, the system should surface a friendly error message and halt execution. Additionally, timeout guardrails should abort long-running external calls, and retry logic should use exponential back-off for transient failures only.

### Can LLMs execute multiple function calls simultaneously?

**Parallel tool calls** are supported natively by OpenAI (enabled by default, disabled via `parallel_tool_calls: false`) and Gemini (using id-correlated calls), allowing a single model turn to request multiple independent function executions. Anthropic's API requires sequential `tool_use` calls, necessitating manual orchestration if parallel execution is required. The `FunctionCallingLoop` class supports this through its `allow_parallel=True` configuration parameter.

### What testing approach validates that function calling works end-to-end?

The recommended strategy includes **integration tests** that simulate complete chat turns, asserting that model outputs containing tool calls lead to correct function execution and valid final answers. Additionally, **property-based tests** verify that deserialized arguments comply with the JSON-Schema defined in the tool interface, while unit tests isolate individual tool function logic in files like `tests/test_<tool>.py`.