# How to Implement Function Calling with Azure OpenAI and External Tools: A Complete Guide

> Learn to implement function calling with Azure OpenAI. Connect models to external tools using JSON schemas for dynamic, natural language responses. Get started today!

- Repository: [Microsoft/generative-ai-for-beginners](https://github.com/microsoft/generative-ai-for-beginners)
- Tags: how-to-guide
- Published: 2026-02-26

---

**Function calling with Azure OpenAI allows models to request external code execution by defining JSON schemas, detecting `function_call` responses, and feeding results back into the chat history for natural language replies.**

If you are building applications with the `microsoft/generative-ai-for-beginners` curriculum, learning how to implement function calling with Azure OpenAI and external tools unlocks the ability to connect LLMs to live APIs, databases, and custom business logic. This pattern transforms a static chatbot into a dynamic agent that can fetch real-time data and take action on behalf of users.

## Understanding the Function Calling Architecture

The implementation follows a strict three-stage pattern that separates intent detection from execution.

### The Three-Stage Pattern

According to the source code in [`11-integrating-with-function-calling/README.md`](https://github.com/microsoft/generative-ai-for-beginners/blob/main/11-integrating-with-function-calling/README.md), the workflow consists of:

1. **Define the function schema** – Provide the model with a JSON description of the function name, purpose, and required parameters. The schema is sent in the `functions` field of the `ChatCompletion` request.
2. **Detect and execute the call** – After the model responds, inspect `response_message.function_call`. If it contains a `name`, map that name to a real Python callable, deserialize the `arguments`, invoke the function (e.g., an HTTP request to the Microsoft Learn Catalog API), and capture its output.
3. **Feed the result back** – Append two new messages to the chat history: one representing the model’s function-call response (role = `assistant`, `function_call` payload) and one containing the actual function output (role = `function`). Then issue a second `ChatCompletion` call so the model can generate a natural-language reply that incorporates the fetched data.

### Why Function Calling Works

This pattern succeeds because it combines **structured output** with **external data retrieval**. The model is forced to return arguments that match the JSON schema, eliminating ambiguous free-form text. By delegating to external APIs (such as the Learn Catalog in the reference implementation), you provide up-to-date information that the model itself does not possess, while maintaining a reusable framework for adding new tools.

## Setting Up Your Azure OpenAI Client

Before implementing function calling, you must configure the client with your Azure-specific credentials. The lesson code in [`11-integrating-with-function-calling/README.md`](https://github.com/microsoft/generative-ai-for-beginners/blob/main/11-integrating-with-function-calling/README.md) demonstrates this setup:

```python
import os, json
from openai import AzureOpenAI
from dotenv import load_dotenv

load_dotenv()
client = AzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    api_version="2023-07-01-preview",
)
deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT")

```

Ensure your `.env` file (copied from the root `.env.copy` template) contains `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, and `AZURE_OPENAI_DEPLOYMENT`.

## Defining Function Schemas for External Tools

The critical step in implementing function calling with Azure OpenAI is describing your external tools using a strict JSON schema. This example from the `microsoft/generative-ai-for-beginners` repository defines a function that queries the Microsoft Learn Catalog API:

```python
functions = [
    {
        "name": "search_courses",
        "description": "Retrieves courses from the search index based on the parameters provided",
        "parameters": {
            "type": "object",
            "properties": {
                "role": {"type": "string", "description": "Learner role (developer, student, …)"},
                "product": {"type": "string", "description": "Product of interest (Azure, Power BI, …)"},
                "level": {"type": "string", "description": "Experience level (beginner, intermediate, advanced)"}
            },
            "required": ["role"]
        },
    }
]

```

The `name` field must match the Python function you will map later. The `description` helps the model understand when to invoke the tool, while `parameters` enforces type safety using JSON Schema syntax.

## Detecting and Executing Function Calls

Once you send the initial request with `function_call="auto"`, you must inspect the response to determine if the model requested a tool execution:

```python
messages = [{"role": "user", "content": "Find me a good course for a beginner student to learn Azure."}]
response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    functions=functions,
    function_call="auto"
)

response_message = response.choices[0].message

if getattr(response_message, "function_call", None) and response_message.function_call.name:
    # Map the name to a real Python function

    def search_courses(role, product=None, level=None):
        import requests
        url = "https://learn.microsoft.com/api/catalog/"
        params = {"role": role, "product": product, "level": level}
        r = requests.get(url, params=params)
        modules = r.json()["modules"]
        return str([{"title": m["title"], "url": m["url"]} for m in modules[:5]])

    available_functions = {"search_courses": search_courses}
    fn_name = response_message.function_call.name
    fn = available_functions[fn_name]
    fn_args = json.loads(response_message.function_call.arguments)
    fn_result = fn(**fn_args)

```

This pattern uses `getattr` to safely check for the `function_call` attribute, then maps the string name to an actual Python callable using a dictionary lookup. The `arguments` are parsed from JSON string to dictionary using `json.loads`.

## Feeding Results Back to the Model

To complete the loop, you must append both the model's function request and the actual function output to the message history before requesting a final natural-language response:

```python

# Add the model's function-call message

messages.append({
    "role": response_message.role,
    "function_call": {
        "name": fn_name,
        "arguments": response_message.function_call.arguments,
    },
    "content": None,
})

# Add the function's output

messages.append({
    "role": "function",
    "name": fn_name,
    "content": fn_result,
})

second_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    functions=functions,
    function_call="auto",
    temperature=0,
)

print(second_response.choices[0].message.content)

```

The message with `role="function"` contains the raw data returned by your external tool, while the `assistant` message with the `function_call` object preserves the context of what the model intended to do. Setting `temperature=0` ensures deterministic output when synthesizing the final answer.

## Summary

Implementing function calling with Azure OpenAI and external tools requires a structured three-stage pattern that separates intent detection from execution:

- **Define schemas using JSON** to describe available functions, their parameters, and descriptions in the `functions` array passed to `chat.completions.create`.
- **Detect calls by inspecting** `response_message.function_call` after setting `function_call="auto"`, then map the function name to a real Python callable and deserialize arguments with `json.loads`.
- **Feed results back** by appending both the assistant's function request and a `role="function"` message containing the tool output, then issue a second completion call to generate the natural language response.

This pattern, as demonstrated in the `microsoft/generative-ai-for-beginners` repository, enables LLMs to interact with live APIs, databases, and custom business logic while maintaining type safety and conversational context.

## Frequently Asked Questions

### What is the difference between function calling and fine-tuning in Azure OpenAI?

Function calling is a runtime capability that allows the model to output structured JSON matching a predefined schema, which your application then executes. Fine-tuning is a training process that modifies the model's weights to improve performance on specific tasks or domains. You use function calling when you need the model to interact with external APIs or tools; you use fine-tuning when you need to change how the model generates text or responds to specific prompts.

### How do I handle errors when an external API call fails during function execution?

When implementing the function execution step, wrap your external API calls in try-except blocks. If the call fails, return a descriptive error message as the function result content. When you feed this back to the model via the `role="function"` message, the LLM can interpret the error and either retry with different parameters, ask the user for clarification, or explain the failure in natural language. Always validate the deserialized arguments with `json.loads` before passing them to your functions to prevent injection errors.

### Can I force the model to call a specific function instead of letting it choose?

Yes, instead of passing `function_call="auto"`, you can pass `function_call={"name": "search_courses"}` (or whatever your function name is) to force the model to generate arguments for that specific function. This is useful when you know the user intent requires a particular tool, or when you want to test the function schema without relying on the model's routing logic. After the forced call completes, you can return to `function_call="auto"` for subsequent turns to allow the model to decide whether additional function calls are needed.