How to Implement Function Calling with Azure OpenAI and External Tools: A Complete Guide

Function calling with Azure OpenAI allows models to request external code execution by defining JSON schemas, detecting function_call responses, and feeding results back into the chat history for natural language replies.

If you are building applications with the microsoft/generative-ai-for-beginners curriculum, learning how to implement function calling with Azure OpenAI and external tools unlocks the ability to connect LLMs to live APIs, databases, and custom business logic. This pattern transforms a static chatbot into a dynamic agent that can fetch real-time data and take action on behalf of users.

Understanding the Function Calling Architecture

The implementation follows a strict three-stage pattern that separates intent detection from execution.

The Three-Stage Pattern

According to the source code in 11-integrating-with-function-calling/README.md, the workflow consists of:

1. Define the function schema – Provide the model with a JSON description of the function name, purpose, and required parameters. The schema is sent in the functions field of the ChatCompletion request.
2. Detect and execute the call – After the model responds, inspect response_message.function_call. If it contains a name, map that name to a real Python callable, deserialize the arguments, invoke the function (e.g., an HTTP request to the Microsoft Learn Catalog API), and capture its output.
3. Feed the result back – Append two new messages to the chat history: one representing the model’s function-call response (role = assistant, function_call payload) and one containing the actual function output (role = function). Then issue a second ChatCompletion call so the model can generate a natural-language reply that incorporates the fetched data.

Why Function Calling Works

This pattern succeeds because it combines structured output with external data retrieval. The model is forced to return arguments that match the JSON schema, eliminating ambiguous free-form text. By delegating to external APIs (such as the Learn Catalog in the reference implementation), you provide up-to-date information that the model itself does not possess, while maintaining a reusable framework for adding new tools.

Setting Up Your Azure OpenAI Client

Before implementing function calling, you must configure the client with your Azure-specific credentials. The lesson code in 11-integrating-with-function-calling/README.md demonstrates this setup:

import os, json
from openai import AzureOpenAI
from dotenv import load_dotenv

load_dotenv()
client = AzureOpenAI(
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    api_version="2023-07-01-preview",
)
deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT")

Ensure your .env file (copied from the root .env.copy template) contains AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, and AZURE_OPENAI_DEPLOYMENT.

Defining Function Schemas for External Tools

The critical step in implementing function calling with Azure OpenAI is describing your external tools using a strict JSON schema. This example from the microsoft/generative-ai-for-beginners repository defines a function that queries the Microsoft Learn Catalog API:

functions = [
    {
        "name": "search_courses",
        "description": "Retrieves courses from the search index based on the parameters provided",
        "parameters": {
            "type": "object",
            "properties": {
                "role": {"type": "string", "description": "Learner role (developer, student, …)"},
                "product": {"type": "string", "description": "Product of interest (Azure, Power BI, …)"},
                "level": {"type": "string", "description": "Experience level (beginner, intermediate, advanced)"}
            },
            "required": ["role"]
        },
    }
]

The name field must match the Python function you will map later. The description helps the model understand when to invoke the tool, while parameters enforces type safety using JSON Schema syntax.

Detecting and Executing Function Calls

Once you send the initial request with function_call="auto", you must inspect the response to determine if the model requested a tool execution:

messages = [{"role": "user", "content": "Find me a good course for a beginner student to learn Azure."}]
response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    functions=functions,
    function_call="auto"
)

response_message = response.choices[0].message

if getattr(response_message, "function_call", None) and response_message.function_call.name:
    # Map the name to a real Python function

    def search_courses(role, product=None, level=None):
        import requests
        url = "https://learn.microsoft.com/api/catalog/"
        params = {"role": role, "product": product, "level": level}
        r = requests.get(url, params=params)
        modules = r.json()["modules"]
        return str([{"title": m["title"], "url": m["url"]} for m in modules[:5]])

    available_functions = {"search_courses": search_courses}
    fn_name = response_message.function_call.name
    fn = available_functions[fn_name]
    fn_args = json.loads(response_message.function_call.arguments)
    fn_result = fn(**fn_args)

This pattern uses getattr to safely check for the function_call attribute, then maps the string name to an actual Python callable using a dictionary lookup. The arguments are parsed from JSON string to dictionary using json.loads.

Feeding Results Back to the Model

To complete the loop, you must append both the model's function request and the actual function output to the message history before requesting a final natural-language response:

# Add the model's function-call message

messages.append({
    "role": response_message.role,
    "function_call": {
        "name": fn_name,
        "arguments": response_message.function_call.arguments,
    },
    "content": None,
})

# Add the function's output

messages.append({
    "role": "function",
    "name": fn_name,
    "content": fn_result,
})

second_response = client.chat.completions.create(
    model=deployment,
    messages=messages,
    functions=functions,
    function_call="auto",
    temperature=0,
)

print(second_response.choices[0].message.content)

The message with role="function" contains the raw data returned by your external tool, while the assistant message with the function_call object preserves the context of what the model intended to do. Setting temperature=0 ensures deterministic output when synthesizing the final answer.

Summary

Implementing function calling with Azure OpenAI and external tools requires a structured three-stage pattern that separates intent detection from execution:

• Define schemas using JSON to describe available functions, their parameters, and descriptions in the functions array passed to chat.completions.create.
• Detect calls by inspecting response_message.function_call after setting function_call="auto", then map the function name to a real Python callable and deserialize arguments with json.loads.
• Feed results back by appending both the assistant's function request and a role="function" message containing the tool output, then issue a second completion call to generate the natural language response.

This pattern, as demonstrated in the microsoft/generative-ai-for-beginners repository, enables LLMs to interact with live APIs, databases, and custom business logic while maintaining type safety and conversational context.

Frequently Asked Questions

What is the difference between function calling and fine-tuning in Azure OpenAI?

Function calling is a runtime capability that allows the model to output structured JSON matching a predefined schema, which your application then executes. Fine-tuning is a training process that modifies the model's weights to improve performance on specific tasks or domains. You use function calling when you need the model to interact with external APIs or tools; you use fine-tuning when you need to change how the model generates text or responds to specific prompts.

How do I handle errors when an external API call fails during function execution?

When implementing the function execution step, wrap your external API calls in try-except blocks. If the call fails, return a descriptive error message as the function result content. When you feed this back to the model via the role="function" message, the LLM can interpret the error and either retry with different parameters, ask the user for clarification, or explain the failure in natural language. Always validate the deserialized arguments with json.loads before passing them to your functions to prevent injection errors.

Can I force the model to call a specific function instead of letting it choose?

Yes, instead of passing function_call="auto", you can pass function_call={"name": "search_courses"} (or whatever your function name is) to force the model to generate arguments for that specific function. This is useful when you know the user intent requires a particular tool, or when you want to test the function schema without relying on the model's routing logic. After the forced call completes, you can return to function_call="auto" for subsequent turns to allow the model to decide whether additional function calls are needed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}