Configuring Model Clients for Azure OpenAI and OpenAI Compatible APIs in Autogen

Autogen provides OpenAIChatCompletionClient for standard OpenAI endpoints and AzureOpenAIChatCompletionClient for Azure deployments, with both classes inheriting from BaseOpenAIChatCompletionClient in _openai_client.py to manage streaming, token accounting, and multiple authentication schemes.

When building agentic workflows with the microsoft/autogen framework, configuring model clients for Azure OpenAI and OpenAI compatible APIs requires understanding the underlying client architecture. The autogen-ext package offers specialized chat completion clients that wrap the OpenAI SDK while supporting both API key and Azure Active Directory authentication methods.

Client Architecture and Core Implementation

Base Client Implementation

The foundation for both clients resides in python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py. The BaseOpenAIChatCompletionClient class implements shared logic for request creation, token counting, and usage aggregation. Concrete implementations OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient begin at line 1520, each constructing their respective SDK clients through _openai_client_from_config and _azure_openai_client_from_config helpers.

Configuration Structures

Typed dictionaries and Pydantic models in python/packages/autogen-ext/src/autogen_ext/models/openai/config/__init__.py define OpenAIClientConfiguration and AzureOpenAIClientConfiguration. Azure configurations require azure_endpoint, azure_deployment, and api_version, while supporting either api_key for key-based auth or azure_ad_token/azure_ad_token_provider for Azure AD.

Request Processing and Streaming

The _process_create_args method converts Autogen LLMMessage objects into OpenAI SDK format, merging tool definitions and validating model capabilities. For streaming responses, _create_stream_chunks and _create_stream_chunks_beta_client handle chunk processing and yield CreateResult objects containing content, reasoning traces, token usage, and log-probabilities.

Configuring Standard OpenAI Endpoints

The OpenAIChatCompletionClient connects to any OpenAI-compatible API endpoint using API key authentication. It builds an AsyncOpenAI instance internally and passes openai_init_kwargs to the underlying SDK.

from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_core.models import UserMessage

client = OpenAIChatCompletionClient(
    model="gpt-4o-2024-08-06",
    temperature=0.2,
    # Reads OPENAI_API_KEY from environment if omitted

)

result = await client.create([UserMessage(content="What is the capital of France?")])
print(result.content)  # → "Paris"

await client.close()

Configuring Azure OpenAI Deployments

API Key Authentication

For key-based authentication, provide api_key alongside the required Azure-specific fields in your AzureOpenAIChatCompletionClient initialization.

Azure Active Directory Authentication

For production environments requiring Azure AD, use the AzureTokenProvider from python/packages/autogen-ext/src/autogen_ext/auth/azure.py with credentials from the Azure SDK.

from autogen_ext.models.openai import AzureOpenAIChatCompletionClient
from autogen_ext.auth.azure import AzureTokenProvider
from azure.identity import DefaultAzureCredential
from autogen_core.models import UserMessage

token_provider = AzureTokenProvider(
    credential=DefaultAzureCredential(),
    scopes=["https://cognitiveservices.azure.com/.default"],
)

client = AzureOpenAIChatCompletionClient(
    model="gpt-4o",
    azure_endpoint="https://myendpoint.openai.azure.com/",
    azure_deployment="my-deployment",
    api_version="2024-06-01",
    azure_ad_token_provider=token_provider,
)

result = await client.create([UserMessage(content="Summarize the latest news.")])
print(result.content)

await client.close()

Loading Clients from Component Configuration

Both clients register as Autogen components (component_type = "model"), enabling deserialization from configuration dictionaries via ChatCompletionClient.load_component. This supports declarative setup for Azure AD providers nested within the config.

from autogen_core.models import ChatCompletionClient

cfg = {
    "provider": "AzureOpenAIChatCompletionClient",
    "config": {
        "model": "gpt-4o-2024-05-13",
        "azure_endpoint": "https://myendpoint.openai.azure.com/",
        "azure_deployment": "my-deployment",
        "api_version": "2024-06-01",
        "azure_ad_token_provider": {
            "provider": "autogen_ext.auth.azure.AzureTokenProvider",
            "config": {
                "provider_kind": "DefaultAzureCredential",
                "scopes": ["https://cognitiveservices.azure.com/.default"]
            }
        }
    }
}
client = ChatCompletionClient.load_component(cfg)

Summary

• Autogen provides two primary clients: OpenAIChatCompletionClient for OpenAI-compatible APIs and AzureOpenAIChatCompletionClient for Azure OpenAI deployments.
• Both inherit from BaseOpenAIChatCompletionClient in _openai_client.py, sharing implementations for streaming, token counting, and request processing.
• Configuration uses typed dictionaries in config/__init__.py, with Azure requiring endpoint, deployment, and API version fields.
• Authentication supports API keys for both services, plus Azure AD tokens via AzureTokenProvider from auth/azure.py for secure Azure deployments.
• Component loading allows dynamic client instantiation from dictionaries using ChatCompletionClient.load_component, supporting complex nested configurations.

Frequently Asked Questions

What is the difference between OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient?

OpenAIChatCompletionClient wraps the standard AsyncOpenAI SDK client for direct OpenAI API or compatible endpoints, while AzureOpenAIChatCompletionClient wraps AsyncAzureOpenAI specifically for Azure OpenAI Service deployments. The Azure variant supports Azure-specific parameters like azure_deployment and azure_ad_token_provider for Active Directory authentication, which the standard client does not require.

How do I authenticate with Azure OpenAI using Managed Identity instead of API keys?

Use the AzureTokenProvider class from autogen_ext.auth.azure with DefaultAzureCredential or another Azure SDK credential class. Pass this provider to AzureOpenAIChatCompletionClient as the azure_ad_token_provider parameter. This approach eliminates hardcoded secrets by leveraging Azure Active Directory and Managed Identity credentials.

Can I use these clients with non-OpenAI endpoints that follow the OpenAI API format?

Yes, OpenAIChatCompletionClient supports any OpenAI-compatible endpoint by specifying a custom base_url in the configuration. The client constructs an AsyncOpenAI instance that sends requests to your specified endpoint while maintaining the same message format and streaming behavior as the standard OpenAI API.

Where is the streaming logic implemented in the Autogen source code?

Streaming logic resides in BaseOpenAIChatCompletionClient within python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py. The methods _create_stream_chunks and _create_stream_chunks_beta_client handle the asynchronous iteration over SDK chunks, aggregating content and usage statistics into CreateResult objects for both standard and beta client implementations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}