How to Integrate LLMs and AI Services into .NET Applications

The dotnet-skills repository provides a production-ready, three-layer architecture for integrating LLMs and AI services into .NET 8+ applications using Microsoft.Extensions.AI (MEAI) abstractions, concrete provider SDKs, and the Microsoft.Agents.AI orchestration framework.

Integrating large language models into .NET projects requires more than simple API calls. The dotnet/skills repository defines a complete stack that separates concerns between abstraction, implementation, and orchestration to keep your code testable and provider-agnostic. This guide walks through the exact patterns found in plugins/dotnet-ai/skills/technology-selection/SKILL.md and the reference implementations in the test fixtures.

Understanding the Three-Layer Architecture

The architecture is built around three concentric layers that prevent vendor lock-in and simplify testing.

Abstraction Layer with MEAI

The Microsoft.Extensions.AI (MEAI) package defines generic contracts for chatting, embeddings, and tool calling. Core interfaces include IChatClient for completions and IEmbeddingGenerator for vector embeddings. By coding against these abstractions, your business logic remains independent of any specific provider.

Provider SDK Layer

Concrete implementations live in provider-specific packages such as OpenAI, Azure.AI.OpenAI, Azure.AI.Inference, or OllamaSharp. These SDKs implement the MEAI interfaces and handle authentication, transport, and model-specific serialization. You register these in dependency injection (DI) without referencing them directly in your application code.

Orchestration Layer with Agent Framework

The Microsoft.Agents.AI package (currently in prerelease) handles multi-step agentic workflows, tool dispatch, and durable context management. Entry points like ChatClientAgent and AgentWorker sit atop the MEAI abstraction to manage the "tool call → result → re-prompt" loop automatically.

Registering the AI Stack in Dependency Injection

Always register the AI client through the MEAI abstraction and let the Agent Framework consume it. This registration pattern from SKILL.md wires the OpenAI SDK to the abstraction layer:

// Program.cs
builder.Services
    .AddMicrosoftExtensionsAI()                     // adds MEAI core services
    .AddChatClient<OpenAIChatClient>(options =>      // concrete provider
    {
        options.ApiKey = configuration["AI:ApiKey"];
        options.Endpoint = new Uri("https://api.openai.com/v1/");
    })
    .UseOpenAIChatClient("gpt-4o-mini-2024-07-18");   // default model

// Optional: Register the Agent Framework for complex workflows
builder.Services.AddAgentsAI();

To swap providers, change only the AddChatClient call—for example, to AddChatClient<AzureOpenAIChatClient>—while the rest of your codebase continues using IChatClient.

Making Simple LLM Calls

For straightforward completions without tool calling, resolve IChatClient from the service provider and call CompleteAsync. This same code works whether the underlying model is hosted on Azure, OpenAI, or a local Ollama server:

var chat = provider.GetRequiredService<IChatClient>();

var response = await chat.CompleteAsync(
    "Summarize the following document:\n\n" + documentText,
    new ChatCompletionOptions { MaxOutputTokens = 200 });

Console.WriteLine(response);

This pattern is demonstrated in tests/dotnet-ai/technology-selection/fixtures/llm-integration-with-meai-abstraction/DocSummary/Program.cs.

Building Agentic Workflows with Tool Calling

When you need function calling or multi-step reasoning, use the Agent Framework. Define tools by implementing ITool, then pass them to a ChatClientAgent:

// Define a tool that fetches a URL
public record FetchUrlTool(string Url) : ITool
{
    public async Task<ToolResult> InvokeAsync()
    {
        using var http = new HttpClient();
        var content = await http.GetStringAsync(Url);
        return new ToolResult(content);
    }
}

// Build an agent that can use the tool
var agent = new ChatClientAgent(chat,
    new[] { new FetchUrlTool("") }  // tool schema auto-generated
);

var result = await agent.InvokeAsync(
    "Read the page https://example.com and list the three top headlines.");

The framework automatically handles retries, guardrails, and context stitching without manual JSON parsing. See the full implementation in tests/dotnet-ai/technology-selection/fixtures/agentic-workflow-with-guardrails/ResearchAgent/Program.cs.

Implementing Retrieval-Augmented Generation (RAG)

For RAG pipelines, register a vector store and use the document ingestion service. The MEAI.DataIngestion package handles parsing, chunking, embedding, and upserting into any database implementing Microsoft.Extensions.VectorData.Abstractions:

// Register Azure AI Search as the vector store
builder.Services.AddVectorStoreAzureSearch(
    configuration.GetConnectionString("AzureSearch"));

// Ingest documents
var ingestion = provider.GetRequiredService<IDocumentIngestor>();
await ingestion.IngestAsync(
    new[] { "doc1.txt", "doc2.pdf" },
    new IngestionOptions { ChunkSize = 1024, Overlap = 128 });

// Query with RAG
var rag = provider.GetRequiredService<IRagService>();
var answer = await rag.AnswerAsync(
    "What are the licensing restrictions for the ML.NET model?",
    new RagOptions { TopK = 5 });

The IRagService abstraction keeps the similarity search portable across vector database providers.

Adding Health Checks and Observability

Register health checks to ensure AI endpoints are reachable before accepting traffic:

builder.Services.AddHealthChecks()
    .AddCheck<OpenAIHealthCheck>("openai");

The repository also includes an AgentRunner that reports evaluation metrics—including latency, token usage, and success rates—to the skill-validator dashboard, as implemented in eng/skill-validator/src/Shared/AgentDiscovery.cs.

Summary

• Start with MEAI to define provider-agnostic contracts (IChatClient, IEmbeddingGenerator).
• Add concrete SDKs (OpenAI, Azure, Ollama) only in DI registration, never in business logic.
• Use the Agent Framework for workflows requiring tool calls, multi-step reasoning, or multi-agent collaboration.
• Leverage vector-store abstractions for RAG pipelines that remain portable across databases.
• Implement health checks and utilize the built-in evaluation pipeline for production monitoring.

Frequently Asked Questions

What is Microsoft.Extensions.AI (MEAI)?

Microsoft.Extensions.AI is a set of abstractions in the dotnet/skills architecture that defines generic interfaces like IChatClient and IEmbeddingGenerator for LLM operations. It allows your application code to remain agnostic to specific providers such as OpenAI or Azure, making it straightforward to swap implementations or mock services for testing.

How do I switch from OpenAI to Azure OpenAI without rewriting code?

Change only the DI registration in Program.cs by replacing AddChatClient<OpenAIChatClient> with AddChatClient<AzureOpenAIChatClient> and updating the configuration parameters. Because your business logic depends only on IChatClient, no other code changes are required, maintaining the layering principle defined in SKILL.md.

When should I use the Agent Framework instead of direct LLM calls?

Use direct LLM calls via IChatClient for simple request-response scenarios like summarization or translation. Use the Agent Framework (ChatClientAgent, AgentWorker) when you need automatic tool invocation, multi-step reasoning loops, or coordinated multi-agent workflows that require the framework to handle the "tool call → execution → re-prompt" cycle.

How does the dotnet-skills repository handle document ingestion for RAG?

The repository uses the MEAI.DataIngestion package to parse files, chunk content, generate embeddings, and upsert vectors into any store implementing Microsoft.Extensions.VectorData.Abstractions. The IDocumentIngestor service handles the pipeline, while IRagService manages the retrieval and generation phase, keeping the RAG implementation portable across vector databases like Azure AI Search.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}