Integrating Ollama via OpenAI-Compatible API Endpoint in AntSK: A Complete Guide

AntSK integrates Ollama by treating it as an OpenAI-compatible service, automatically rewriting HTTP requests to redirect OpenAI API calls to a local Ollama server while using Semantic Kernel abstractions for chat completions and embeddings.

AntSK seamlessly connects to local LLMs through Ollama's OpenAI-compatible API endpoint, allowing developers to use familiar OpenAI-style interfaces without modifying application logic. By leveraging the AIType.Ollama classification and custom HTTP handlers in the aidotnet/antsk repository, the framework transparently routes requests to Ollama's /v1/chat/completions and /v1/embeddings endpoints.

How AntSK Connects to Ollama Through the OpenAI-Compatible API

The integration architecture consists of three coordinated components that bridge Ollama's local server with AntSK's AI service layer.

Model registration establishes the connection parameters in the database, where AIType.Ollama signals the framework to apply Ollama-specific handling. HTTP client adaptation intercepts outbound OpenAI API requests via OpenAIHttpClientHandlerUtil and rewrites the destination URI to point at the local Ollama instance. Semantic-Kernel registration binds these configured clients to the kernel's chat completion and embedding services, using standard OpenAI connectors with a placeholder API key.

In src/AntSK.Domain/Domain/Service/KernelService.cs, the WithTextGenerationByAIType method contains the case AIType.Ollama clause that triggers this specialized registration path.

Configuring the Ollama Model Registration

To enable Ollama support, administrators create model records specifying the local server endpoint and model identifier. The AIType enum value determines which integration path the framework executes.

The essential configuration requires:

• AIType set to Ollama or OllamaEmbedding (defined in src/AntSK.Domain/Domain/Model/Enum/AIModelType.cs)
• EndPoint pointing to the Ollama server base URL (typically http://127.0.0.1:11434/)
• ModelKey populated with a dummy value (authentication is handled by Ollama locally)

var ollamaModel = new AIModels
{
    ModelName = "llama3.2",
    ModelKey  = "dummy",               // Required but unused for local Ollama
    EndPoint  = "http://127.0.0.1:11434/",
    AIType    = AIType.Ollama,
    MaxLength = 4096
};
_aiModelsRepository.Insert(ollamaModel);

HTTP Client Adaptation and URL Rewriting

AntSK uses OpenAIHttpClientHandlerUtil to create a specialized HttpClient equipped with OpenAIHttpClientHandler. This handler intercepts requests destined for https://api.openai.com and rewrites the URI to target the configured Ollama base URL.

The handler processes both chat completion and embedding routes in its SendAsync method:

// Inside OpenAIHttpClientHandler.SendAsync
if (request.RequestUri.LocalPath == "/v1/chat/completions")
{
    var uriBuilder = new UriBuilder(request.RequestUri)
    {
        Scheme = $"{protocol}://{hostnew}/",
        Host = host,
        Path = route + "v1/chat/completions",
    };
    // apply port if any …
    request.RequestUri = uriBuilder.Uri;
}

This approach allows the Semantic Kernel's standard OpenAI connectors to function unmodified while actually communicating with Ollama's compatible endpoints at /v1/chat/completions and /v1/embeddings.

Semantic Kernel Service Registration

When constructing a kernel instance via KernelService.GetKernelByApp, the framework detects the AIType.Ollama classification and registers an OpenAI chat completion service with the adapted HTTP client. This registration uses a dummy API key ("NotNull") since Ollama does not require authentication tokens.

The registration occurs in src/AntSK.Domain/Domain/Service/KernelService.cs:

// Within WithTextGenerationByAIType method
case AIType.Ollama:
    builder.AddOpenAIChatCompletion(
        modelId: chatModel.ModelName,
        apiKey: "NotNull",              // Placeholder required by Semantic Kernel
        httpClient: chatHttpClient);    // Pre-configured with Ollama URL rewriter
    break;

For embedding models, a similar pattern applies using AIType.OllamaEmbedding, which registers OpenAI text embedding services pointing to the same local endpoint.

Pulling Models via the Ollama CLI Integration

AntSK includes a model management helper that invokes the Ollama CLI directly from the web interface. The OllamaService class (defined in src/AntSK.Domain/Domain/Service/OllamaService.cs and contract src/AntSK.Domain/Domain/Interface/IOllamaService.cs) spawns a process executing ollama pull <model> and streams the output to the UI.

The UI component in src/AntSK/Pages/Setting/AIModel/AddModel.razor.cs triggers this download when users click the "下载模型" (Download Model) button:

await _ollamaService.OllamaPull(_aiModel.ModelName);

The service implementation captures stdout and stderr from the CLI process, broadcasting progress updates through the LogMessageReceived event to provide real-time feedback during model downloads.

Summary

• AntSK treats Ollama as an OpenAI-compatible provider by rewriting HTTP requests to redirect from api.openai.com to the local Ollama server endpoint.
• Model configuration requires AIType.Ollama and an endpoint URL like http://127.0.0.1:11434/ stored in the AIModels repository.
• HTTP adaptation happens through OpenAIHttpClientHandler, which modifies request URIs to target /v1/chat/completions and /v1/embeddings on the Ollama instance.
• Kernel registration uses standard AddOpenAIChatCompletion with a dummy API key ("NotNull") and the custom HTTP client from OpenAIHttpClientHandlerUtil.
• Model management is facilitated by OllamaService.OllamaPull, which executes the ollama pull CLI command and streams results to the admin UI.

Frequently Asked Questions

How does AntSK handle authentication when connecting to Ollama?

AntSK passes a placeholder API key ("NotNull") to satisfy the Semantic Kernel OpenAI connector's requirement for non-null authentication headers. Since Ollama runs locally and does not validate API keys, the actual credential value is ignored, allowing seamless integration without exposing real OpenAI keys.

Can I use both chat and embedding models from Ollama in the same AntSK application?

Yes. AntSK supports simultaneous use of Ollama models for both operations by setting AIType.Ollama for chat completion models and AIType.OllamaEmbedding for embedding models. Both types use the same HTTP client adaptation logic but register different Semantic Kernel services (chat completion versus text embedding) in KernelService.WithTextGenerationByAIType.

What URL paths does AntSK rewrite to support Ollama's API?

The OpenAIHttpClientHandler specifically intercepts requests to /v1/chat/completions and /v1/embeddings, rewriting the host and scheme to match the configured Ollama base URL (e.g., http://127.0.0.1:11434). This allows the OpenAI SDK to generate requests that Ollama's OpenAI-compatible server can process without protocol modifications.

How do I download a new model into Ollama from the AntSK interface?

Navigate to the AI model settings page and click the "下载模型" button. This invokes OllamaService.OllamaPull, which executes ollama pull <model-name> as a subprocess and displays real-time CLI output in the interface. Ensure the Ollama CLI is installed and available in the system PATH for this feature to function.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}