# How FluidVoice Manages AI Services: Architecture and Provider System

> Discover how FluidVoice manages AI services with its layered provider architecture. Explore seamless integration from cloud OpenAI APIs to local Apple Intelligence models.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: architecture
- Published: 2026-06-30

---

**FluidVoice manages AI services through a layered provider architecture that separates provider metadata, runtime selection, and request handling, supporting everything from cloud OpenAI-compatible APIs to local Apple Intelligence models.**

The `altic-dev/FluidVoice` repository implements a modular AI service management system that allows the macOS application to seamlessly switch between cloud providers and on-device models. By abstracting AI interactions behind a common protocol and centralizing provider configuration in a singleton repository, FluidVoice maintains clean separation between UI logic and backend AI implementations.

## Provider Catalogue: The ModelRepository Singleton

At the core of FluidVoice's AI management is **ModelRepository**, a singleton that acts as the central source of truth for all built-in providers. Located in [`Sources/Fluid/Services/ModelRepository.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ModelRepository.swift), this class maintains provider IDs, default models, base URLs, and utility methods for endpoint detection.

The repository defines available providers as static identifiers and offers configuration lookup methods:

```swift
final class ModelRepository {
    static let shared = ModelRepository()
    
    static var builtInProviderIDs: [String] { 
        ["openai", "anthropic", "groq", "apple-intelligence"] 
    }
    
    func defaultModels(for providerID: String) -> [String] { … }
    func defaultBaseURL(for providerID: String) -> String { … }
    func isLocalEndpoint(_ urlString: String) -> Bool { … }
}

```

When the UI needs to determine which provider is selected, it uses `ModelRepository.providerKey(for:)` to read from `UserDefaults`, ensuring consistent configuration access across views like `CommandModeView` and `RewriteModeView`.

## Abstract AI Contract: The AIProvider Protocol

All AI backends in FluidVoice conform to the **AIProvider** protocol defined in [`Sources/Fluid/Networking/AIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/AIProvider.swift). This minimal contract ensures that any provider—whether cloud-based or local—can process text generation requests through a unified interface.

The protocol signature requires implementations to handle asynchronous text processing:

```swift
protocol AIProvider {
    func process(systemPrompt: String,
                 userText: String,
                 model: String,
                 apiKey: String,
                 baseURL: String,
                 stream: Bool) async -> String
}

```

This abstraction allows the application to swap providers at runtime without modifying the transcription enhancement logic that consumes AI services.

## Cloud Provider Implementation: OpenAICompatibleProvider

The **OpenAICompatibleProvider** class handles requests to any OpenAI-compatible endpoint, including OpenAI, Anthropic, Groq, and custom local servers. Implemented in [`Sources/Fluid/Networking/AIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/AIProvider.swift), this provider includes intelligent endpoint detection and provider-specific parameter adaptation.

Key features of the implementation include local endpoint detection and Groq-specific handling:

```swift
final class OpenAICompatibleProvider: AIProvider {
    // Detects local endpoints (localhost, 127.*, 10.*, 192.168.*, 172.16-31.*)
    private func isLocalEndpoint(_ urlString: String) -> Bool { … }
    
    // Detects Groq "gpt-oss" models requiring reasoning_effort field
    private func isGptOssModel(_ modelName: String) -> Bool { … }
    
    func process(systemPrompt: String, userText: String, 
                 model: String, apiKey: String, 
                 baseURL: String, stream: Bool) async -> String {
        // Builds endpoint (appends "/chat/completions" if needed)
        // Encodes request with system + user messages, temperature
        // Skips Authorization header for local endpoints
        // Parses response and returns text content
    }
}

```

When `isLocalEndpoint` returns true, the provider omits the `Authorization` header, enabling seamless integration with local LLM servers like Ollama or LM Studio without API keys.

## On-Device Provider: AppleIntelligenceProvider

For macOS 26 (Tahoe) and later, FluidVoice supports **Apple Intelligence** through the `AppleIntelligenceProvider` class in [`Sources/Fluid/Networking/AppleIntelligenceProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/AppleIntelligenceProvider.swift). This provider leverages Apple's `FoundationModels` framework to run language models entirely on-device, eliminating network latency and preserving privacy.

The implementation creates a `LanguageModelSession` and processes prompts locally:

```swift
final class AppleIntelligenceProvider {
    func process(systemPrompt: String, userText: String) async throws -> String {
        let session = LanguageModelSession()
        let fullPrompt = "\(systemPrompt)\n\n\(userText)"
        let response = try await session.respond(to: fullPrompt)
        return response.content
    }
}

```

Before instantiation, the UI checks `AppleIntelligenceService.isAvailable` to verify that the host system supports on-device inference.

## Function-Calling Layer: FunctionCallingProvider

To support advanced workflows like calendar event creation or MCP (Model Context Protocol) tools, FluidVoice implements **FunctionCallingProvider** in [`Sources/Fluid/Networking/FunctionCallingProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/FunctionCallingProvider.swift). This layer extends the OpenAI-compatible flow to handle tool definitions, execute function calls, and continue conversations after tool execution.

The provider manages the full tool-calling lifecycle:

```swift
final class FunctionCallingProvider {
    func processWithTools(userText: String,
                         conversationHistory: [ChatMessage],
                         tools: [[String: Any]],
                         model: String,
                         apiKey: String,
                         baseURL: String) async -> LLMResult {
        // Builds request with tools array and tool_choice = "auto"
        // Sends request using shared endpoint-building logic
        // Parses tool_calls from response
        // Returns .toolCalls, .textResponse, or .error enum case
    }
}

```

This implementation allows views to construct dynamic tool descriptions and handle multi-turn conversations where the LLM may request tool execution before providing a final text response.

## Runtime Selection and UI Integration

The AI service management flow completes in the UI layer, where views determine which provider to instantiate based on user preferences:

1. **User Selection**: Settings UI stores the selected provider ID using `ModelRepository.providerKey(for:)`
2. **Provider Instantiation**: Views like `CommandModeView` and `RewriteModeView` read the stored ID and create the appropriate provider:
   - If ID is `"apple-intelligence"` and `AppleIntelligenceService.isAvailable` → instantiate `AppleIntelligenceProvider`
   - Otherwise → instantiate `OpenAICompatibleProvider`
3. **Request Execution**: The view calls either `process()` for simple text generation or `FunctionCallingProvider.processWithTools()` for tool-enabled workflows
4. **Response Handling**: Results flow back through the provider abstraction to the UI without view-layer knowledge of the specific backend

This architecture ensures that adding a new AI provider requires only implementing the `AIProvider` protocol and registering the provider in `ModelRepository`, without touching transcription logic or UI code.

## Summary

- **ModelRepository** serves as the centralized configuration hub for provider metadata, default models, and endpoint detection in [`Sources/Fluid/Services/ModelRepository.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ModelRepository.swift).
- The **AIProvider** protocol abstracts all AI interactions behind a single `process()` method, enabling provider-agnostic transcription enhancement.
- **OpenAICompatibleProvider** handles cloud and local OpenAI-compatible endpoints with automatic local-endpoint detection and Groq-specific parameter handling.
- **AppleIntelligenceProvider** enables on-device inference using Apple's FoundationModels framework for macOS 26+ systems.
- **FunctionCallingProvider** extends the architecture to support MCP tools and multi-turn function-calling conversations.
- Runtime selection occurs in UI views that read `UserDefaults` via `ModelRepository` and instantiate the appropriate concrete provider class.

## Frequently Asked Questions

### What is the AIProvider protocol in FluidVoice?

The **AIProvider** protocol is the abstract contract defined in [`Sources/Fluid/Networking/AIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/AIProvider.swift) that all AI backends must implement. It requires a single asynchronous `process()` method accepting system prompts, user text, model identifiers, API credentials, and streaming preferences, returning a generated string. This protocol enables FluidVoice to treat OpenAI-compatible APIs, Groq, Anthropic, and Apple Intelligence as interchangeable services.

### How does FluidVoice handle local AI endpoints versus cloud APIs?

FluidVoice automatically detects local endpoints through the `isLocalEndpoint()` method in both `ModelRepository` and `OpenAICompatibleProvider`, which checks for localhost, 127.x.x.x, 10.x.x.x, 192.168.x.x, and 172.16-31.x.x addresses. When a local endpoint is detected, the `OpenAICompatibleProvider` omits the `Authorization` header from HTTP requests, allowing seamless integration with local LLM servers like Ollama without requiring API keys.

### How does FluidVoice support Apple Intelligence on-device models?

For macOS 26 (Tahoe) and later, FluidVoice provides the `AppleIntelligenceProvider` class in [`Sources/Fluid/Networking/AppleIntelligenceProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Networking/AppleIntelligenceProvider.swift). This provider uses Apple's `FoundationModels` framework to create a `LanguageModelSession` that processes prompts locally without network traffic. The UI checks `AppleIntelligenceService.isAvailable` before instantiation to ensure the host hardware supports on-device inference.

### How does function calling work in FluidVoice?

Function calling is handled by the `FunctionCallingProvider` class, which builds requests containing tool definitions and parses the `tool_calls` field from LLM responses. When a model returns tool calls instead of text, the provider returns a `.toolCalls` enum case containing the function name and arguments. The executing view then runs the appropriate tool (such as calendar APIs) and can continue the conversation by passing the results back through the provider, enabling multi-turn tool-assisted workflows.