How FluidVoice Manages AI Services: Architecture and Provider System

FluidVoice manages AI services through a layered provider architecture that separates provider metadata, runtime selection, and request handling, supporting everything from cloud OpenAI-compatible APIs to local Apple Intelligence models.

The altic-dev/FluidVoice repository implements a modular AI service management system that allows the macOS application to seamlessly switch between cloud providers and on-device models. By abstracting AI interactions behind a common protocol and centralizing provider configuration in a singleton repository, FluidVoice maintains clean separation between UI logic and backend AI implementations.

Provider Catalogue: The ModelRepository Singleton

At the core of FluidVoice's AI management is ModelRepository, a singleton that acts as the central source of truth for all built-in providers. Located in Sources/Fluid/Services/ModelRepository.swift, this class maintains provider IDs, default models, base URLs, and utility methods for endpoint detection.

The repository defines available providers as static identifiers and offers configuration lookup methods:

final class ModelRepository {
    static let shared = ModelRepository()
    
    static var builtInProviderIDs: [String] { 
        ["openai", "anthropic", "groq", "apple-intelligence"] 
    }
    
    func defaultModels(for providerID: String) -> [String] { … }
    func defaultBaseURL(for providerID: String) -> String { … }
    func isLocalEndpoint(_ urlString: String) -> Bool { … }
}

When the UI needs to determine which provider is selected, it uses ModelRepository.providerKey(for:) to read from UserDefaults, ensuring consistent configuration access across views like CommandModeView and RewriteModeView.

Abstract AI Contract: The AIProvider Protocol

All AI backends in FluidVoice conform to the AIProvider protocol defined in Sources/Fluid/Networking/AIProvider.swift. This minimal contract ensures that any provider—whether cloud-based or local—can process text generation requests through a unified interface.

The protocol signature requires implementations to handle asynchronous text processing:

protocol AIProvider {
    func process(systemPrompt: String,
                 userText: String,
                 model: String,
                 apiKey: String,
                 baseURL: String,
                 stream: Bool) async -> String
}

This abstraction allows the application to swap providers at runtime without modifying the transcription enhancement logic that consumes AI services.

Cloud Provider Implementation: OpenAICompatibleProvider

The OpenAICompatibleProvider class handles requests to any OpenAI-compatible endpoint, including OpenAI, Anthropic, Groq, and custom local servers. Implemented in Sources/Fluid/Networking/AIProvider.swift, this provider includes intelligent endpoint detection and provider-specific parameter adaptation.

Key features of the implementation include local endpoint detection and Groq-specific handling:

final class OpenAICompatibleProvider: AIProvider {
    // Detects local endpoints (localhost, 127.*, 10.*, 192.168.*, 172.16-31.*)
    private func isLocalEndpoint(_ urlString: String) -> Bool { … }
    
    // Detects Groq "gpt-oss" models requiring reasoning_effort field
    private func isGptOssModel(_ modelName: String) -> Bool { … }
    
    func process(systemPrompt: String, userText: String, 
                 model: String, apiKey: String, 
                 baseURL: String, stream: Bool) async -> String {
        // Builds endpoint (appends "/chat/completions" if needed)
        // Encodes request with system + user messages, temperature
        // Skips Authorization header for local endpoints
        // Parses response and returns text content
    }
}

When isLocalEndpoint returns true, the provider omits the Authorization header, enabling seamless integration with local LLM servers like Ollama or LM Studio without API keys.

On-Device Provider: AppleIntelligenceProvider

For macOS 26 (Tahoe) and later, FluidVoice supports Apple Intelligence through the AppleIntelligenceProvider class in Sources/Fluid/Networking/AppleIntelligenceProvider.swift. This provider leverages Apple's FoundationModels framework to run language models entirely on-device, eliminating network latency and preserving privacy.

The implementation creates a LanguageModelSession and processes prompts locally:

final class AppleIntelligenceProvider {
    func process(systemPrompt: String, userText: String) async throws -> String {
        let session = LanguageModelSession()
        let fullPrompt = "\(systemPrompt)\n\n\(userText)"
        let response = try await session.respond(to: fullPrompt)
        return response.content
    }
}

Before instantiation, the UI checks AppleIntelligenceService.isAvailable to verify that the host system supports on-device inference.

Function-Calling Layer: FunctionCallingProvider

To support advanced workflows like calendar event creation or MCP (Model Context Protocol) tools, FluidVoice implements FunctionCallingProvider in Sources/Fluid/Networking/FunctionCallingProvider.swift. This layer extends the OpenAI-compatible flow to handle tool definitions, execute function calls, and continue conversations after tool execution.

The provider manages the full tool-calling lifecycle:

final class FunctionCallingProvider {
    func processWithTools(userText: String,
                         conversationHistory: [ChatMessage],
                         tools: [[String: Any]],
                         model: String,
                         apiKey: String,
                         baseURL: String) async -> LLMResult {
        // Builds request with tools array and tool_choice = "auto"
        // Sends request using shared endpoint-building logic
        // Parses tool_calls from response
        // Returns .toolCalls, .textResponse, or .error enum case
    }
}

This implementation allows views to construct dynamic tool descriptions and handle multi-turn conversations where the LLM may request tool execution before providing a final text response.

Runtime Selection and UI Integration

The AI service management flow completes in the UI layer, where views determine which provider to instantiate based on user preferences:

1. User Selection: Settings UI stores the selected provider ID using ModelRepository.providerKey(for:)
2. Provider Instantiation: Views like CommandModeView and RewriteModeView read the stored ID and create the appropriate provider:
   • If ID is "apple-intelligence" and AppleIntelligenceService.isAvailable → instantiate AppleIntelligenceProvider
   • Otherwise → instantiate OpenAICompatibleProvider
3. Request Execution: The view calls either process() for simple text generation or FunctionCallingProvider.processWithTools() for tool-enabled workflows
4. Response Handling: Results flow back through the provider abstraction to the UI without view-layer knowledge of the specific backend

This architecture ensures that adding a new AI provider requires only implementing the AIProvider protocol and registering the provider in ModelRepository, without touching transcription logic or UI code.

Summary

• ModelRepository serves as the centralized configuration hub for provider metadata, default models, and endpoint detection in Sources/Fluid/Services/ModelRepository.swift.
• The AIProvider protocol abstracts all AI interactions behind a single process() method, enabling provider-agnostic transcription enhancement.
• OpenAICompatibleProvider handles cloud and local OpenAI-compatible endpoints with automatic local-endpoint detection and Groq-specific parameter handling.
• AppleIntelligenceProvider enables on-device inference using Apple's FoundationModels framework for macOS 26+ systems.
• FunctionCallingProvider extends the architecture to support MCP tools and multi-turn function-calling conversations.
• Runtime selection occurs in UI views that read UserDefaults via ModelRepository and instantiate the appropriate concrete provider class.

Frequently Asked Questions

What is the AIProvider protocol in FluidVoice?

The AIProvider protocol is the abstract contract defined in Sources/Fluid/Networking/AIProvider.swift that all AI backends must implement. It requires a single asynchronous process() method accepting system prompts, user text, model identifiers, API credentials, and streaming preferences, returning a generated string. This protocol enables FluidVoice to treat OpenAI-compatible APIs, Groq, Anthropic, and Apple Intelligence as interchangeable services.

How does FluidVoice handle local AI endpoints versus cloud APIs?

FluidVoice automatically detects local endpoints through the isLocalEndpoint() method in both ModelRepository and OpenAICompatibleProvider, which checks for localhost, 127.x.x.x, 10.x.x.x, 192.168.x.x, and 172.16-31.x.x addresses. When a local endpoint is detected, the OpenAICompatibleProvider omits the Authorization header from HTTP requests, allowing seamless integration with local LLM servers like Ollama without requiring API keys.

How does FluidVoice support Apple Intelligence on-device models?

For macOS 26 (Tahoe) and later, FluidVoice provides the AppleIntelligenceProvider class in Sources/Fluid/Networking/AppleIntelligenceProvider.swift. This provider uses Apple's FoundationModels framework to create a LanguageModelSession that processes prompts locally without network traffic. The UI checks AppleIntelligenceService.isAvailable before instantiation to ensure the host hardware supports on-device inference.

How does function calling work in FluidVoice?

Function calling is handled by the FunctionCallingProvider class, which builds requests containing tool definitions and parses the tool_calls field from LLM responses. When a model returns tool calls instead of text, the provider returns a .toolCalls enum case containing the function name and arguments. The executing view then runs the appropriate tool (such as calendar APIs) and can continue the conversation by passing the results back through the provider, enabling multi-turn tool-assisted workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}