How FluidVoice Communicates with Cloud AI Providers: Architecture Deep Dive

FluidVoice uses a layered protocol-based architecture where LLMClient coordinates requests through the AIProvider protocol, with OpenAICompatibleProvider handling OpenAI-standard HTTP transactions including streaming, retries, and local endpoint detection.

FluidVoice is an open-source iOS dictation app that routes speech-to-text output through cloud-based LLM APIs for formatting and processing. Understanding how FluidVoice communicates with cloud AI providers reveals a well-designed abstraction that supports multiple providers while maintaining clean separation between transport logic and provider-specific implementation.

The AIProvider Protocol: Defining the Contract

At the foundation of FluidVoice's cloud communication sits the AIProvider protocol. Located in FluidVoice/Models/AIProvider.swift, this protocol establishes the minimal surface area required for any cloud AI integration.

// FluidVoice/Models/AIProvider.swift
protocol AIProvider {
    func process(
        systemPrompt: String,
        userText: String,
        model: String,
        apiKey: String,
        baseURL: String,
        stream: Bool
    ) async throws -> String
}

The protocol accepts six parameters that fully describe an LLM interaction: the system prompt for behavior shaping, user text as the actual input, model identifier, API key for authentication, base URL for endpoint customization, and stream flag for response handling mode. This design allows FluidVoice to swap providers without changing calling code.

OpenAICompatibleProvider: Standard HTTP Implementation

OpenAICompatibleProvider in FluidVoice/Services/OpenAICompatibleProvider.swift provides the concrete implementation used for OpenAI, Azure OpenAI, and compatible services like Ollama or LM Studio.

Request Construction

The provider builds a standard OpenAI chat completions request body:

// FluidVoice/Services/OpenAICompatibleProvider.swift
private func buildRequestBody(
    systemPrompt: String,
    userText: String,
    model: String,
    stream: Bool
) -> [String: Any] {
    var body: [String: Any] = [
        "model": model,
        "messages": [
            ["role": "system", "content": systemPrompt],
            ["role": "user", "content": userText]
        ],
        "stream": stream
    ]
    // Additional parameters like temperature, max_tokens applied here
    return body
}

Endpoint Path Selection

The provider intelligently selects between the modern responses API and legacy chat completions endpoint based on the model identifier:

// FluidVoice/Services/OpenAICompatibleProvider.swift
private func endpointPath(for model: String) -> String {
    // o1, o3, and newer reasoning models use /responses
    if model.hasPrefix("o1") || model.hasPrefix("o3") {
        return "/v1/responses"
    }
    return "/v1/chat/completions"
}

Local Endpoint Detection and Authentication

OpenAICompatibleProvider detects local endpoints to conditionally skip API key authentication:

// FluidVoice/Services/OpenAICompatibleProvider.swift
private func requiresAuthHeader(baseURL: String) -> Bool {
    let localPrefixes = ["http://localhost", "http://127.0.0.1", "http://0.0.0.0"]
    return !localPrefixes.contains { baseURL.hasPrefix($0) }
}

private func buildURLRequest(
    baseURL: String,
    body: [String: Any],
    apiKey: String
) -> URLRequest {
    var request = URLRequest(url: URL(string: baseURL + endpointPath)!)
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    
    if requiresAuthHeader(baseURL: baseURL) {
        request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
    }
    
    request.httpBody = try? JSONSerialization.data(withJSONObject: body)
    return request
}

This allows seamless use of local inference servers like Ollama without exposing dummy API keys.

LLMClient: The Unified Coordination Layer

LLMClient in FluidVoice/Services/LLMClient.swift serves as the primary interface for all cloud AI communication in FluidVoice. It orchestrates the entire request lifecycle.

Initialization and Provider Injection

// FluidVoice/Services/LLMClient.swift
class LLMClient {
    private let provider: AIProvider
    private let session: URLSession
    
    init(provider: AIProvider = OpenAICompatibleProvider()) {
        self.provider = provider
        self.session = URLSession(configuration: .default)
    }
}

The dependency injection pattern enables testing with mock providers.

Request Execution with Retry Logic

LLMClient implements exponential backoff for transient failures:

// FluidVoice/Services/LLMClient.swift
func sendRequest(
    systemPrompt: String,
    userText: String,
    model: String,
    apiKey: String,
    baseURL: String,
    stream: Bool = false,
    maxRetries: Int = 3
) async throws -> LLMResponse {
    var lastError: Error?
    
    for attempt in 0..<maxRetries {
        do {
            return try await executeRequest(
                systemPrompt: systemPrompt,
                userText: userText,
                model: model,
                apiKey: apiKey,
                baseURL: baseURL,
                stream: stream
            )
        } catch let error where isRetryable(error) {
            lastError = error
            let delay = calculateBackoff(attempt: attempt)
            try await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000))
        }
    }
    
    throw LLMError.maxRetriesExceeded(underlying: lastError)
}

Streaming Response Parsing

For streaming requests, LLMClient parses Server-Sent Events (SSE) chunks:

// FluidVoice/Services/LLMClient.swift
private func parseStream(
    bytes: AsyncThrowingStream<Data, Error>
) -> AsyncThrowingStream<StreamEvent, Error> {
    AsyncThrowingStream { continuation in
        Task {
            var buffer = ""
            for try await data in bytes {
                buffer.append(String(data: data, encoding: .utf8) ?? "")
                
                while let lineEnd = buffer.firstIndex(of: "\n") {
                    let line = String(buffer[..<lineEnd])
                    buffer.removeSubrange(...lineEnd)
                    
                    if line.hasPrefix("data: ") {
                        let payload = String(line.dropFirst(6))
                        if let event = parseSSEEvent(payload) {
                            continuation.yield(event)
                        }
                    }
                }
            }
            continuation.finish()
        }
    }
}

Thinking Block Extraction

FluidVoice extracts reasoning content from models that expose chain-of-thought:

// FluidVoice/Services/LLMClient.swift
private func extractThinking(from content: String) -> (thinking: String?, response: String) {
    // Parse <thinking> or <reasoning> tags if present
    guard let startTag = content.range(of: "<thinking>"),
          let endTag = content.range(of: "</thinking>") else {
        return (nil, content)
    }
    
    let thinking = String(content[startTag.upperBound..<endTag.lowerBound])
    let response = content[..<startTag.lowerBound] + content[endTag.upperBound...]
    return (thinking.trimmingCharacters(in: .whitespacesAndNewlines),
            String(response).trimmingCharacters(in: .whitespacesAndNewlines))
}

Tool Call Handling

For models supporting function calling, LLMClient parses and formats tool invocations:

// FluidVoice/Services/LLMClient.swift
private func parseToolCalls(from message: [String: Any]) -> [ToolCall]? {
    guard let toolCallsArray = message["tool_calls"] as? [[String: Any]] else {
        return nil
    }
    
    return toolCallsArray.compactMap { call in
        guard let id = call["id"] as? String,
              let function = call["function"] as? [String: Any],
              let name = function["name"] as? String,
              let arguments = function["arguments"] as? String else {
            return nil
        }
        return ToolCall(id: id, functionName: name, arguments: arguments)
    }
}

Complete Usage Example

Here's how FluidVoice's view layer invokes cloud AI providers through LLMClient:

// Example: Formatting dictated text through cloud AI
let client = LLMClient()

do {
    let response = try await client.sendRequest(
        systemPrompt: "Format the following dictation as a professional email.",
        userText: "hey john just wanted to follow up on the meeting tomorrow at three",
        model: "gpt-4o",
        apiKey: "sk-...",
        baseURL: "https://api.openai.com",
        stream: true
    )
    
    // Handle streaming updates
    for try await event in response.stream {
        switch event {
        case .text(let delta):
            updateUI(with: delta)
        case .thinking(let reasoning):
            showThinkingIndicator(reasoning)
        case .toolCall(let call):
            executeTool(call)
        }
    }
} catch LLMError.maxRetriesExceeded {
    showError("Service temporarily unavailable")
} catch LLMError.invalidResponse {
    showError("Unexpected response format")
}

Key Source Files

• [FluidVoice/Models/AIProvider.swift](https://github.com/altic-dev/FluidVoice/blob/main/FluidVoice/Models/AIProvider.swift) — Protocol definition for cloud AI integration
• [FluidVoice/Services/OpenAICompatibleProvider.swift](https://github.com/altic-dev/FluidVoice/blob/main/FluidVoice/Services/OpenAICompatibleProvider.swift) — OpenAI-standard HTTP implementation with local endpoint detection
• [FluidVoice/Services/LLMClient.swift](https://github.com/altic-dev/FluidVoice/blob/main/FluidVoice/Services/LLMClient.swift) — Unified client with retries, streaming, and response parsing
• [FluidVoice/Models/LLMResponse.swift](https://github.com/altic-dev/FluidVoice/blob/main/FluidVoice/Models/LLMResponse.swift) — Response types including thinking blocks and tool calls

Summary

• FluidVoice communicates with cloud AI providers through a three-layer architecture: the AIProvider protocol, OpenAICompatibleProvider implementation, and LLMClient coordinator.
• AIProvider defines a single async method contract that all providers must fulfill, enabling swappable backends.
• OpenAICompatibleProvider handles OpenAI-standard HTTP transport, endpoint path selection (chat completions vs. responses API), and intelligent API key omission for local endpoints.
• LLMClient manages the complete request lifecycle: timeout configuration, exponential backoff retries, SSE streaming parse, thinking block extraction, and tool call handling.
• Local endpoint detection in OpenAICompatibleProvider allows seamless use of Ollama and LM Studio without authentication headers.

Frequently Asked Questions

What cloud AI providers does FluidVoice support?

FluidVoice supports any OpenAI-compatible API through OpenAICompatibleProvider. This includes OpenAI's official API, Azure OpenAI Service, Anthropic (via OpenAI compatibility layer), and local inference servers like Ollama and LM Studio that implement the OpenAI REST interface.

How does FluidVoice handle authentication for local AI endpoints?

OpenAICompatibleProvider detects local endpoints by checking if the base URL starts with http://localhost, http://127.0.0.1, or http://0.0.0.0. When a local endpoint is detected, the Authorization header is omitted entirely, allowing local servers to accept requests without API keys. Remote endpoints always receive the Bearer token authentication.

Can FluidVoice stream responses from cloud AI providers?

Yes, LLMClient fully supports Server-Sent Events streaming through its stream parameter. When enabled, the client parses SSE chunks incrementally, yielding StreamEvent values containing text deltas, thinking blocks, or tool calls. This enables real-time UI updates as the LLM generates content.

What retry logic does FluidVoice implement for failed requests?

LLMClient implements exponential backoff with jitter, attempting up to 3 retries by default. Retryable errors include network timeouts, 5xx server errors, and transient stream failures. The delay between attempts scales as 2^attempt * baseDelay with randomization to prevent thundering herd problems against cloud AI provider rate limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}