LLMClient Architecture: How FluidVoice Handles SSE Streaming and Tool Calls

LLMClient is a stateless singleton that centralizes AI networking in FluidVoice, separating configuration, request execution with exponential back‑off retries, and streaming response handling via SSE lines that parse tool calls and thinking tokens in real time.

FluidVoice’s LLMClient serves as the unified networking layer for all AI‑driven features, from Transcription to Command and Rewrite modes. Implemented in the altic-dev/FluidVoice repository, this architecture follows a strict separation of concerns across configuration, request execution, and streaming response processing. Understanding this LLMClient architecture is essential for developers integrating streaming LLM responses or function‑calling capabilities into Swift applications.

Core Design Principles

The LLMClient architecture is deliberately stateless and implements a single shared singleton pattern. This design prevents redundant connection overhead while ensuring thread-safe access across the application’s various AI modes.

Three-Layer Separation

The implementation separates concerns into distinct layers:

Concern	Implementation	Responsibility
Configuration	LLMClient.Config struct	Encapsulates request payload including messages, model, base URL, API key, streaming flags, tools array, temperature, max‑tokens, and retry policies
Request Execution	call(_:) → buildRequest(_:) → executeWithRetry(request:config:)	Constructs JSON bodies for Chat‑Completions or the newer Responses API, logs cURL equivalents for debugging, and manages automatic retries
Stream Processing	processStreaming(request:config:) and processResponsesStreaming(request:config:)	Parses HTTP‑SSE (data:) lines, handles JSON deltas, extracts thinking tokens, and manages tool‑call state machines

Configuration and Request Building

All network requests originate from the call(_:) method, which delegates to specialized builders depending on the API target.

The Config Struct

The LLMClient.Config struct defines the complete request contract:

struct Config {
    let messages: [Message]
    let model: String
    let baseURL: URL
    let apiKey: String
    let stream: Bool
    let tools: [Tool]?
    let temperature: Double
    let maxTokens: Int
    let retryPolicy: RetryPolicy
    var onEvent: ((StreamEvent) -> Void)?
}

Building the Network Request

The buildRequest(_:) method constructs the URL request and determines whether to use the legacy Chat‑Completions endpoint or the modern Responses API based on the configuration. As implemented in Sources/FluidVoice/Services/LLMClient.swift, the method generates a cURL log output for debugging before execution:

private func buildRequest(_ config: Config) throws -> URLRequest {
    var request = URLRequest(url: config.baseURL.appendingPathComponent("v1/chat/completions"))
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    request.setValue("Bearer \(config.apiKey)", forHTTPHeaderField: "Authorization")
    
    let body = RequestBody(
        model: config.model,
        messages: config.messages,
        stream: config.stream,
        tools: config.tools,
        temperature: config.temperature,
        max_tokens: config.maxTokens
    )
    
    request.httpBody = try JSONEncoder().encode(body)
    logCurl(request) // Debug logging
    return request
}

SSE Streaming and Tool Call Handling

When stream is enabled in the configuration, the LLMClient switches to Server-Sent Events (SSE) processing rather than standard JSON responses.

Processing Server-Sent Events

The processStreaming(request:config:) method establishes the connection and reads lines prefixed with data:. Each line contains a partial JSON delta that must be accumulated and parsed:

private func processStreaming(request: URLRequest, config: Config) async throws {
    let (stream, _) = try await URLSession.shared.bytes(for: request)
    
    for try await line in stream.lines {
        guard line.hasPrefix("data:") else { continue }
        let jsonString = line.dropFirst(5).trimmingCharacters(in: .whitespaces)
        
        if jsonString == "[DONE]" { break }
        
        let delta = try JSONDecoder().decode(StreamDelta.self, from: jsonString.data(using: .utf8)!)
        handleDelta(delta, config: config)
    }
}

Parsing Tool Calls and Thinking Tokens

The stream processor handles two special content types within the SSE deltas:

• Thinking tokens: Extracted from dedicated fields and stripped of <thought> wrapper tags before delivery to the UI
• Tool calls: Accumulated across multiple SSE events until the state machine detects a complete function call

As the stream processes each chunk, the handleDelta(_:config:) method appends content to a buffer and checks for the tool_calls field:

private func handleDelta(_ delta: StreamDelta, config: Config) {
    if let content = delta.choices.first?.delta.content {
        config.onEvent?(.content(content))
    }
    
    if let toolCalls = delta.choices.first?.delta.tool_calls {
        processToolCallDelta(toolCalls, config: config)
    }
    
    if let thinking = delta.choices.first?.delta.thinking {
        let cleaned = thinking.replacingOccurrences(of: "<thought>", with: "")
        config.onEvent?(.thinking(cleaned))
    }
}

State Machine for Tool Execution

Tool calls arrive fragmented across multiple SSE events. The LLMClient maintains an internal state machine that buffers partial JSON until a complete tool call is assembled:

private func processToolCallDelta(_ deltas: [ToolCallDelta], config: Config) {
    for delta in deltas {
        if let index = delta.index {
            toolCallBuffer[index] = (toolCallBuffer[index] ?? "") + (delta.function?.arguments ?? "")
            
            if delta.function?.arguments?.contains("}") == true {
                // Complete tool call detected
                if let json = toolCallBuffer[index]?.data(using: .utf8),
                   let call = try? JSONDecoder().decode(ToolCall.self, from: json) {
                    config.onEvent?(.toolCall(call))
                    toolCallBuffer.removeValue(forKey: index)
                }
            }
        }
    }
}

Retry Logic and Error Handling

The executeWithRetry(request:config:) method implements exponential back‑off for network failures. According to the source code, this wraps the streaming processors to ensure resilience against transient connectivity issues:

private func executeWithRetry(request: URLRequest, config: Config) async throws {
    var attempt = 0
    let maxAttempts = config.retryPolicy.maxAttempts
    
    while attempt < maxAttempts {
        do {
            if config.stream {
                try await processStreaming(request: request, config: config)
            } else {
                try await processNonStreaming(request: request, config: config)
            }
            return
        } catch {
            attempt += 1
            if attempt >= maxAttempts { throw error }
            let delay = pow(2.0, Double(attempt)) // Exponential back‑off
            try await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000))
        }
    }
}

Summary

• LLMClient operates as a stateless singleton in altic-dev/FluidVoice, providing centralized AI networking for transcription, command, and rewrite features.
• The architecture separates configuration (LLMClient.Config), request building (buildRequest(_:)), and stream processing (processStreaming).
• SSE streaming parses data: lines incrementally, extracting content deltas, thinking tokens, and fragmented tool calls.
• A state machine buffers partial tool call JSON until complete function arguments are assembled, then emits them via the configuration’s callback.
• Exponential back‑off retry logic ensures resilience against transient network failures during streaming sessions.

Frequently Asked Questions

How does LLMClient handle connection failures during streaming?

When a connection drops during an active SSE stream, the executeWithRetry(request:config:) method catches the network error and implements exponential back‑off according to the RetryPolicy specified in the configuration. The client will attempt reconnection up to the configured maximum, with delays doubling between attempts (2 seconds, 4 seconds, 8 seconds, etc.) before surfacing the final error to the caller.

What is the difference between processStreaming and processResponsesStreaming?

processStreaming(request:config:) handles the legacy Chat‑Completions API format, parsing standard SSE deltas containing choices arrays. processResponsesStreaming(request:config:) targets the newer Responses API structure, which uses different JSON schemas for streaming outputs. Both methods share the same underlying SSE line-parsing logic but apply distinct JSON decoders to handle their respective API formats.

How are tool calls extracted from SSE deltas?

Tool calls arrive fragmented across multiple SSE events. The LLMClient maintains an internal buffer dictionary indexed by the tool call ID. As each delta arrives containing partial JSON arguments, the processToolCallDelta(_:config:) method appends the fragment to the buffer. When the accumulated string forms valid JSON (detected by closing braces), the method decodes the complete ToolCall struct and emits it through the configuration’s event callback.

Is LLMClient thread-safe for concurrent requests?

Yes. The singleton implementation uses Swift’s actor isolation and URLSession’s inherent thread safety to handle concurrent requests. The call(_:) method is async/await based, allowing multiple simultaneous streaming connections to different endpoints or models without blocking the main thread. Each request maintains its own isolated Config instance, preventing cross-contamination of state between concurrent operations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}