# LLMClient Architecture: How FluidVoice Handles SSE Streaming and Tool Calls

> Explore the LLMClient architecture powering FluidVoice. Learn how it centralizes AI networking, manages SSE streaming, and parses tool calls in real time for efficient AI communication.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: architecture
- Published: 2026-07-03

---

**LLMClient is a stateless singleton that centralizes AI networking in FluidVoice, separating configuration, request execution with exponential back‑off retries, and streaming response handling via SSE lines that parse tool calls and thinking tokens in real time.**

FluidVoice’s `LLMClient` serves as the unified networking layer for all AI‑driven features, from Transcription to Command and Rewrite modes. Implemented in the `altic-dev/FluidVoice` repository, this architecture follows a strict separation of concerns across configuration, request execution, and streaming response processing. Understanding this LLMClient architecture is essential for developers integrating streaming LLM responses or function‑calling capabilities into Swift applications.

## Core Design Principles

The LLMClient architecture is deliberately **stateless** and implements a single shared singleton pattern. This design prevents redundant connection overhead while ensuring thread-safe access across the application’s various AI modes.

### Three-Layer Separation

The implementation separates concerns into distinct layers:

| Concern | Implementation | Responsibility |
|---------|---------------|--------------|
| **Configuration** | `LLMClient.Config` struct | Encapsulates request payload including messages, model, base URL, API key, streaming flags, tools array, temperature, max‑tokens, and retry policies |
| **Request Execution** | `call(_:)` → `buildRequest(_:)` → `executeWithRetry(request:config:)` | Constructs JSON bodies for Chat‑Completions or the newer *Responses* API, logs cURL equivalents for debugging, and manages automatic retries |
| **Stream Processing** | `processStreaming(request:config:)` and `processResponsesStreaming(request:config:)` | Parses HTTP‑SSE (`data:`) lines, handles JSON deltas, extracts thinking tokens, and manages tool‑call state machines |

## Configuration and Request Building

All network requests originate from the `call(_:)` method, which delegates to specialized builders depending on the API target.

### The Config Struct

The `LLMClient.Config` struct defines the complete request contract:

```swift
struct Config {
    let messages: [Message]
    let model: String
    let baseURL: URL
    let apiKey: String
    let stream: Bool
    let tools: [Tool]?
    let temperature: Double
    let maxTokens: Int
    let retryPolicy: RetryPolicy
    var onEvent: ((StreamEvent) -> Void)?
}

```

### Building the Network Request

The `buildRequest(_:)` method constructs the URL request and determines whether to use the legacy Chat‑Completions endpoint or the modern *Responses* API based on the configuration. As implemented in [`Sources/FluidVoice/Services/LLMClient.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/FluidVoice/Services/LLMClient.swift), the method generates a cURL log output for debugging before execution:

```swift
private func buildRequest(_ config: Config) throws -> URLRequest {
    var request = URLRequest(url: config.baseURL.appendingPathComponent("v1/chat/completions"))
    request.httpMethod = "POST"
    request.setValue("application/json", forHTTPHeaderField: "Content-Type")
    request.setValue("Bearer \(config.apiKey)", forHTTPHeaderField: "Authorization")
    
    let body = RequestBody(
        model: config.model,
        messages: config.messages,
        stream: config.stream,
        tools: config.tools,
        temperature: config.temperature,
        max_tokens: config.maxTokens
    )
    
    request.httpBody = try JSONEncoder().encode(body)
    logCurl(request) // Debug logging
    return request
}

```

## SSE Streaming and Tool Call Handling

When `stream` is enabled in the configuration, the LLMClient switches to Server-Sent Events (SSE) processing rather than standard JSON responses.

### Processing Server-Sent Events

The `processStreaming(request:config:)` method establishes the connection and reads lines prefixed with `data:`. Each line contains a partial JSON delta that must be accumulated and parsed:

```swift
private func processStreaming(request: URLRequest, config: Config) async throws {
    let (stream, _) = try await URLSession.shared.bytes(for: request)
    
    for try await line in stream.lines {
        guard line.hasPrefix("data:") else { continue }
        let jsonString = line.dropFirst(5).trimmingCharacters(in: .whitespaces)
        
        if jsonString == "[DONE]" { break }
        
        let delta = try JSONDecoder().decode(StreamDelta.self, from: jsonString.data(using: .utf8)!)
        handleDelta(delta, config: config)
    }
}

```

### Parsing Tool Calls and Thinking Tokens

The stream processor handles two special content types within the SSE deltas:

- **Thinking tokens**: Extracted from dedicated fields and stripped of `<thought>` wrapper tags before delivery to the UI
- **Tool calls**: Accumulated across multiple SSE events until the state machine detects a complete function call

As the stream processes each chunk, the `handleDelta(_:config:)` method appends content to a buffer and checks for the `tool_calls` field:

```swift
private func handleDelta(_ delta: StreamDelta, config: Config) {
    if let content = delta.choices.first?.delta.content {
        config.onEvent?(.content(content))
    }
    
    if let toolCalls = delta.choices.first?.delta.tool_calls {
        processToolCallDelta(toolCalls, config: config)
    }
    
    if let thinking = delta.choices.first?.delta.thinking {
        let cleaned = thinking.replacingOccurrences(of: "<thought>", with: "")
        config.onEvent?(.thinking(cleaned))
    }
}

```

### State Machine for Tool Execution

Tool calls arrive fragmented across multiple SSE events. The LLMClient maintains an internal state machine that buffers partial JSON until a complete tool call is assembled:

```swift
private func processToolCallDelta(_ deltas: [ToolCallDelta], config: Config) {
    for delta in deltas {
        if let index = delta.index {
            toolCallBuffer[index] = (toolCallBuffer[index] ?? "") + (delta.function?.arguments ?? "")
            
            if delta.function?.arguments?.contains("}") == true {
                // Complete tool call detected
                if let json = toolCallBuffer[index]?.data(using: .utf8),
                   let call = try? JSONDecoder().decode(ToolCall.self, from: json) {
                    config.onEvent?(.toolCall(call))
                    toolCallBuffer.removeValue(forKey: index)
                }
            }
        }
    }
}

```

## Retry Logic and Error Handling

The `executeWithRetry(request:config:)` method implements exponential back‑off for network failures. According to the source code, this wraps the streaming processors to ensure resilience against transient connectivity issues:

```swift
private func executeWithRetry(request: URLRequest, config: Config) async throws {
    var attempt = 0
    let maxAttempts = config.retryPolicy.maxAttempts
    
    while attempt < maxAttempts {
        do {
            if config.stream {
                try await processStreaming(request: request, config: config)
            } else {
                try await processNonStreaming(request: request, config: config)
            }
            return
        } catch {
            attempt += 1
            if attempt >= maxAttempts { throw error }
            let delay = pow(2.0, Double(attempt)) // Exponential back‑off
            try await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000))
        }
    }
}

```

## Summary

- **LLMClient** operates as a stateless singleton in `altic-dev/FluidVoice`, providing centralized AI networking for transcription, command, and rewrite features.
- The architecture separates **configuration** (`LLMClient.Config`), **request building** (`buildRequest(_:)`), and **stream processing** (`processStreaming`).
- **SSE streaming** parses `data:` lines incrementally, extracting content deltas, thinking tokens, and fragmented tool calls.
- A **state machine** buffers partial tool call JSON until complete function arguments are assembled, then emits them via the configuration’s callback.
- **Exponential back‑off** retry logic ensures resilience against transient network failures during streaming sessions.

## Frequently Asked Questions

### How does LLMClient handle connection failures during streaming?

When a connection drops during an active SSE stream, the `executeWithRetry(request:config:)` method catches the network error and implements exponential back‑off according to the `RetryPolicy` specified in the configuration. The client will attempt reconnection up to the configured maximum, with delays doubling between attempts (2 seconds, 4 seconds, 8 seconds, etc.) before surfacing the final error to the caller.

### What is the difference between `processStreaming` and `processResponsesStreaming`?

`processStreaming(request:config:)` handles the legacy Chat‑Completions API format, parsing standard SSE deltas containing `choices` arrays. `processResponsesStreaming(request:config:)` targets the newer *Responses* API structure, which uses different JSON schemas for streaming outputs. Both methods share the same underlying SSE line-parsing logic but apply distinct JSON decoders to handle their respective API formats.

### How are tool calls extracted from SSE deltas?

Tool calls arrive fragmented across multiple SSE events. The LLMClient maintains an internal buffer dictionary indexed by the tool call ID. As each delta arrives containing partial JSON arguments, the `processToolCallDelta(_:config:)` method appends the fragment to the buffer. When the accumulated string forms valid JSON (detected by closing braces), the method decodes the complete `ToolCall` struct and emits it through the configuration’s event callback.

### Is LLMClient thread-safe for concurrent requests?

Yes. The singleton implementation uses Swift’s actor isolation and `URLSession`’s inherent thread safety to handle concurrent requests. The `call(_:)` method is async/await based, allowing multiple simultaneous streaming connections to different endpoints or models without blocking the main thread. Each request maintains its own isolated `Config` instance, preventing cross-contamination of state between concurrent operations.