# How FluidVoice Handles Threading for Audio Processing: A Complete Architecture Breakdown

> Discover how FluidVoice manages audio processing threading. Learn how dedicated dispatch queues prevent UI blocking and ensure safe audio object lifetimes for real-time performance.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: architecture
- Published: 2026-08-14

---

**FluidVoice isolates every stage of audio capture, processing, and shutdown on dedicated dispatch queues, ensuring the real-time Core Audio callback never blocks the UI and audio object lifetimes are safely managed.**

Audio processing demands strict separation between real-time signal paths and application logic. In `altic-dev/FluidVoice`, this separation is achieved through a carefully designed queue hierarchy that spans capture, lifecycle management, and resource cleanup. This article examines the threading architecture as implemented in the Swift source code, mapping each responsibility to its specific queue and quality-of-service level.

## Core Audio Capture and the Worker Queue

The **audio capture worker** bridges the gap between Core Audio's real-time callback and the application processing pipeline. In [`DirectCoreAudioInput.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioInput.swift), this worker runs on a dedicated high-priority queue:

```swift
// DirectCoreAudioInput.swift lines 401-404
workerQueue = DispatchQueue(
    label: "com.fluidvoice.audio.direct-input-consumer",
    qos: .userInteractive
)

```

The Core Audio callback itself performs minimal work: it writes PCM packets into a **lock-free ring buffer**. The heavy lifting happens when `start()` enqueues `consumePackets` onto `workerQueue`. This method repeatedly `peek`s packets, invokes the user-provided `packetHandler`, then `consume`s them — all without touching the real-time thread.

```swift
// Set up capture with packet handler
let handler: DirectCoreAudioPacketHandler = { samples, frames, rate, hostTime, sampleTime in
    // Resample, compute levels, feed ASR pipeline
}
let input = try DirectCoreAudioInput(deviceID: deviceID, packetHandler: handler)

// Schedules consumer on workerQueue
try input.start()

```

This design guarantees **real-time safety**: the callback never blocks, while processing still runs at `.userInteractive` priority to minimize latency.

## Lifecycle Serialization with the Controller Queue

Device state changes — preparation, start, stop, and format invalidation — must execute atomically to prevent race conditions. [`DirectCoreAudioLifecycleController.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioLifecycleController.swift) enforces this through `lifecycleQueue`:

```swift
// DirectCoreAudioLifecycleController.swift lines 62-66
lifecycleQueue = DispatchQueue(
    label: "com.fluidvoice.audio.direct-lifecycle",
    qos: .userInitiated
)

```

Every mutating operation passes through this serial queue. When you call `prepare()` or `start()`, the work suspends until the queue ensures no conflicting transitions are in progress.

```swift
let controller = DirectCoreAudioLifecycleController(
    packetHandler: handler,
    onFormatInvalidated: { invalidation in
        // Handle format change, e.g., recreate input
    }
)

// Serialized prepare and start
await controller.prepare(deviceID: deviceID, deviceName: "Built-in Mic", reason: "user_start")
await controller.start(deviceID: deviceID, deviceName: "Built-in Mic", reason: "user_start")

```

## Hardware Change Listeners on a Dedicated Queue

Core Audio property-change callbacks — for device liveness, format changes, and similar events — arrive on unpredictable threads. `DirectCoreAudioLifecycleController` registers listener blocks on `listenerQueue` to isolate these callbacks:

```swift
// DirectCoreAudioLifecycleController.swift lines 66-69
listenerQueue = DispatchQueue(
    label: "com.fluidvoice.audio.direct-format-listeners",
    qos: .userInitiated
)

```

Listener implementations are lightweight: they mark the format dirty and forward a `FormatInvalidation` event to the lifecycle controller. No processing occurs here — that work is deferred to `lifecycleQueue`.

## AVAudioEngine Retirement on Background Queues

`AVAudioEngine` deallocation can stall the calling thread due to internal synchronization. [`AudioEngineRetirementDrain.swift`](https://github.com/altic-dev/FluidVoice/blob/main/AudioEngineRetirementDrain.swift) solves this by moving engine release to a background queue:

```swift
// AudioEngineRetirementDrain.swift lines 31-33
queue = DispatchQueue(
    label: "app.fluidvoice.audio-engine-retirement",
    qos: .utility
)

```

Tokens wrapping retired engines are scheduled serially on this queue. Applications can fire-and-forget or await completion:

```swift
let engine = AVAudioEngine()
let token = AudioEngineRetirementToken(engine)

// Non-blocking retirement
AudioEngineRetirementDrain().schedule(token)

// Or wait for release before building new engine
await AudioEngineRetirementDrain().releaseAndWait(token)

```

## UI Feedback and General Background Work

Two additional queue patterns appear throughout the codebase:

- **Transcription sound playback**: [`TranscriptionSoundPlayer.swift`](https://github.com/altic-dev/FluidVoice/blob/main/TranscriptionSoundPlayer.swift) uses a `.userInteractive` queue (`playbackQueue`) to play UI feedback without main-thread stalls — see lines 8-10.

- **Global utility work**: Resampling, level calculation, and format validation dispatch to `DispatchQueue.global(qos: .utility)`, as shown in [`DirectCoreAudioLifecycleController.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioLifecycleController.swift) lines 78-80.

## Queue Hierarchy Summary

| Responsibility | Queue Location | QoS | Source File |
|----------------|---------------|-----|-------------|
| Packet drain from Core Audio | `com.fluidvoice.audio.direct-input-consumer` | `.userInteractive` | [`DirectCoreAudioInput.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioInput.swift) |
| Device lifecycle transitions | `com.fluidvoice.audio.direct-lifecycle` | `.userInitiated` | [`DirectCoreAudioLifecycleController.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioLifecycleController.swift) |
| Hardware property listeners | `com.fluidvoice.audio.direct-format-listeners` | `.userInitiated` | [`DirectCoreAudioLifecycleController.swift`](https://github.com/altic-dev/FluidVoice/blob/main/DirectCoreAudioLifecycleController.swift) |
| Engine deallocation | `app.fluidvoice.audio-engine-retirement` | `.utility` | [`AudioEngineRetirementDrain.swift`](https://github.com/altic-dev/FluidVoice/blob/main/AudioEngineRetirementDrain.swift) |
| UI feedback sounds | `app.fluidvoice.transcription-sounds` | `.userInteractive` | [`TranscriptionSoundPlayer.swift`](https://github.com/altic-dev/FluidVoice/blob/main/TranscriptionSoundPlayer.swift) |
| General background processing | `DispatchQueue.global(qos: .utility)` | `.utility` | Various |

## Summary

FluidVoice's threading architecture for audio processing rests on three principles:

- **Real-time callbacks stay minimal** — only ring buffer writes occur on the Core Audio thread.
- **State mutations are serialized** — `lifecycleQueue` prevents races during device transitions.
- **Expensive operations leave the main thread** — engine retirement and background processing run at appropriate QoS levels.

Each queue is purpose-built with explicit quality-of-service assignments, ensuring the UI remains responsive even under heavy audio load.

## Frequently Asked Questions

### What prevents the Core Audio callback from blocking in FluidVoice?

The callback performs only lock-free ring buffer writes. `DirectCoreAudioInput` drains packets on `workerQueue` (`.userInteractive` QoS), so heavy processing never executes on the real-time thread.

### Why does FluidVoice use a separate queue for AVAudioEngine deallocation?

`AVAudioEngine` teardown involves internal synchronization that can pause the caller. `AudioEngineRetirementDrain` moves this to a `.utility` queue, preventing UI stalls — particularly important when rapidly switching capture configurations.

### How does FluidVoice handle concurrent device format changes?

All format-change events flow through `listenerQueue`, which delegates to `lifecycleQueue`. Because `lifecycleQueue` is serial, format invalidations and lifecycle transitions (start/stop/prepare) execute atomically without races against packet processing.

### What QoS level does FluidVoice use for audio processing, and why?

Packet handling uses `.userInteractive` to minimize latency, while lifecycle coordination uses `.userInitiated` and background work uses `.utility`. This hierarchy matches Apple's guidance: audio that affects immediate perception gets highest priority, while maintenance tasks defer appropriately.