# How FluidVoice Manages Audio Input for Transcription: A Deep Dive into ASRService

> Discover how FluidVoice uses ASRService to manage audio input for transcription including microphone permissions, real-time buffering, and thread-safe sample delivery to transcription providers.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: deep-dive
- Published: 2026-06-30

---

**FluidVoice manages audio input for transcription through the `ASRService` class, which orchestrates microphone permissions, real-time buffering via `AudioCapturePipeline`, and thread-safe sample delivery to multiple transcription providers.**

The open-source **FluidVoice** repository (`altic-dev/FluidVoice`) implements a sophisticated speech-to-text pipeline designed for low-latency dictation. Understanding how this Swift-based application captures and processes microphone input reveals best practices for bridging CoreAudio's real-time threads with modern Swift concurrency.

## The ASRService Architecture

At the center of FluidVoice's audio management strategy sits the **`ASRService`** class defined in [`Sources/Fluid/Services/ASRService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift). This singleton-style service owns the entire lifecycle of audio capture, from permission checks to final transcription delivery.

### Initialization and Permission Handling

Before capturing audio, `ASRService.initialize()` (lines [729-734](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L729-L734)) verifies microphone access and prepares the audio subsystem:

- Queries `AVCaptureDevice.authorizationStatus(for: .audio)` and stores the result in `micStatus`
- Registers listeners for audio route changes
- Preloads cached ASR models to minimize latency during recording startup

This initialization ensures the app respects user privacy while maintaining readiness for instant dictation triggers.

### Starting the Recording Session

When the user initiates dictation, `await asrService.start()` (lines [842-870](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L842-L870)) executes a sequential setup process:

1. **Permission validation** – Returns immediately if `micStatus != .authorized`
2. **State reset** – Clears `finalText`, `audioBuffer`, and previous transcription state
3. **Session configuration** – Calls `configureSession()` to set the `AVAudioSession` category and sample rate
4. **Engine startup** – Lazily instantiates `AVAudioEngine` and invokes `engine.start()`
5. **Tap installation** – `setupEngineTap()` adds a real-time tap on the input node that streams raw PCM to the **audio capture pipeline**
6. **Streaming activation** – If the selected model supports it, `startStreamingTranscription()` begins feeding samples to the `TranscriptionProvider`

## Real-Time Audio Capture Pipeline

The **`AudioCapturePipeline`** (inner class within [`ASRService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/ASRService.swift), lines [618-629](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L618-L629)) handles the critical path between CoreAudio's real-time thread and the app's transcription logic. This component operates under `NSLock` protection to prevent race conditions between audio capture and Swift concurrency.

### Format Conversion and Resampling

Raw audio from `AVAudioEngine` rarely arrives in the format required by modern ASR models. The pipeline's `toMono16k` method (lines [274-286](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L274-L286)) performs essential preprocessing:

- **Direct copy** when the buffer is already 16 kHz mono Float32
- **Down-mixing** to mono via `downmixToMono` for multi-channel inputs
- **Resampling** to 16 kHz via `resampleTo16k` for high-sample-rate devices

Simultaneously, the pipeline computes audio levels using Accelerate framework's `vDSP_svesq` (lines [335-363](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L335-L363)), converting RMS values to normalized decibel readings for UI visualization via `audioLevelSubject`.

### Thread-Safe Buffering

Converted samples land in the **`ThreadSafeAudioBuffer`** ([`Sources/Fluid/Services/ThreadSafeAudioBuffer.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ThreadSafeAudioBuffer.swift), lines [5-22](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ThreadSafeAudioBuffer.swift#L5-L22)), an `NSLock`-protected wrapper around `[Float]`. This buffer provides atomic operations:

- `append(_:)` – Called by the real-time audio thread
- `getPrefix(_:)` and `clear()` – Called by the transcription task
- `getAll()` – Retrieves complete recording for final processing

This architecture eliminates data races while maintaining the throughput necessary for real-time transcription.

## Streaming Transcription Flow

Once recording begins, `startStreamingTranscription()` launches a persistent `Task` that bridges the buffer to ASR providers. The workflow operates in a tight loop:

1. Extracts pending samples from `audioBuffer` using `getPrefix`
2. Invokes `transcriptionProvider.transcribeStreaming(_:)` with the `[Float]` array
3. Updates `partialTranscription` and publishes results to the UI

The **`TranscriptionProvider`** protocol (implemented in files like [`FluidAudioProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/FluidAudioProvider.swift) and [`WhisperProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/WhisperProvider.swift)) abstracts multiple backends including **FluidAudio**, **Parakeet**, **Whisper**, **AppleSpeech**, and **Nemotron**. Provider selection occurs via the computed property at lines [667-685](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L667-L685) based on `SettingsStore.shared.selectedSpeechModel`.

## Stopping and Finalizing Transcription

When dictation ends, `await asrService.stop()` (lines [998-1064](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift#L998-L1064)) performs graceful teardown:

- Disables the capture pipeline via `audioCapturePipeline.setRecordingEnabled(false)`
- Stops `AVAudioEngine` and removes the input tap
- Awaits completion of the streaming task with `stopStreamingTimerAndAwait`
- Retrieves remaining samples via `audioBuffer.getAll()`, applies padding for short recordings, and executes final transcription via `transcriptionProvider.transcribe(_:)`

This ensures no audio samples are lost between the last streaming chunk and the recording termination.

## UI Integration and Audio Visualization

FluidVoice's interface components consume audio levels through **`audioLevelPublisher`**, a Combine publisher exposed by `ASRService`. Views like [`BottomOverlayView.swift`](https://github.com/altic-dev/FluidVoice/blob/main/BottomOverlayView.swift) and [`CommandModeView.swift`](https://github.com/altic-dev/FluidVoice/blob/main/CommandModeView.swift) subscribe to these normalized CGFloat values (ranging 0.0 to 1.0) to drive animated waveform visualizations that provide immediate feedback during speech capture.

```swift
import Fluid

// Initialize the transcription service
let asr = ASRService()
await asr.initialize()

// Begin recording when user activates dictation
Task {
    await asr.start()
}

// Drive UI waveform with real-time audio levels
let cancellable = asr.audioLevelPublisher
    .sink { level in
        // level is CGFloat [0,1] representing normalized dB
        waveformView.updateAmplitude(level)
    }

// Stop and retrieve final text
Task {
    let transcription = await asr.stop()
    textEditor.insert(transcription)
}

```

## Summary

- **ASRService** ([`ASRService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/ASRService.swift)) centralizes microphone permission handling, engine configuration, and transcription orchestration
- **AudioCapturePipeline** converts incoming `AVAudioPCMBuffer` to 16 kHz mono Float32 while computing RMS levels for visualization
- **ThreadSafeAudioBuffer** provides lock-protected storage shared between real-time audio threads and Swift concurrency transcription tasks
- Multiple **TranscriptionProvider** implementations (Whisper, FluidAudio, etc.) consume raw `[Float]` samples through a unified streaming interface
- The system uses `AVAudioEngine` input taps protected by `NSLock` to safely bridge CoreAudio callbacks with async/await transcription logic

## Frequently Asked Questions

### How does FluidVoice handle microphone permissions?

FluidVoice queries `AVCaptureDevice.authorizationStatus(for: .audio)` during `ASRService.initialize()` (lines 729-734) and stores the result in the `micStatus` property. The `start()` method guards against unauthorized access, ensuring the app complies with iOS privacy requirements before activating `AVAudioEngine`.

### What audio format does FluidVoice use for transcription?

The system standardizes all input to **16 kHz mono Float32** samples. The `AudioCapturePipeline.toMono16k` method (lines 274-286) automatically down-mixes multi-channel audio and resamples higher sample rates, ensuring compatibility with ASR providers while minimizing processing overhead.

### How does FluidVoice prevent audio data loss during transcription?

The **`ThreadSafeAudioBuffer`** class wraps a `[Float]` array with an `NSLock` to synchronize access between the real-time CoreAudio tap (which appends samples) and the transcription `Task` (which reads and clears chunks). This lock-based approach prevents race conditions without blocking the audio thread for significant durations.

### Can FluidVoice transcribe audio in real-time while recording?

Yes, when using models that support streaming (checked via `SettingsStore.shared.selectedSpeechModel.supportsStreaming`), `ASRService` launches a background task via `startStreamingTranscription()`. This task repeatedly pulls samples from the thread-safe buffer and pushes them to the provider's `transcribeStreaming(_:)` method, updating partial results on the main actor for immediate UI feedback.