# How FluidVoice Implements the TranscriptionProvider Protocol for Pluggable Speech Recognition

> FluidVoice uses the TranscriptionProvider protocol to integrate pluggable speech recognition, supporting Whisper GGUF and Apple's native APIs for flexible backends.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: deep-dive
- Published: 2026-08-14

---

**FluidVoice abstracts all speech-to-text functionality behind a single Swift protocol called `TranscriptionProvider`, enabling runtime-swappable backends ranging from Whisper GGUF models to Apple's native speech APIs.**

The `TranscriptionProvider` protocol in the FluidVoice repository (`altic-dev/FluidVoice`) defines a clean contract that any speech recognition engine must satisfy. Located at [`Sources/Fluid/Services/TranscriptionProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/TranscriptionProvider.swift), this protocol enables the app to switch between seven distinct transcription backends without modifying UI code.

## Core Protocol Requirements in TranscriptionProvider.swift

The protocol mandates four categories of functionality that every provider must implement:

### Identity and Availability Reporting

Each provider exposes runtime metadata through three computed properties:

- `name` – A human-readable string identifying the backend (e.g., "Whisper", "Apple Speech")
- `isAvailable` – Whether the provider can execute on the current hardware and OS version
- `isReady` – Whether the provider's models are loaded and transcription can begin immediately

These properties allow `ASRService` to filter unavailable providers and display appropriate options in the settings UI.

### Model Lifecycle Management

Providers handle their own asset preparation through:

```swift
func prepare(progressHandler: ((ModelPreparationProgress) -> Void)?) async throws
func clearCache() async
func modelsExistOnDisk() -> Bool

```

The `prepare(progressHandler:)` method downloads, verifies, and loads model files. `WhisperProvider` uses this to fetch GGUF files from HuggingFace, reporting progress through the `ModelPreparationProgress` struct defined in the same file. Cache inspection and cleanup enable the app to manage disk usage without provider-specific logic.

### Transcription Operations

The core transcription method signature:

```swift
func transcribe(_ samples: [Float]) async throws -> ASRTranscriptionResult

```

This accepts 16 kHz mono PCM audio samples and returns structured results. The protocol provides default implementations for four specialized variants:

- `transcribeStreaming(_:)` – For real-time dictation with partial results
- `transcribeFinal(_:)` – For optimized final-pass transcription
- `transcribeDictionaryTraining(_:)` – For vocabulary customization workflows
- `transcribeFile(at:)` – For direct file-level processing

Default fallbacks route these to the base `transcribe(_:)` method, but providers can override for optimized paths.

### Behavioral Configuration Flags

Two boolean flags guide `ASRService` orchestration:

- `shouldClearCacheAfterCancellation` – Whether to purge model files when transcription is interrupted
- `prefersNativeFileTranscription` – Whether to use `transcribeFile(at:)` instead of loading audio into memory

## Concrete Provider Implementations

FluidVoice ships seven providers conforming to `TranscriptionProvider`, each targeting a specific backend architecture:

### WhisperProvider

**[`Sources/Fluid/Services/WhisperProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/WhisperProvider.swift)**

Wraps **transcribe.cpp** for local Whisper inference using GGUF quantized models. Key characteristics:

- Downloads model files from HuggingFace Hub with integrity verification
- Automatically selects Metal GPU on Apple Silicon, CPU fallback on Intel
- Performs memory availability checks before model loading
- Reports granular preparation progress including download, decompression, and initialization phases

### FluidAudioProvider

**[`Sources/Fluid/Services/FluidAudioProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/FluidAudioProvider.swift)**

Integrates the proprietary **FluidAudio** library optimized for Apple Silicon. Focuses on low-latency streaming transcription with minimal CPU overhead.

### ParakeetRealtimeProvider

**[`Sources/Fluid/Services/ParakeetRealtimeProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ParakeetRealtimeProvider.swift)**

Implements true streaming transcription through the **Parakeet** end-of-utterance (EOU) pipeline. Suitable for continuous dictation scenarios requiring immediate partial results.

### AppleSpeechProvider

**[`Sources/Fluid/Services/AppleSpeechProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/AppleSpeechProvider.swift)**

Wraps Apple's public **SFSpeechRecognizer** API. Handles permission requests, locale selection, and network-dependent recognition automatically.

### AppleSpeechAnalyzerProvider

**[`Sources/Fluid/Services/AppleSpeechAnalyzerProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/AppleSpeechAnalyzerProvider.swift)**

Targets **SpeechAnalyzer**, a newer API available starting macOS 26. Provides improved accuracy over `SFSpeechRecognizer` for supported systems.

### ExternalCoreMLTranscriptionProvider

**[`Sources/Fluid/Services/ExternalCoreMLTranscriptionProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ExternalCoreMLTranscriptionProvider.swift)**

Loads user-supplied or bundled **Core ML** models. Supports native file-wise transcription when `prefersNativeFileTranscription` returns `true`, avoiding unnecessary audio decoding.

### NemotronProvider

**[`Sources/Fluid/Services/NemotronProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/NemotronProvider.swift)**

Wraps the **Nemotron** large language model inference engine for high-quality transcription. Includes optional vocabulary rescoring for domain-specific accuracy improvements.

## Runtime Provider Selection via ASRService

The `ASRService` class at [`Sources/Fluid/Services/ASRService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ASRService.swift) orchestrates provider instantiation without hardcoding specific implementations:

```swift
// ASRService constructs providers based on SettingsStore selection
let asr = ASRService()

// Access the currently selected provider
let provider = asr.transcriptionProvider

// Prepare with progress UI feedback
await provider.prepare { progress in
    print("Phase: \(progress.phase), Progress: \(progress.fractionCompleted ?? 0)")
}

// Execute transcription
let samples: [Float] = // 16 kHz mono PCM buffer
let result = try await provider.transcribe(samples)

```

Provider selection persists through `SettingsStore` ([`Sources/Fluid/Persistence/SettingsStore.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Persistence/SettingsStore.swift)), which serializes the user's choice to `UserDefaults`. When the user changes models or switches providers, `ASRService.resetTranscriptionProvider()` (referenced in [`WelcomeView.swift`](https://github.com/altic-dev/FluidVoice/blob/main/WelcomeView.swift)) disposes the existing instance and constructs a replacement.

## Default Implementation Strategy

The protocol extension in [`TranscriptionProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/TranscriptionProvider.swift) supplies no-op implementations for optional behaviors:

```swift
// Providers only override what they optimize
extension TranscriptionProvider {
    func transcribeStreaming(_ samples: [Float]) async throws -> ASRTranscriptionResult {
        try await transcribe(samples)  // Default: treat as regular transcription
    }
    
    var shouldClearCacheAfterCancellation: Bool { false }
    var prefersNativeFileTranscription: Bool { false }
}

```

This design minimizes boilerplate—`AppleSpeechProvider` implements only essential methods, while `WhisperProvider` overrides preparation and caching for its GGUF lifecycle.

## Summary

- **`TranscriptionProvider`** at [`Sources/Fluid/Services/TranscriptionProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/TranscriptionProvider.swift) defines the complete contract for speech recognition backends
- **Seven concrete implementations** cover local inference (Whisper, Nemotron), Apple APIs (SFSpeechRecognizer, SpeechAnalyzer), proprietary libraries (FluidAudio, Parakeet), and custom Core ML models
- **`ASRService`** manages provider lifecycle and exposes a unified API to SwiftUI views
- **Default protocol implementations** reduce provider boilerplate while preserving optimization opportunities
- **Runtime switching** through `SettingsStore` and `resetTranscriptionProvider()` enables user-controlled backend selection without code changes

## Frequently Asked Questions

### What audio format does TranscriptionProvider expect?

All `transcribe(_:)` implementations receive `[Float]` arrays containing **16 kHz mono PCM samples**. Providers that need different formats—such as `ExternalCoreMLTranscriptionProvider` with `prefersNativeFileTranscription`—can opt into file-level processing to handle conversion internally.

### Can I add a custom transcription backend to FluidVoice?

Yes. Create a new Swift class conforming to `TranscriptionProvider`, implement the required properties and methods, then register it in [`ASRService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/ASRService.swift). The default implementations in the protocol extension handle streaming and final-pass fallbacks automatically.

### How does FluidVoice handle model downloads for WhisperProvider?

`WhisperProvider.prepare(progressHandler:)` downloads GGUF files from HuggingFace using `URLSession` with background transfer support. Progress updates flow through `ModelPreparationProgress`, which includes download fraction, decompression status, and Metal shader compilation phases. Failed downloads automatically retry with exponential backoff.

### Why does ASRService exist instead of calling providers directly?

`ASRService` decouples UI code from provider-specific initialization logic. It reads `SettingsStore` preferences, constructs the appropriate provider instance, manages memory pressure notifications, and coordinates model preparation across view lifecycle changes. This centralization prevents view controllers from holding stateful provider references.