How AppleSpeechProvider Works in FluidVoice: Architecture, Implementation, and Advantages

AppleSpeechProvider wraps macOS's native SFSpeechRecognizer to provide instant, zero-download speech transcription for FluidVoice users on macOS 10.15 and later. It implements the app's generic TranscriptionProvider protocol, converting raw audio buffers into text using the operating system's built-in speech models without requiring external model downloads.

FluidVoice (altic-dev/FluidVoice) ships with a built-in transcription backend called AppleSpeechProvider that leverages macOS system capabilities for local speech recognition. Unlike cloud-based or neural engine providers that require downloading large language models, this provider relies entirely on the operating system's native speech framework. This architecture makes it immediately available on compatible Macs while maintaining a minimal memory footprint, ideal for users on older hardware or those prioritizing privacy.

Core Architecture and Implementation

AppleSpeechProvider is defined in Sources/Fluid/Services/AppleSpeechProvider.swift and serves as the legacy speech recognition implementation for macOS versions preceding the newer Apple Speech Analyzer API.

Protocol Conformance and Availability Checks

The provider conforms to FluidVoice's TranscriptionProvider protocol, declaring name = "Apple Speech (Legacy)" and determining availability through SFSpeechRecognizer.authorizationStatus(). It maintains a lazy-initialized SFSpeechRecognizer? property that is recreated whenever the user changes the selected locale via SettingsStore.shared.selectedAppleSpeechLocale.

The prepare(progressHandler:) method handles the authorization flow by calling requestAuthorization(), updating the provider's isReady state based on the returned authorization status. This ensures the provider only attempts transcription after securing user permission through the standard macOS privacy dialog.

Audio Pipeline and Buffer Conversion

FluidVoice captures raw audio as [Float] arrays at 16 kHz mono. The provider's createPCMBuffer(from:) method converts these samples into AVAudioPCMBuffer instances compatible with SFSpeechRecognizer. This conversion uses AVAudioFormat configured for the specific sample rate and channel layout, performing a fast memory copy via UnsafeBufferPointer to minimize overhead during real-time transcription.

Async/Await Bridging

Because SFSpeechRecognizer uses a callback-based API, AppleSpeechProvider bridges it to Swift's async/await pattern using withCheckedThrowingContinuation. The transcribe(_:) method constructs an SFSpeechAudioBufferRecognitionRequest, appends the converted PCM buffer, and initiates a recognitionTask. Results are wrapped in ASRTranscriptionResult objects, while errors such as missing permissions, unavailable recognizers, or silent audio are converted into clear NSError domains and logged via DebugLogger.

Integration with FluidVoice's ASRService

The provider integrates with the rest of the application through ASRService (defined in Sources/Fluid/Services/ASRService.swift). This central service acts as a router, selecting the appropriate transcription backend based on user preferences stored in SettingsStore.shared.selectedSpeechModel.

When a user selects Apple Speech in the settings UI, or when the newer Apple Speech Analyzer is unavailable (pre-macOS 26), ASRService returns an instance from getAppleSpeechProvider() through its provider switch case. The service delegates all transcription calls to this instance while handling lifecycle management and error propagation.

The user-visible description displayed in the Voice Engine settings panel comes from VoiceEngineSettingsViewModel.modelDescriptionText (in Sources/Fluid/UI/AISettings/VoiceEngineSettingsViewModel.swift), which returns explanatory text about the legacy provider using built-in macOS speech recognition.

Key Advantages of AppleSpeechProvider

Using the system speech framework provides several distinct benefits over third-party or downloadable model approaches:

• Zero-download readiness – The provider reports modelsExistOnDisk = true immediately because it uses macOS's bundled speech models. No large files are downloaded or cached in FluidVoice's application support directories.

• Cross-architecture compatibility – Runs natively on both Intel and Apple Silicon Macs running macOS 10.15 (Catalina) or later, with no separate builds required for different chipsets.

• Privacy-first processing – Speech recognition occurs locally on device when possible. FluidVoice never ships its own acoustic models or maintains transcription caches, leaving network fallback decisions to the operating system's privacy controls.

• Low memory footprint – Because the heavy lifting and model storage reside in the OS, the provider maintains minimal heap allocation during transcription, making it suitable for older Macs with limited RAM.

• Broad language support – Automatically inherits all languages supported by the macOS system speech recognizer, updating as Apple expands language coverage through OS updates.

• Graceful fallback – On macOS 26 and later, FluidVoice can upgrade users to AppleSpeechAnalyzerProvider while keeping the legacy provider available as a compatibility option for older systems.

Usage Examples

Direct Provider Usage

You can instantiate and use the provider directly for low-level transcription tasks:

import Fluid

func transcribeAudioBuffer(_ samples: [Float]) async throws -> String {
    let provider = AppleSpeechProvider()
    
    // Request macOS microphone/speech recognition permission
    try await provider.prepare(progressHandler: nil)
    
    // Transcribe 16kHz mono Float array
    let result = try await provider.transcribe(samples)
    return result.text
}

Note that prepare triggers the system permission dialog if the user hasn't previously granted speech recognition access. The transcribe method accepts raw Float arrays and returns structured results containing the recognized text and confidence scores.

Via ASRService

For most applications, use the high-level ASRService which handles provider selection automatically:

import Fluid

func performDictation(_ audioSamples: [Float]) async throws -> String {
    let asrService = ASRService.shared
    
    // Prepares the currently selected provider (AppleSpeechProvider if selected)
    try await asrService.ensureAsrReady()
    
    // ASRService routes to AppleSpeechProvider internally based on SettingsStore
    let result = try await asrService.transcribe(audioSamples)
    return result.text
}

Under the hood, ASRService reads SettingsStore.shared.selectedSpeechModel, resolves it to the appropriate provider instance, and manages the transcription lifecycle.

Runtime Provider Switching

Switch between speech providers programmatically based on OS availability:

// Select legacy Apple Speech
SettingsStore.shared.selectedSpeechModel = .appleSpeech

// On newer systems, upgrade to the analyzer API
if #available(macOS 26.0, *) {
    SettingsStore.shared.selectedSpeechModel = .appleSpeechAnalyzer
}

Changes to selectedSpeechModel immediately affect subsequent transcription requests, and the settings UI updates its description text automatically through VoiceEngineSettingsViewModel.

Summary

• AppleSpeechProvider wraps SFSpeechRecognizer in Sources/Fluid/Services/AppleSpeechProvider.swift to provide system-level transcription.
• It requires no model downloads and reports immediate readiness through modelsExistOnDisk = true.
• Audio conversion happens via createPCMBuffer(from:), bridging raw Float arrays to AVAudioPCMBuffer for the system API.
• ASRService routes transcription requests to this provider when users select Apple Speech or when running on pre-macOS 26 systems.
• The provider offers cross-architecture support, local processing, and minimal memory usage compared to neural model-based alternatives.

Frequently Asked Questions

What macOS version is required for AppleSpeechProvider?

AppleSpeechProvider requires macOS 10.15 (Catalina) or later, as it depends on the SFSpeechRecognizer framework introduced in that version. The provider automatically checks availability through SFSpeechRecognizer.authorizationStatus() before initializing.

Does AppleSpeechProvider download speech models?

No. AppleSpeechProvider uses the speech recognition models bundled with macOS. It sets modelsExistOnDisk = true immediately upon initialization and never downloads additional files, making it ideal for offline use or metered internet connections.

How does AppleSpeechProvider handle permissions?

The provider requests speech recognition authorization through the standard macOS privacy framework. The prepare(progressHandler:) method calls requestAuthorization(), and the provider becomes ready only after the user grants permission through the system dialog. Errors from denied permissions are converted to specific NSError domains for proper error handling in the UI.

What is the difference between AppleSpeechProvider and AppleSpeechAnalyzerProvider?

AppleSpeechProvider (the "Legacy" provider) uses the traditional SFSpeechRecognizer API available since macOS 10.15. AppleSpeechAnalyzerProvider (available in macOS 26+) uses a newer Apple speech analysis API with potentially improved accuracy and features. FluidVoice automatically falls back to AppleSpeechProvider when the analyzer API is unavailable, ensuring compatibility across macOS versions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}