PrivateAIIntegrationService in FluidVoice: How Local AI Enhancement Works

PrivateAIIntegrationService is an actor-based singleton that mediates between FluidVoice’s dictation pipeline and a locally-hosted LLM, providing a clean async API for model loading, pre-warming, and text enhancement while falling back to a no-op shim when the private AI feature is unavailable.

FluidVoice is an open-source dictation application that leverages on-device AI to improve transcription quality without transmitting data to external servers. The PrivateAIIntegrationService serves as the central nervous system for this capability, abstracting the complexity of local inference behind a thread-safe, Swift-native interface. Implemented in Sources/Fluid/Services/PrivateAIIntegrationService.swift, this service coordinates model discovery, downloading, and runtime management to deliver low-latency text enhancement.

What Is PrivateAIIntegrationService?

PrivateAIIntegrationService is a Swift actor that functions as a singleton, accessible via PrivateAIIntegrationService.shared. It discovers whether a private AI backend is compiled into the app by checking PrivateAIProviderFeature.shared.isAvailable. When the feature is present, the service forwards all calls to the concrete implementation held in PrivateAIProviderRegistry.integration; otherwise, it falls back to UnavailableAIIntegrationShim, a no-op implementation that simply echoes the original dictation text to ensure the app continues functioning.

This architecture hides the complexity of model handling, runtime configuration, and error handling behind a unified facade. The service is referenced throughout the dictation pipeline, including in DictationPostProcessingService for post-processing transcriptions and ContentView for UI-level enhancements.

Public API Methods

The service exposes a focused set of async methods defined in PrivateAIIntegrationService.swift:

• status(for:) – Returns a PrivateAIStatus enum indicating runtime states including unavailable, missingModel, loading, or ready.

• loadModel(_:) – Instructs the backend to initialize a specific model, transitioning through download, verification, and runtime boot phases.

• prewarmDictation() – Optimizes first-request latency by pre-loading the KV-cache (prefix-cache) before actual dictation begins.

• enhanceDictation(_:runtime:context:) – Accepts raw transcription text and returns an EnhancementResult containing corrected punctuation, smart completions, and latency metrics.

• unloadCachedRuntime(reason:) and shutdownForTermination() – Provide clean teardown of the local inference engine when the app quits or the user disables the feature.

The concrete implementation relies on the PrivateAIIntegrationProviding protocol defined in PrivateAIProvider.swift, which supplies configuredModelID, selectedModel, modelDirectoryURL, and isLocalRuntimeConfigured properties.

How Local AI Enhancement Works

Local AI enhancement follows a five-phase pipeline orchestrated by the integration service:

Model Discovery and Selection

The UI persists the user's chosen model ID in UserDefaults using the key provided by PrivateAIProviderFeature.shared.selectedModelDefaultsKey. The service retrieves this via the provider's configuredModelID property, which maps to a PrivateAIRegisteredModel describing the artifact's filename, download URL, and SHA-256 hash.

Model Preparation and Verification

When a model is first needed, prepareModel(_:progressHandler:) validates the local filesystem against the expected path returned by expectedLocalModelURL(for:). If the model file is missing, the provider downloads the artifact using the downloadURL from the registered model. Progress updates flow through the PrivateAIModelDownloadProgressHandler closure. After download, the file is verified against its SHA-256 hash and moved into the model directory located at Application Support/FluidVoice/<ProviderName>/Models.

Runtime Initialization

Once the model file is local and verified, loadModel(_:) triggers the provider's concrete implementation—typically booting a local inference bridge such as FluidIntelligence. The runtime transitions through loading to ready states, exposed via the isLocalRuntimeConfigured flag.

Pre-warming for Low Latency

To eliminate cold-start delays, prewarmDictation() optionally initializes a prefix-cache (KV-cache). This primes the inference engine so the first live dictation request incurs minimal token-generation latency, ensuring a responsive user experience.

Dictation Enhancement Pipeline

During active dictation, enhanceDictation(_:runtime:context:) forwards the raw Whisper transcription alongside a RuntimeConfiguration (containing provider ID, model identifier, API key, and local path) and an AppContext (app name, bundle ID, window title). The provider returns an EnhancementResult containing:

• outputText – the corrected/enhanced transcription
• backendKind – identifier string (e.g., "private-ai")
• latencyMilliseconds – optional performance metric

If the private AI feature is unavailable (when #if PRIVATE_AI_PROVIDER is false), the service routes calls to UnavailableAIIntegrationShim, which immediately returns the original text unchanged.

Key Source Files and Implementation Details

The local AI enhancement pipeline is distributed across the following files:

• Sources/Fluid/Services/PrivateAIIntegrationService.swift – Core actor that routes all private-AI operations and manages the enhancement lifecycle.

• Sources/Fluid/Services/PrivateAIProvider.swift – Defines the PrivateAIIntegrationProviding protocol, the provider registry, and model registration logic.

• Sources/Fluid/Services/PrivateAIProviderFeature.swift – Supplies feature-availability checks and configuration keys for UserDefaults.

• Sources/Fluid/Services/UnavailableAIIntegrationShim.swift – No-op fallback implementation used when the private-AI backend is not compiled into the binary.

• Sources/Fluid/Services/DictationPostProcessingService.swift – Integration point that calls enhanceDictation after initial Whisper transcription.

• Sources/Fluid/ContentView.swift – UI entry point that invokes the enhancement service for on-demand text improvements.

Practical Implementation Examples

Checking Private AI Availability

let isAvailable = PrivateAIProviderFeature.shared.isAvailable
print("Private AI available? \(isAvailable)")

Source: PrivateAIProviderFeature.shared in [PrivateAIProvider.swift](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIProvider.swift)

Loading a Model and Pre-warming the Runtime

let service = PrivateAIIntegrationService.shared
let model = PrivateAIProviderFeature.shared.model(id: "gpt-4-mini")!

Task {
    // Prepare/download the model (optional progress handler)
    let localURL = try await service.prepareModel(model) { progress in
        print("Download progress: \(progress.fractionCompleted ?? 0)")
    }

    // Load the model into the local runtime
    let status = try await service.loadModel(model)
    print("Load status: \(status.state) – \(status.message ?? "")")

    // Warm-up the dictation engine
    await service.prewarmDictation()
}

Source: prepareModel and loadModel in [PrivateAIIntegrationService.swift](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift)

Enhancing Dictation with Local AI

let runtime = PrivateAIIntegrationService.RuntimeConfiguration(
    selectedProviderID: PrivateAIProviderFeature.shared.providerID,
    providerKey: "my-local-key",
    baseURL: "",                     // unused for local runtime
    model: "gpt-4-mini",
    apiKey: "",                      // unused for local runtime
    localModelPath: nil,
    usesStablePromptPrefixKVCache: true
)

let context = PrivateAIIntegrationService.AppContext(
    appName: "FluidVoice",
    bundleID: Bundle.main.bundleIdentifier!,
    windowTitle: "Notes",
    appVersion: Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String
)

Task {
    do {
        let result = try await service.enhanceDictation(
            "hey whats the weather today",
            runtime: runtime,
            context: context
        )
        print("Enhanced: \(result.outputText)")
    } catch {
        print("Enhancement failed: \(error)")
    }
}

Source: enhanceDictation in [PrivateAIIntegrationService.swift](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift)

UI Integration from ContentView

let response = try await PrivateAIIntegrationService.shared.enhanceDictation(
    rawTranscription,
    runtime: currentRuntime,
    context: appContext
)

Source: [ContentView.swift](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/ContentView.swift)

Summary

• PrivateAIIntegrationService is an actor-based singleton that abstracts local LLM complexity behind a unified async API.
• The service automatically detects feature availability via PrivateAIProviderFeature.shared.isAvailable and falls back to a no-op shim when unavailable.
• The enhancement pipeline follows a strict sequence: model discovery → download and verification → runtime initialization → pre-warming → dictation enhancement.
• Local models are stored in Application Support/FluidVoice/<ProviderName>/Models and verified using SHA-256 hashes.
• Pre-warming via prewarmDictation() minimizes first-request latency by initializing the KV-cache before user interaction.

Frequently Asked Questions

What happens if the private AI feature is not compiled into the app?

The service routes all calls to UnavailableAIIntegrationShim, which immediately returns the original dictation text unchanged. This ensures FluidVoice continues to function normally for users who have not enabled or compiled the private AI backend.

Where are local model files stored on disk?

Model files are stored in the Application Support directory under FluidVoice/<ProviderName>/Models. The exact path is accessible via the provider's modelDirectoryURL and expectedLocalModelURL(for:) methods, which construct platform-appropriate URLs for the current user's home directory.

How does the service verify model integrity after downloading?

During prepareModel(_:progressHandler:), the service downloads the model using the downloadURL and sha256 hash specified in PrivateAIRegisteredModel. After download completes, it verifies the file against the SHA-256 hash before moving it to the final model directory, ensuring the local runtime loads only verified artifacts.

Is PrivateAIIntegrationService thread-safe?

Yes, the service is implemented as a Swift actor, which guarantees that all state mutations and internal operations are serialized and thread-safe. This design allows safe concurrent access from multiple UI components and background tasks using Swift's structured concurrency model.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}