# PrivateAIIntegrationService in FluidVoice: How Local AI Enhancement Works

> Discover how PrivateAIIntegrationService enables local AI enhancement in FluidVoice. Understand its seamless integration with LLMs and graceful fallback mechanisms for a private dictation experience.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: internals
- Published: 2026-07-03

---

**PrivateAIIntegrationService is an actor-based singleton that mediates between FluidVoice’s dictation pipeline and a locally-hosted LLM, providing a clean async API for model loading, pre-warming, and text enhancement while falling back to a no-op shim when the private AI feature is unavailable.**

FluidVoice is an open-source dictation application that leverages on-device AI to improve transcription quality without transmitting data to external servers. The `PrivateAIIntegrationService` serves as the central nervous system for this capability, abstracting the complexity of local inference behind a thread-safe, Swift-native interface. Implemented in [`Sources/Fluid/Services/PrivateAIIntegrationService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift), this service coordinates model discovery, downloading, and runtime management to deliver low-latency text enhancement.

## What Is PrivateAIIntegrationService?

`PrivateAIIntegrationService` is a Swift **actor** that functions as a singleton, accessible via `PrivateAIIntegrationService.shared`. It discovers whether a private AI backend is compiled into the app by checking `PrivateAIProviderFeature.shared.isAvailable`. When the feature is present, the service forwards all calls to the concrete implementation held in `PrivateAIProviderRegistry.integration`; otherwise, it falls back to `UnavailableAIIntegrationShim`, a no-op implementation that simply echoes the original dictation text to ensure the app continues functioning.

This architecture hides the complexity of model handling, runtime configuration, and error handling behind a unified facade. The service is referenced throughout the dictation pipeline, including in `DictationPostProcessingService` for post-processing transcriptions and `ContentView` for UI-level enhancements.

## Public API Methods

The service exposes a focused set of async methods defined in [`PrivateAIIntegrationService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/PrivateAIIntegrationService.swift):

- **`status(for:)`** – Returns a `PrivateAIStatus` enum indicating runtime states including `unavailable`, `missingModel`, `loading`, or `ready`.

- **`loadModel(_:)`** – Instructs the backend to initialize a specific model, transitioning through download, verification, and runtime boot phases.

- **`prewarmDictation()`** – Optimizes first-request latency by pre-loading the KV-cache (prefix-cache) before actual dictation begins.

- **`enhanceDictation(_:runtime:context:)`** – Accepts raw transcription text and returns an `EnhancementResult` containing corrected punctuation, smart completions, and latency metrics.

- **`unloadCachedRuntime(reason:)`** and **`shutdownForTermination()`** – Provide clean teardown of the local inference engine when the app quits or the user disables the feature.

The concrete implementation relies on the `PrivateAIIntegrationProviding` protocol defined in [`PrivateAIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/PrivateAIProvider.swift), which supplies `configuredModelID`, `selectedModel`, `modelDirectoryURL`, and `isLocalRuntimeConfigured` properties.

## How Local AI Enhancement Works

Local AI enhancement follows a five-phase pipeline orchestrated by the integration service:

### Model Discovery and Selection

The UI persists the user's chosen model ID in `UserDefaults` using the key provided by `PrivateAIProviderFeature.shared.selectedModelDefaultsKey`. The service retrieves this via the provider's `configuredModelID` property, which maps to a `PrivateAIRegisteredModel` describing the artifact's filename, download URL, and SHA-256 hash.

### Model Preparation and Verification

When a model is first needed, `prepareModel(_:progressHandler:)` validates the local filesystem against the expected path returned by `expectedLocalModelURL(for:)`. If the model file is missing, the provider downloads the artifact using the `downloadURL` from the registered model. Progress updates flow through the `PrivateAIModelDownloadProgressHandler` closure. After download, the file is verified against its SHA-256 hash and moved into the **model directory** located at `Application Support/FluidVoice/<ProviderName>/Models`.

### Runtime Initialization

Once the model file is local and verified, `loadModel(_:)` triggers the provider's concrete implementation—typically booting a local inference bridge such as `FluidIntelligence`. The runtime transitions through `loading` to `ready` states, exposed via the `isLocalRuntimeConfigured` flag.

### Pre-warming for Low Latency

To eliminate cold-start delays, `prewarmDictation()` optionally initializes a **prefix-cache** (KV-cache). This primes the inference engine so the first live dictation request incurs minimal token-generation latency, ensuring a responsive user experience.

### Dictation Enhancement Pipeline

During active dictation, `enhanceDictation(_:runtime:context:)` forwards the raw Whisper transcription alongside a `RuntimeConfiguration` (containing provider ID, model identifier, API key, and local path) and an `AppContext` (app name, bundle ID, window title). The provider returns an `EnhancementResult` containing:
- **`outputText`** – the corrected/enhanced transcription
- **`backendKind`** – identifier string (e.g., "private-ai")
- **`latencyMilliseconds`** – optional performance metric

If the private AI feature is unavailable (when `#if PRIVATE_AI_PROVIDER` is false), the service routes calls to `UnavailableAIIntegrationShim`, which immediately returns the original text unchanged.

## Key Source Files and Implementation Details

The local AI enhancement pipeline is distributed across the following files:

- **[`Sources/Fluid/Services/PrivateAIIntegrationService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift)** – Core actor that routes all private-AI operations and manages the enhancement lifecycle.

- **[`Sources/Fluid/Services/PrivateAIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIProvider.swift)** – Defines the `PrivateAIIntegrationProviding` protocol, the provider registry, and model registration logic.

- **[`Sources/Fluid/Services/PrivateAIProviderFeature.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIProviderFeature.swift)** – Supplies feature-availability checks and configuration keys for `UserDefaults`.

- **[`Sources/Fluid/Services/UnavailableAIIntegrationShim.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/UnavailableAIIntegrationShim.swift)** – No-op fallback implementation used when the private-AI backend is not compiled into the binary.

- **[`Sources/Fluid/Services/DictationPostProcessingService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/DictationPostProcessingService.swift)** – Integration point that calls `enhanceDictation` after initial Whisper transcription.

- **[`Sources/Fluid/ContentView.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/ContentView.swift)** – UI entry point that invokes the enhancement service for on-demand text improvements.

## Practical Implementation Examples

### Checking Private AI Availability

```swift
let isAvailable = PrivateAIProviderFeature.shared.isAvailable
print("Private AI available? \(isAvailable)")

```

*Source:* `PrivateAIProviderFeature.shared` in [[`PrivateAIProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/PrivateAIProvider.swift)](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIProvider.swift)

### Loading a Model and Pre-warming the Runtime

```swift
let service = PrivateAIIntegrationService.shared
let model = PrivateAIProviderFeature.shared.model(id: "gpt-4-mini")!

Task {
    // Prepare/download the model (optional progress handler)
    let localURL = try await service.prepareModel(model) { progress in
        print("Download progress: \(progress.fractionCompleted ?? 0)")
    }

    // Load the model into the local runtime
    let status = try await service.loadModel(model)
    print("Load status: \(status.state) – \(status.message ?? "")")

    // Warm-up the dictation engine
    await service.prewarmDictation()
}

```

*Source:* `prepareModel` and `loadModel` in [[`PrivateAIIntegrationService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/PrivateAIIntegrationService.swift)](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift)

### Enhancing Dictation with Local AI

```swift
let runtime = PrivateAIIntegrationService.RuntimeConfiguration(
    selectedProviderID: PrivateAIProviderFeature.shared.providerID,
    providerKey: "my-local-key",
    baseURL: "",                     // unused for local runtime
    model: "gpt-4-mini",
    apiKey: "",                      // unused for local runtime
    localModelPath: nil,
    usesStablePromptPrefixKVCache: true
)

let context = PrivateAIIntegrationService.AppContext(
    appName: "FluidVoice",
    bundleID: Bundle.main.bundleIdentifier!,
    windowTitle: "Notes",
    appVersion: Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String
)

Task {
    do {
        let result = try await service.enhanceDictation(
            "hey whats the weather today",
            runtime: runtime,
            context: context
        )
        print("Enhanced: \(result.outputText)")
    } catch {
        print("Enhancement failed: \(error)")
    }
}

```

*Source:* `enhanceDictation` in [[`PrivateAIIntegrationService.swift`](https://github.com/altic-dev/FluidVoice/blob/main/PrivateAIIntegrationService.swift)](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/PrivateAIIntegrationService.swift)

### UI Integration from ContentView

```swift
let response = try await PrivateAIIntegrationService.shared.enhanceDictation(
    rawTranscription,
    runtime: currentRuntime,
    context: appContext
)

```

*Source:* [[`ContentView.swift`](https://github.com/altic-dev/FluidVoice/blob/main/ContentView.swift)](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/ContentView.swift)

## Summary

- **PrivateAIIntegrationService** is an actor-based singleton that abstracts local LLM complexity behind a unified async API.
- The service automatically detects feature availability via `PrivateAIProviderFeature.shared.isAvailable` and falls back to a no-op shim when unavailable.
- The enhancement pipeline follows a strict sequence: **model discovery** → **download and verification** → **runtime initialization** → **pre-warming** → **dictation enhancement**.
- Local models are stored in `Application Support/FluidVoice/<ProviderName>/Models` and verified using SHA-256 hashes.
- Pre-warming via `prewarmDictation()` minimizes first-request latency by initializing the KV-cache before user interaction.

## Frequently Asked Questions

### What happens if the private AI feature is not compiled into the app?

The service routes all calls to `UnavailableAIIntegrationShim`, which immediately returns the original dictation text unchanged. This ensures FluidVoice continues to function normally for users who have not enabled or compiled the private AI backend.

### Where are local model files stored on disk?

Model files are stored in the Application Support directory under `FluidVoice/<ProviderName>/Models`. The exact path is accessible via the provider's `modelDirectoryURL` and `expectedLocalModelURL(for:)` methods, which construct platform-appropriate URLs for the current user's home directory.

### How does the service verify model integrity after downloading?

During `prepareModel(_:progressHandler:)`, the service downloads the model using the `downloadURL` and `sha256` hash specified in `PrivateAIRegisteredModel`. After download completes, it verifies the file against the SHA-256 hash before moving it to the final model directory, ensuring the local runtime loads only verified artifacts.

### Is PrivateAIIntegrationService thread-safe?

Yes, the service is implemented as a Swift **actor**, which guarantees that all state mutations and internal operations are serialized and thread-safe. This design allows safe concurrent access from multiple UI components and background tasks using Swift's structured concurrency model.