PrivateAIIntegrationService: Managing Local AI Runtime in FluidVoice

The PrivateAIIntegrationService is a Swift actor singleton that provides thread-safe management of FluidVoice’s optional local AI backend, handling model installation, runtime state, and dictation enhancement through a provider abstraction pattern.

The PrivateAIIntegrationService serves as the central runtime coordinator for FluidVoice's optional "Private AI" capabilities. Implemented in the altic-dev/FluidVoice repository, this service abstracts the complexities of local Core ML model management behind a type-safe, concurrent-friendly API that gracefully degrades when the private AI feature is unavailable.

Architecture Overview

The service follows a façade pattern that separates high-level operations from provider-specific implementations. This architecture allows FluidVoice to compile without the heavy AI dependencies when the PRIVATE_AI_PROVIDER flag is disabled, while maintaining a consistent API surface.

Core Components

• PrivateAIIntegrationService – Declared as an actor singleton (static let shared) in Sources/Fluid/Services/PrivateAIIntegrationService.swift, exposing high-level operations like model loading, pre-warming, and shutdown.
• PrivateAIIntegrationProviding – Protocol defined in Sources/Fluid/Services/PrivateAIProvider.swift that specifies the contract for any AI backend, including model directory handling and dictation processing.
• PrivateAIProviderFeature – Feature flag that gates compilation of the concrete provider via #if PRIVATE_AI_PROVIDER.
• UnavailableAIIntegrationShim – Null-object implementation in Sources/Fluid/Services/PrivateAIIntegrationService.swift that satisfies the protocol but reports the runtime as unavailable, ensuring the app compiles and runs without the private AI module.
• PrivateAIProviderBootstrap – Static installer in Sources/Fluid/Services/PrivateAIProvider.swift that registers the concrete provider at app launch via PrivateAIProviderBridge.install().

Runtime Management Flow

The service manages the local AI lifecycle through eight distinct phases, each delegating to the underlying provider while maintaining thread safety through actor isolation.

1. Provider Selection

The service accesses the current implementation via PrivateAIIntegrationService.provider, which dynamically resolves to either the registered runtime (PrivateAIProviderRegistry.integration) or the UnavailableAIIntegrationShim if the feature is not compiled.

2. Model Installation Verification

Before loading, the service validates model presence using file-system helpers:

• isModelInstalled(_:) checks if a model exists at modelDirectoryURL
• canRemoveInstalledModel(_:) verifies removal permissions
• removeInstalledModel(_:) deletes the model bundle
• expectedLocalModelURL(for:) generates the canonical path for validation

3. Model Preparation

The prepareModel(_:progressHandler:) method delegates download and setup to the provider. If the shim is active, this throws PrivateAIUnavailableError immediately.

4. Runtime Configuration

The computed property isLocalRuntimeConfigured returns a Boolean indicating whether a valid local model path has been selected and persists in user preferences.

5. Status Monitoring and Loading

• status(for:) returns a PrivateAIStatus enum with states: unavailable, missingModel, configured, loading, ready, or failed
• loadModel(_:) triggers the provider’s initialization sequence, transitioning the state to .ready upon success

6. Pre-warming for Latency Reduction

Calling prewarmDictation() instructs the provider to load the model and prime the KV-cache before the first user request, eliminating cold-start delays during dictation sessions.

7. Dictation Enhancement

The enhanceDictation(_:runtime:context:) method accepts raw transcription text along with:

• RuntimeConfiguration – Contains selectedProviderID, localModelPath, usesStablePromptPrefixKVCache, and API credentials
• AppContext – Provides environment metadata including appName, bundleID, windowTitle, and appVersion

The provider returns an EnhancementResult containing the processed outputText and optional latency metrics.

8. Graceful Shutdown

shutdownForTermination() invokes unloadCachedRuntime(reason:) to release native resources (Core ML contexts, GPU memory, file handles) before the app exits, preventing memory leaks or crashes during termination.

Thread Safety Through Actor Isolation

The underlying AI runtime holds substantial native resources including Core ML models, GPU contexts, and file handles. By declaring PrivateAIIntegrationService as an actor, all instance methods—loadModel, enhanceDictation, and unloadCachedRuntime—execute serially on the actor's isolated queue. This prevents race conditions during model swapping or concurrent dictation requests that could corrupt the runtime state or leak memory.

Public status queries expose non-isolated static members, allowing UI code to check isLocalRuntimeConfigured or access shared without await suspension, while write operations remain isolated for safety.

Working with the Service

The following pattern demonstrates complete integration, from singleton access to shutdown:

import Fluid

// Access the singleton actor
let aiService = PrivateAIIntegrationService.shared

// Verify configuration without awaiting
if await PrivateAIIntegrationService.isLocalRuntimeConfigured {
    do {
        // Load the selected model into memory
        let status = try await aiService.loadModel(PrivateAIIntegrationService.selectedModel)
        print("Model state: \(status.state)")
        
        // Pre-warm to reduce first-use latency
        await aiService.prewarmDictation()
        
        // Configure runtime parameters
        let runtime = PrivateAIIntegrationService.RuntimeConfiguration(
            selectedProviderID: PrivateAIProviderFeature.shared.providerID,
            providerKey: "local-api-key",
            baseURL: "http://localhost:8000",
            model: PrivateAIIntegrationService.selectedModel.id,
            apiKey: "local-api-key",
            localModelPath: PrivateAIIntegrationService.configuredLocalModelPath,
            usesStablePromptPrefixKVCache: true
        )
        
        // Provide application context for enhancement
        let context = PrivateAIIntegrationService.AppContext(
            appName: "FluidVoice",
            bundleID: Bundle.main.bundleIdentifier ?? "",
            windowTitle: "Dictation Session",
            appVersion: Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String
        )
        
        // Enhance raw dictation text
        let result = try await aiService.enhanceDictation(
            "meeting notes from today discuss q3 roadmap",
            runtime: runtime,
            context: context
        )
        print("Enhanced: \(result.outputText)")
        
    } catch {
        print("AI operation failed: \(error)")
    }
}

// Release resources before termination
await aiService.shutdownForTermination()

Summary

• PrivateAIIntegrationService is an actor singleton in Sources/Fluid/Services/PrivateAIIntegrationService.swift that manages FluidVoice's local AI lifecycle.
• It uses the PrivateAIIntegrationProviding protocol to abstract concrete implementations, falling back to UnavailableAIIntegrationShim when the PRIVATE_AI_PROVIDER feature flag is disabled.
• The service handles model installation verification, loading, pre-warming, and enhancement through thread-safe actor isolation.
• Runtime state flows through PrivateAIStatus enum values: unavailable, missingModel, configured, loading, ready, and failed.
• Non-isolated static members allow UI queries without await, while instance methods ensure serialized access to native AI resources.

Frequently Asked Questions

What happens if the Private AI feature is not compiled into the app?

When the PRIVATE_AI_PROVIDER compiler flag is absent, PrivateAIProviderBootstrap skips registration and PrivateAIIntegrationService.provider resolves to UnavailableAIIntegrationShim. This shim implements the same protocol but throws PrivateAIUnavailableError for any operation requiring the actual runtime, allowing the app to compile and run without the heavy Core ML dependencies while maintaining API consistency.

How does the service prevent race conditions during model loading?

The service is declared as a Swift actor, which isolates its mutable state and instance methods (such as loadModel and enhanceDictation) to a single serial queue. This ensures that model loading, unloading, and inference operations execute atomically, preventing simultaneous access that could corrupt the Core ML model state or leak GPU resources.

What is the purpose of prewarming the dictation pipeline?

Calling prewarmDictation() instructs the provider to load the model weights and initialize the KV-cache before the first user dictation. This eliminates cold-start latency—often several seconds for large language models—ensuring that the first enhanceDictation request returns immediately rather than blocking while the runtime initializes.

Where does the service look for installed local models?

The service constructs model paths using modelDirectoryURL and validates installation against expectedLocalModelURL(for:), typically resolving to the app's sandboxed Documents or Application Support directory. The isModelInstalled(_:) method performs file-system checks to confirm the model bundle exists at the expected path before attempting to load it into memory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}