How to Fine-Tune FluidVoice Models for Specific Needs: A Complete Guide to Custom Prompts and LLM Configuration

You can fine-tune FluidVoice by creating custom dictation prompt profiles in SettingsStore.swift and configuring LLMClient.Config parameters to control model behavior, temperature, and provider selection without retraining any weights.

FluidVoice is an open-source macOS dictation application that uses large language models (LLMs) to post-process raw speech-to-text output. Unlike traditional fine-tuning that requires model retraining, FluidVoice offers a flexible prompt-based approach to adaptation. This guide explains how to leverage the LLMClient abstraction and SettingsStore persistence layer to tailor transcription quality for any domain.

---

Understanding FluidVoice's Fine-Tuning Architecture

The fine-tuning system in FluidVoice centers on three core components that work together to transform raw transcription into polished, domain-specific text.

The LLMClient Abstraction

Sources/Fluid/Services/LLMClient.swift defines the central LLMClient class with a Config struct that exposes every parameter sent to the LLM:

struct Config {
    var model: String
    var providerID: String
    var streaming: Bool
    var temperature: Double
    var maxTokens: Int
    var messages: [Message]
    var extraParameters: [String: Any]
}

The shared singleton provides the call(config:) method that all services use to execute requests. This unified entry point ensures consistent error handling, response parsing (including special "thinking" tags), and provider routing.

SettingsStore Persistence

Sources/Fluid/Persistence/SettingsStore.swift maintains:

• Selected transcription model (Nemotron Speech 3.5, Parakeet Flash, Whisper, etc.)
• Custom dictation prompt profiles — named system prompt templates
• Provider credentials — API keys stored in macOS keychain
• Custom provider definitions — for self-hosted or enterprise endpoints

Service Layer Integration

Three services demonstrate production patterns for fine-tuned LLM usage:

• RewriteModeService.swift — LLM configs for "write-mode" editing
• CommandModeService.swift — LLM configs for voice commands
• DictationPostProcessingService.swift — Applies custom prompts to raw transcription

---

Method 1: Create Custom Dictation Prompt Profiles

The fastest way to fine-tune FluidVoice is defining domain-specific system prompts that prepend every LLM request.

Step-by-Step Profile Creation

import Fluid

// Define a medical transcription profile
let medicalProfile = PromptConfiguration(
    name: "MedicalNotes",
    systemPrompt: """
    You are a medical scribe. Transcribe speech into concise, clinically-appropriate notes.
    Use proper medical terminology, and format output as:
    • Chief Complaint
    • History of Present Illness
    • Assessment & Plan
    """,
    temperature: 0.2,
    model: "gpt-4o-mini"
)

// Persist to SettingsStore
SettingsStore.shared.dictationPromptConfigurations["MedicalNotes"] = medicalProfile

Key parameters to adjust:

• temperature — Lower values (0.0–0.3) increase determinism for structured outputs; higher values (0.7–1.0) enable creative variation
• model — Any provider-supported identifier (GPT-4, Claude, Llama, etc.)
• systemPrompt — Instructions that shape tone, format, vocabulary, and reasoning style

---

Method 2: Configure LLM Request Parameters

Fine-tuning through LLMClient.Config lets you control inference behavior without changing prompts.

Building a Custom Configuration

func makeMedicalLLMConfig(messages: [Message]) -> LLMClient.Config {
    var cfg = LLMClient.Config(
        model: "gpt-4o-mini",
        providerID: "openai",
        streaming: false,
        temperature: 0.2,
        maxTokens: 1500
    )
    
    // Provider-specific parameters
    cfg.extraParameters["top_p"] = 0.95
    cfg.extraParameters["frequency_penalty"] = 0.5
    
    // Inject system + user messages
    let systemMessage = Message(
        role: .system,
        content: SettingsStore.shared.dictationPromptConfigurations["MedicalNotes"]?.systemPrompt ?? ""
    )
    cfg.messages = [systemMessage] + messages
    
    return cfg
}

Executing the Request

let userMessages = [
    Message(role: .user, content: "Patient reports chest pain radiating to the left arm.")
]

let config = makeMedicalLLMConfig(messages: userMessages)

Task {
    do {
        let response = try await LLMClient.shared.call(config)
        // response.content contains the fine-tuned output
        print(response.content)
    } catch {
        print("LLM error: \(error.localizedDescription)")
    }
}

---

Method 3: Add Custom LLM Providers

FluidVoice supports any HTTP-compatible endpoint, enabling local or enterprise deployment.

Registering a Self-Hosted Provider

// Add Ollama local instance
SettingsStore.shared.customProviders["ollama-local"] = ProviderInfo(
    displayName: "Ollama (local)",
    baseURL: URL(string: "http://127.0.0.1:11434/api")!,
    apiKey: ""  // No authentication for local server
)

Targeting the Custom Provider

var localConfig = LLMClient.Config(
    model: "llama3.1:8b",
    providerID: "ollama-local",  // Matches registered provider key
    streaming: true,
    temperature: 0.7,
    maxTokens: 1024
)

localConfig.messages = userMessages

Task {
    let response = try await LLMClient.shared.call(localConfig)
    // Streaming responses process chunks as they arrive
}

Provider routing happens automatically — LLMClient.shared.call selects the correct base URL and authentication method based on providerID.

---

Method 4: Service-Specific Fine-Tuning

Different FluidVoice modes benefit from specialized configurations.

Dictation Post-Processing

DictationPostProcessingService.swift applies prompt profiles to raw speech transcriptions. The service:

1. Retrieves active profile from SettingsStore
2. Constructs LLMClient.Config with transcription context
3. Calls LLMClient.shared.call for refinement
4. Inserts result into target application

Rewrite Mode

RewriteModeService.swift transforms selected text through LLM rewriting. Typical configuration:

var rewriteConfig = LLMClient.Config(
    model: SettingsStore.shared.selectedLLMModel,
    providerID: SettingsStore.shared.selectedProviderID,
    streaming: true,  // Enable for real-time preview
    temperature: 0.5,  // Balance creativity with accuracy
    maxTokens: 2000
)

Command Mode

CommandModeService.swift interprets voice commands. Uses lower temperature for reliable intent classification:

var commandConfig = LLMClient.Config(
    model: "gpt-4o-mini",
    providerID: "groq",  // Low-latency provider for responsive commands
    streaming: false,
    temperature: 0.1,  // Highly deterministic
    maxTokens: 256
)

---

Summary

• Fine-tune without retraining — FluidVoice uses prompt engineering and inference parameters to adapt LLM behavior rather than model weights
• Central configuration — LLMClient.Config in Sources/Fluid/Services/LLMClient.swift exposes complete control over model selection, temperature, streaming, and provider-specific options
• Persistent profiles — SettingsStore.swift maintains named prompt configurations that act as reusable fine-tuning templates
• Flexible deployment — Custom providers enable local (Ollama), cloud (OpenAI, Groq), or enterprise endpoints with uniform LLMClient.shared.call usage
• Service integration — DictationPostProcessingService, RewriteModeService, and CommandModeService demonstrate production patterns for applying fine-tuned configurations

---

Frequently Asked Questions

What models can I use to fine-tune FluidVoice?

Any model accessible through your chosen provider. FluidVoice's LLMClient accepts arbitrary model identifiers in Config.model — GPT-4, Claude, Llama, Mistral, or custom fine-tuned models on OpenAI, Groq, or self-hosted endpoints. The transcription model (Nemotron, Parakeet, Whisper) remains separate from the post-processing LLM.

Does fine-tuning require internet access or API costs?

No — you can route all LLM traffic to local infrastructure. Register a custom provider pointing to localhost (Ollama, llama.cpp, or compatible servers) and set empty apiKey. The only network traffic is what you explicitly configure; raw audio processing happens on-device.

How do I share fine-tuned configurations across devices?

SettingsStore persists to UserDefaults on macOS. Export and import dictationPromptConfigurations and customProviders dictionaries as JSON. The PromptConfiguration and ProviderInfo types are Codable, enabling simple serialization for team distribution or version control.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}