What Kind of Data Does FluidVoice Use for Training?

FluidVoice trains on audio recordings of the user's voice paired with the textual target that should be recognized, requiring three successful pronunciation captures before marking a custom dictionary entry as ready for inference.

The open-source FluidVoice speech-to-text system (available at altic-dev/FluidVoice) implements a user-driven training pipeline that combines raw audio samples with explicit text labels. This approach allows the system to learn custom vocabulary, proper nouns, and specialized terminology that standard speech recognition models often miss. The training data flows through a tightly integrated stack of CoreAudio capture, SwiftUI state management, and validation logic.

Audio Data: Raw PCM Samples from Microphone

FluidVoice captures uncompressed audio data using low-level CoreAudio APIs. The CoreAudioCaptureSupport.c module handles the actual hardware interaction, streaming raw PCM samples from the device microphone through a C-based implementation bridged to Swift via CoreAudioCaptureSupportBridge.h.

// From CustomDictionaryView.swift - audio capture storage
@State private var trainingPronunciationEnrollments: [PronunciationEnrollmentCapture] = []

Each successful capture appends a PronunciationEnrollmentCapture object to this array. These captures contain the raw audio buffers that feed into the training pipeline. The system does not use pre-recorded datasets or cloud-sourced audio—all training audio originates from the local user's microphone during active enrollment sessions.

Text Data: Target Phrases and Variants

Alongside audio, FluidVoice stores the intended transcription and any alternative spellings or phrasings:

Data Field	Source File	Purpose
trainingReplacement	CustomDictionaryView.swift	Primary text the user wants recognized
trainingVariants	CustomDictionaryView.swift	Alternative textual representations of the same spoken phrase

The trainingVariants array allows users to specify multiple written forms for acoustically similar utterances—critical for handling homophones, abbreviations, or context-dependent spellings.

Training Completion Criteria

The system enforces a minimum sample threshold before accepting a trained entry. As implemented in CustomDictionaryView.swift, the readiness check compares the enrollment count against CustomDictionaryTrainingMerge.readyCoveredCount (default value: 3):

// Validation logic from CustomDictionaryView.swift
self.trainingPronunciationEnrollments.count >= CustomDictionaryTrainingMerge.readyCoveredCount

Additional state variables track training progress:

• trainingSampleCount — increments per successful capture
• trainingReadinessProgress — drives the progress bar UI
• trainingFinalOutputIsReady — boolean flag enabling save action

UI Flow for Data Collection

The AutomaticDictionaryCorrectionOverlay.swift view orchestrates the user-facing training experience. It presents step-by-step instructions, visual feedback during recording, and error states when audio quality falls below thresholds. This overlay ensures that captured data meets quality standards before inclusion in the training set.

Code Example: Minimal Training Session

The following SwiftUI pattern mirrors FluidVoice's core training loop:

import SwiftUI

struct SimpleTrainingView: View {
    @State private var trainingReplacement = ""
    @State private var trainingPronunciationEnrollments: [PronunciationEnrollmentCapture] = []
    @State private var trainingSampleCount = 0
    @State private var isRecording = false

    var body: some View {
        VStack(spacing: 20) {
            TextField("Enter phrase to train", text: $trainingReplacement)
                .textFieldStyle(.roundedBorder)

            Button(isRecording ? "Stop" : "Start") {
                isRecording.toggle()
                if isRecording {
                    startRecording()
                } else {
                    stopRecording()
                }
            }
            .buttonStyle(.borderedProminent)

            ProgressView("Samples collected",
                         value: Double(trainingSampleCount),
                         total: 3)
                .padding()
        }
        .padding()
    }

    private func startRecording() {
        // CoreAudioCaptureSupport is used under the hood.
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
            let capture = PronunciationEnrollmentCapture(audioData: Data())
            trainingPronunciationEnrollments.append(capture)
            trainingSampleCount += 1
        }
    }

    private func stopRecording() {
        // Stops the CoreAudio stream in production implementation.
    }
}

Privacy and Data Sovereignty

Because FluidVoice's training data consists entirely of locally captured, user-generated audio and text, no training material leaves the device unless explicitly exported by the user. This architecture eliminates cloud dependency for custom vocabulary and ensures sensitive terminology (medical, legal, proprietary) remains under user control.

Summary

• FluidVoice uses paired audio-text data for custom dictionary training: spoken recordings plus target transcriptions.
• Three successful captures are required per entry, enforced by trainingPronunciationEnrollments.count >= CustomDictionaryTrainingMerge.readyCoveredCount.
• CoreAudio APIs in CoreAudioCaptureSupport.c provide raw PCM samples; SwiftUI state in CustomDictionaryView.swift manages the training lifecycle.
• All data originates locally from the user's microphone; no external datasets or cloud processing are involved.

Frequently Asked Questions

Does FluidVoice use pre-trained speech models or external datasets?

FluidVoice operates on user-generated training data captured in real-time. The repository contains no embedded speech corpora or third-party audio datasets. All trainingPronunciationEnrollments come from live CoreAudio sessions, making the system adaptable to individual voices without dependency on remote services.

What audio format does FluidVoice store for training?

The system stores raw PCM audio buffers wrapped in PronunciationEnrollmentCapture objects. The CoreAudioCaptureSupport.c implementation handles format conversion from hardware-native streams to the internal representation used by the training pipeline. Specific sample rates and bit depths are determined by the CoreAudio device configuration.

Can users edit training data after capture?

The trainingVariants array in CustomDictionaryView.swift permits textual modifications to the target phrase. However, recorded audio enrollments in trainingPronunciationEnrollments are immutable once captured—users must re-record audio if quality is insufficient. The UI enforces this by requiring trainingSampleCount to reach the threshold before enabling save.

How does FluidVoice handle training failures or poor audio quality?

The AutomaticDictionaryCorrectionOverlay.swift view provides real-time feedback during enrollment. If trainingPronunciationEnrollments fails to accumulate (due to silence detection, noise thresholds, or hardware errors), the progress indicator stalls and the overlay displays corrective guidance. The trainingFinalOutputIsReady flag remains false until all quality gates pass.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}