How to Implement Custom Vocabulary and Word Boosting with ParakeetProvider in FluidVoice

Use ParakeetVocabularyStore to persist boost terms in JSON, enable vocabularyBoostingEnabled in SettingsStore, and let FluidAudioProvider inject the tokenized bundle into the transcription pipeline.

FluidVoice is an open-source transcription framework that ships with the Parakeet speech recognition engine (supporting Parakeet v2, v3, and Flash variants). You can enhance recognition accuracy for domain-specific terms, product names, or uncommon words by implementing custom vocabulary boosting through a three-layer persistence, UI, and runtime architecture.

Understanding the Vocabulary Boost Architecture

The boost workflow is split into three distinct layers that handle storage, user interaction, and runtime injection.

Persistence Layer (ParakeetVocabularyStore.swift)

The ParakeetVocabularyStore manages a JSON file named parakeet_custom_vocabulary.json stored in the app's Application Support directory. This file contains an array of VocabularyConfig.Term objects following this schema:

// Sources/Fluid/Services/ParakeetVocabularyStore.swift
struct VocabularyConfig: Codable {
    struct Term: Codable, Hashable {
        let text: String                // word or phrase to boost
        let weight: Float?              // optional priority (higher → stronger)
        let aliases: [String]           // alternate spellings / pronunciations
    }
    let alpha: Float?
    let minCtcScore: Float?
    let terms: [Term]
}

The store automatically creates the file if it does not exist via ensureVocabularyFileExists(), falling back to a default template. It caps user-managed terms to Defaults.maxTerms (256 entries) and provides methods to load and save terms without overwriting backend-controlled tuning values like alpha or minCtcScore.

UI Layer (CustomDictionaryView.swift)

The Custom Dictionary pane in CustomDictionaryView.swift hosts the Custom Words section where users manage boost terms. The Boosting toggle (vocabBoostingEnabled) enables or disables the feature at runtime. When users press Add Word, the UI creates a new ParakeetVocabularyStore.VocabularyConfig.Term and persists it:

// Sources/Fluid/UI/CustomDictionaryView.swift
self.boostTerms.append(newTerm)
try ParakeetVocabularyStore.shared.saveUserBoostTerms(self.boostTerms)

The view loads persisted terms on launch via loadUserBoostTerms() and populates the editing interface.

Runtime Integration (FluidAudioProvider.swift)

When a transcription session starts, FluidAudioProvider checks if boosting is enabled and loads the tokenized vocabulary bundle:

// Sources/Fluid/Services/FluidAudioProvider.swift
if SettingsStore.shared.vocabularyBoostingEnabled {
    if let vocabBundle = try await ParakeetVocabularyStore.shared.loadTokenizedVocabularyBundle() {
        self.audioEngine.setCustomVocabulary(vocabBundle.vocabulary, ctcModels: vocabBundle.ctcModels)
    }
}

If the bundle is empty or boosting is disabled, the engine falls back to the default Parakeet model.

Configuring the Vocabulary JSON Schema

Each boost term requires a text field and optionally accepts weight and aliases. The weight parameter acts as a priority multiplier—higher values increase the likelihood of recognition. Aliases accommodate phonetic variations or common misspellings.

Key files:

• Sources/Fluid/Services/ParakeetVocabularyStore.swift – Core JSON store, validation, and schema definitions.
• Sources/Fluid/UI/CustomDictionaryView.swift – User interface for term management.

Loading and Saving Boost Terms Programmatically

Access the shared vocabulary store to manipulate boost terms outside the UI:

// Load existing user terms (normalizes aliases and enforces max count)
var boostTerms = try ParakeetVocabularyStore.shared.loadUserBoostTerms()

// Create a new term with high priority weight
let technicalTerm = ParakeetVocabularyStore.VocabularyConfig.Term(
    text: "Kubernetes",
    weight: 15.0,
    aliases: ["k8s", "kube"]
)

// Persist the updated list
boostTerms.append(technicalTerm)
try ParakeetVocabularyStore.shared.saveUserBoostTerms(boostTerms)

The saveUserBoostTerms(_:) method writes a fresh JSON file while preserving global tuning parameters. The loadUserBoostTerms() method returns only user-managed entries, filtering out system defaults.

Tokenizing and Merging with Custom Dictionary

Before injection into the audio pipeline, terms must be tokenized via CtcTokenizer and merged with the plain-text custom dictionary (SettingsStore.shared.customDictionaryEntries).

The loadResolvedConfig() method merges boost terms with dictionary replacements, while loadTokenizedVocabularyBundle(maxTerms:) creates the CustomVocabularyContext required by FluidAudio:

// Sources/Fluid/Services/ParakeetVocabularyStore.swift
let tokenizedBundle = try await ParakeetVocabularyStore.shared.loadTokenizedVocabularyBundle()

This method caps the list to 256 terms, sorts by weight, and returns both the vocabulary array and compiled CTC models for Apple Silicon devices.

Enabling Boosting at Runtime

Toggle the vocabulary boosting feature through SettingsStore:

// Enable the feature globally
SettingsStore.shared.vocabularyBoostingEnabled = true

// Start transcription with ParakeetFlash provider
let asr = ASRService()
await asr.start(provider: .parakeetFlash)

When vocabularyBoostingEnabled is true, FluidAudioProvider automatically calls loadTokenizedVocabularyBundle() and injects the context into setCustomVocabulary(_:ctcModels:).

Complete Implementation Example

Below is a complete workflow for programmatically adding a boost term and starting a transcription session:

import Fluid

// 1. Define a boost term with phonetic aliases
let newTerm = ParakeetVocabularyStore.VocabularyConfig.Term(
    text: "OpenAI‑GPT",
    weight: 12.0,
    aliases: ["open a i g p t", "o pen ai gee pee tee"]
)

// 2. Load, append, and persist
var terms = try ParakeetVocabularyStore.shared.loadUserBoostTerms()
terms.append(newTerm)
try ParakeetVocabularyStore.shared.saveUserBoostTerms(terms)

// 3. Enable boosting and start transcription
SettingsStore.shared.vocabularyBoostingEnabled = true
let service = ASRService()
await service.start(provider: .parakeetFlash)

Summary

• Storage: ParakeetVocabularyStore manages parakeet_custom_vocabulary.json in Application Support with a 256-term limit.
• Schema: Terms include text, optional weight (higher values = stronger boosting), and aliases for phonetic variations.
• Persistence: Use loadUserBoostTerms() and saveUserBoostTerms(_:) to programmatically modify vocabulary.
• Runtime: FluidAudioProvider checks SettingsStore.shared.vocabularyBoostingEnabled and injects the tokenized bundle via setCustomVocabulary(_:ctcModels:).
• Sources: Key implementations reside in ParakeetVocabularyStore.swift, CustomDictionaryView.swift, and FluidAudioProvider.swift.

Frequently Asked Questions

What is the maximum number of boost terms supported?

FluidVoice caps custom vocabulary at 256 terms via Defaults.maxTerms in ParakeetVocabularyStore.swift. The loadUserBoostTerms() method automatically truncates lists exceeding this limit to prevent performance degradation during CTC tokenization.

Does word boosting work with all Parakeet variants?

Yes. The boost bundle loads for Parakeet v2, v3, and Flash variants. The ASRService provider enum (.parakeet, .parakeetFlash) handles the specific model selection, but FluidAudioProvider applies the same CustomVocabularyContext regardless of the Parakeet version selected.

How does weight affect recognition priority?

The weight parameter in VocabularyConfig.Term acts as a scalar multiplier for the CTC path scores. Higher values (typically 10.0–20.0) increase the probability that the ASR engine selects your custom term over phonetically similar alternatives. Weights are optional; if omitted, terms receive default baseline scoring.

Can I import vocabulary from external sources?

Yes. The DictionaryTransferService.swift file handles import/export via DictionaryAPIController. You can programmatically populate parakeet_custom_vocabulary.json by constructing VocabularyConfig.Term objects from external CSV or JSON sources and calling saveUserBoostTerms(_:), or use the public API endpoints if exposing vocabulary management to remote services.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}