How FluidVoice Performs Post‑Processing on Transcriptions: A Complete Technical Guide

How FluidVoice Performs Post‑Processing on Transcriptions: A Complete Technical Guide

FluidVoice applies a five‑stage post‑processing pipeline to every raw ASR result, running filler‑word removal, custom‑dictionary correction, spoken‑punctuation formatting, optional AI‑driven refinement, and debug logging before the final text reaches the user.

FluidVoice is an open‑source dictation app that transforms raw speech‑to‑text output into polished, publication‑ready prose. Understanding how FluidVoice performs post‑processing on transcriptions is essential for developers customizing the pipeline or users optimizing their dictation workflow. This article examines the exact sequence, source files, and implementation details of each processing stage.

---

The Five Stages of Transcription Post‑Processing

The post‑processing pipeline runs synchronously after every audio chunk finishes recognition. According to the FluidVoice source code, the stages execute in a strict order designed to preserve semantic intent while cleaning surface‑level artifacts.

1. Filler‑Word Removal

The first transformation strips verbal crutches that ASR systems faithfully transcribe. Common targets include "um", "uh", "like", and "you know".

• Implementation location: ASRService.swift at line 2318
• Entry point: ASRService.removeFillerWords(result.text)

This method operates on the raw string before any other normalization occurs, ensuring filler words do not interfere with subsequent dictionary matching or punctuation detection.

2. Custom‑Dictionary Correction

Users define personal dictionaries in SettingsStore to enforce preferred spellings, technical terminology, or name substitutions. This stage applies those mappings globally.

• Implementation location: ASRService.swift at line 2311
• Entry point: ASRService.applyCustomDictionary(textWithoutFillers)

Dictionary rules are loaded from persistent storage and applied as string replacement operations. The corrected text then flows into punctuation formatting.

3. Spoken‑Punctuation Formatting

Dictation users speak punctuation commands aloud ("comma", "period", "new line"). This stage detects those commands and renders proper symbols with correct spacing and capitalization.

• Orchestration: ASRService.applySpokenPunctuationFormatting(dictionaryText) at line 2315 of ASRService.swift
• Core logic: ASRService+SpokenPunctuationFormatting.swift at line 26

The formatter consults phrase rules and action rules stored in SettingsStore. These rules define:

• Which spoken phrases trigger punctuation insertion
• Whether to capitalize the next word
• How much whitespace to inject around symbols

This separation of rule data from execution logic allows runtime customization without code changes.

4. AI‑Driven Post‑Processing (Optional)

When enabled, FluidVoice offloads raw transcription to a configurable LLM before running stages 1–3. The AI can correct grammar, expand abbreviations, or reformat structure.

• Configuration gate: DictationAIPostProcessingGate.isConfigured() at line 4
• HTTP endpoint registration: LocalAPIRouter.swift at line 25 (/v1/postprocess)
• Request handler: InferenceAPIController.postprocess

The flow works as follows:

1. If DictationAIPostProcessingGate.isConfigured() returns true, the raw transcription is sent to the local API endpoint
2. The selected provider (OpenAI, Groq, or local LLM) returns processed text
3. The AI response is merged with the original (or replaces it, depending on configuration)
4. Stages 1–3 run on the merged result

This optional stage executes before the standard pipeline, ensuring AI‑introduced filler words or formatting still get cleaned by the deterministic stages.

5. Debug Logging

The final stage captures the fully processed output for troubleshooting:

DebugLogger.shared.debug("After post‑processing: '\(outputText)'", source: "ASRService")

This appears at line 2319 of ASRService.swift and aids developers in tracing transformation chains.

---

Complete Pipeline Implementation

The following Swift code demonstrates the full post‑processing sequence as implemented in FluidVoice:

// 1️⃣ Raw result from the ASR provider
let rawResult = try await provider.transcribeFinal(pcm)

// 2️⃣ Remove filler words
let withoutFillers = ASRService.removeFillerWords(rawResult.text)

// 3️⃣ Apply the custom dictionary
let dictCorrected = ASRService.applyCustomDictionary(withoutFillers)

// 4️⃣ Insert spoken punctuation
let finalText = ASRService.applySpokenPunctuationFormatting(dictCorrected)

// 5️⃣ (Optional) AI post‑processing – performed before steps 2‑4 if enabled
if DictationAIPostProcessingGate.isConfigured() {
    let aiResponse = await LocalAPIClient.postprocess(text: rawResult.text)
    // … merge aiResponse, then run steps 2‑4 …
}

Note the conditional reordering: when AI post‑processing is active, steps 2–4 run twice—once on the raw result (for preview/logging) and once on the AI‑refined output for final delivery.

---

Key Source Files

File	Responsibility
Sources/Fluid/Services/ASRService.swift	Orchestrates the core post‑processing flow; contains filler‑word removal, dictionary application, and punctuation formatting entry points
Sources/Fluid/Services/ASRService+SpokenPunctuationFormatting.swift	Implements phrase‑matching and symbol‑insertion logic for spoken punctuation
Sources/Fluid/Services/DictationAIPostProcessingGate.swift	Validates AI provider configuration and feature flags
Sources/Fluid/Services/LocalAPI/LocalAPIRouter.swift	Registers HTTP routes including /v1/postprocess
Sources/Fluid/Services/LocalAPI/InferenceAPIController.swift	Executes LLM inference calls for AI post‑processing
Sources/Fluid/Persistence/SettingsStore.swift	Persists user dictionaries, punctuation rules, and AI provider settings

---

Customizing the Pipeline

Developers can modify FluidVoice transcription post‑processing behavior through several extension points:

• Add filler words: Extend removeFillerWords in ASRService.swift with additional pattern matches
• Dynamic dictionaries: Update SettingsStore entries at runtime; changes apply to the next transcription
• Custom punctuation rules: Define new phrase/action pairs in SettingsStore without touching formatter code
• AI model swapping: Implement alternate InferenceAPIController backends while preserving the /v1/postprocess contract

---

Summary

• FluidVoice post‑processing runs five sequential stages on every transcription: filler‑word removal, custom‑dictionary correction, spoken‑punctuation formatting, optional AI refinement, and debug logging
• The deterministic pipeline (stages 1–3) executes in ASRService.swift with formatter logic residing in ASRService+SpokenPunctuationFormatting.swift
• AI post‑processing gates through DictationAIPostProcessingGate.isConfigured() and routes through LocalAPIRouter.swift before re‑entering the standard pipeline
• All user‑configurable data lives in SettingsStore, enabling runtime customization without recompilation

---

Frequently Asked Questions

What filler words does FluidVoice remove by default?

The base implementation in ASRService.removeFillerWords targets English fillers including "um", "uh", "like", "you know", and "so". The exact list is hardcoded in ASRService.swift at line 2318. Developers can extend this method to support additional languages or domain‑specific disfluencies.

Can I disable AI post‑processing while keeping other features?

Yes. AI post‑processing is entirely optional. If DictationAIPostProcessingGate.isConfigured() returns false—either because no provider is selected or because the configuration is incomplete—the pipeline skips the /v1/postprocess call and proceeds directly with deterministic stages 1–3.

How does spoken‑punctuation formatting handle ambiguous commands?

The formatter in ASRService+SpokenPunctuationFormatting.swift resolves ambiguity through phrase rules from SettingsStore. These rules specify exact spoken strings and their corresponding actions, including capitalization behavior and spacing. Users can override defaults or add custom commands without modifying source code.

Where does FluidVoice store custom dictionary entries?

Persistent storage for dictionaries, punctuation rules, and AI configuration is managed by SettingsStore.swift. This class abstracts the underlying persistence mechanism (likely Core Data or UserDefaults) and provides the rule sets consumed by applyCustomDictionary and the spoken‑punctuation formatter.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}