# How Context Biasing Improves ASR Accuracy for Domain-Specific Terms in FluidAudio

> Enhance ASR accuracy for domain-specific terms with context biasing in FluidAudio. Learn how this technique boosts recognition by weighting rare words during CTC decoding.

- Repository: [Fluid Inference/fluidaudio](https://github.com/fluidinference/fluidaudio)
- Tags: deep-dive
- Published: 2026-03-02

---

**Context biasing boosts ASR accuracy by adding a configurable log-probability weight to domain-specific terms during CTC decoding, offsetting the acoustic model's bias against rare or out-of-vocabulary words.**

FluidAudio implements an advanced **context biasing** system that enables real-time recognition of domain-specific terminology—such as product names, medical jargon, or technical commands—without retraining the underlying acoustic model. According to the fluidinference/fluidaudio source code, the system uses a three-component pipeline that dynamically rescores CTC hypotheses against a custom vocabulary during streaming transcription.

## The Three-Pillar Architecture of Context Biasing

FluidAudio’s context biasing implementation relies on tightly coupled components that handle data management, algorithmic constants, and execution logic.

### CustomVocabularyContext: Runtime Term Management

The `CustomVocabularyContext` class, defined in [`Sources/FluidAudio/ASR/CustomVocabulary/CustomVocabularyContext.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/CustomVocabulary/CustomVocabularyContext.swift), serves as the runtime container for domain-specific terms. It parses JSON vocabulary files and stores per-term parameters including **context biasing weight (CBW)**, CTC-score thresholds, and optional token IDs. When `CustomVocabularyContext.load(from:)` parses a vocabulary file, it sanitizes each term and fills missing parameters using defaults from the system's constant registry.

### ContextBiasingConstants: Algorithmic Tuning

All magic numbers governing the rescoring behavior live in [`Sources/FluidAudio/ASR/CustomVocabulary/ContextBiasingConstants.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/CustomVocabulary/ContextBiasingConstants.swift). This centralizes critical thresholds such as `defaultAlpha` (the base CBW), similarity floors, and adaptive scaling parameters. The file provides `rescorerConfig(forVocabSize:)`, which returns vocabulary-size-aware configurations—large vocabularies receive tighter similarity floors (`minSimilarity = 0.60`) and slightly reduced CBW values (`cbw = 2.5`) to prevent false positives.

### VocabularyRescorer: CTC-Based Execution Engine

The actual biasing logic executes in [`Sources/FluidAudio/ASR/CustomVocabulary/Rescorer/VocabularyRescorer.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/CustomVocabulary/Rescorer/VocabularyRescorer.swift). This CTC-based rescoring engine compares acoustic evidence for hypothesized words against boosted terms via `ctcTokenRescore(...)`. It enforces hierarchical similarity thresholds and applies the adaptive CBW to log-probabilities, determining whether a domain term should replace the generic hypothesis in the final transcript.

## How Context Biasing Works: The Execution Pipeline

The system follows a six-stage pipeline during streaming ASR sessions, orchestrated by `StreamingAsrManager` in [`Sources/FluidAudio/ASR/Streaming/StreamingAsrManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/Streaming/StreamingAsrManager.swift).

### 1. Vocabulary Loading and Validation

The process begins when `CustomVocabularyContext.load(from:)` ingests a JSON or text file containing domain terms. The method validates each entry against `ContextBiasingConstants` defaults, ensuring every term carries required parameters like `defaultMinSimilarity` for fuzzy matching guards.

### 2. CTC Keyword Spotting

The `CtcKeywordSpotter` runs the acoustic model on audio buffers, producing frame-wise log-probabilities for every token in the vocabulary. This provides the raw acoustic scores that the rescoring engine will modify.

### 3. Rescoring Engine Configuration

`StreamingAsrManager.configureVocabularyBoosting(...)` constructs a `VocabularyRescorer` via an async factory. The factory checks `ContextBiasingConstants.useBkTree` to determine whether to enable BK-tree indexing for O(log V) fuzzy matching performance, and selects the appropriate configuration via `rescorerConfig(forVocabSize:)`.

### 4. Adaptive Bias Weight Calculation

Inside `VocabularyRescorer.Config.adaptiveCbw(baseCbw:tokenCount:)`, the system calculates a dynamic CBW. The bias weight scales by the ratio of the term's token count to a reference count, giving longer phrases a larger boost to compensate for CTC's cumulative acoustic penalty across multiple tokens.

### 5. Real-Time Rescoring During Streaming

For each confirmed audio chunk processed by `processWindow`, the system executes `VocabularyRescorer.ctcTokenRescore(...)`:

- Evaluates string similarity using Levenshtein distance against the original hypothesis
- Enforces hierarchical thresholds: `minSimilarityFloor` → `shortWordSimilarity` → `stopwordSpanSimilarity`
- Adds the adaptive CBW to the term's CTC log-probability
- Replaces the original word if the adjusted score exceeds the baseline by the configured margin

### 6. Result Propagation

The rescored transcript flows back to `StreamingAsrManager`, which updates the UI via `StreamingTranscriptionUpdate`. The final `finish()` call assembles the complete text using rescored confirmed portions, ensuring domain-specific terms survive the final output even when the raw acoustic model initially missed them.

## Why Context Biasing Solves Domain-Specific Recognition Failures

Context biasing addresses specific failure modes inherent to general-purpose acoustic models:

**Rare or out-of-vocabulary words.** When the acoustic model has never encountered a term like a proprietary product name, its CTC score collapses. The CBW—defaulting to values around 2.5–3.0 plus adaptive scaling—adds a substantial log-probability boost that makes rare terms competitive with generic hypotheses.

**Long multi-token sequences.** CTC applies a penalty per token, causing cumulative scores to drop over longer phrases. The `adaptiveCbw` mechanism scales the boost with token count, effectively neutralizing this length-based penalty.

**Phonetically similar competitors.** Terms like "nvidia" versus "nvida" create confusion. Hierarchical similarity thresholds ensure only candidates meeting Levenshtein-based guards (floor → short-word → stopword span) qualify for boosting, eliminating false positives from near-miss words.

**Large vocabulary scalability.** As vocabulary size grows, false-positive risk increases. The `rescorerConfig(forVocabSize:)` method automatically tightens similarity floors and reduces CBW for large vocabularies, maintaining precision without manual tuning.

**Computational efficiency.** Linear scanning across thousands of terms would introduce latency. The optional BK-tree lookup (`ContextBiasingConstants.useBkTree`) provides logarithmic-time candidate retrieval, enabling real-time performance even with extensive custom dictionaries.

## Implementing Context Biasing in Swift

The following end-to-end example demonstrates enabling context biasing in a streaming session using FluidAudio's public API:

```swift
import FluidAudio

// 1️⃣  Load a custom vocabulary file (JSON format produced by FluidAudio tooling)
let vocabURL = URL(fileURLWithPath: "/path/to/myDomainVocab.json")
let customVocab = try CustomVocabularyContext.load(from: vocabURL)

// 2️⃣  Download the CTC models (once per app run)
let ctcModels = try await CtcModels.downloadAndLoad()

// 3️⃣  Create a streaming manager with the default configuration
let streamingMgr = StreamingAsrManager()

// 4️⃣  Enable vocabulary boosting – we keep the default adaptive config
try await streamingMgr.configureVocabularyBoosting(
    vocabulary: customVocab,
    ctcModels: ctcModels,
    config: nil               // uses ContextBiasingConstants.rescorerConfig(...)
)

// 5️⃣  Start streaming from the microphone
try await streamingMgr.start(source: .microphone)

// 6️⃣  Feed audio buffers from AVAudioEngine (example)
audioEngine?.inputNode.installTap(
    onBus: 0,
    bufferSize: 2048,
    format: audioEngine?.inputNode.outputFormat(forBus: 0)
) { buffer, _ in
    streamingMgr.streamAudio(buffer)
}

// 7️⃣  Receive live updates
let updates = streamingMgr.transcriptionUpdates
for await update in updates {
    print("[\(update.isConfirmed ? "✔︎" : "…")] \(update.text)")
}

// 8️⃣  When finished, obtain the final, bias‑aware transcript
let finalTranscript = try await streamingMgr.finish()
print("✅ Final: \(finalTranscript)")

```

Key implementation details from the fluidinference/fluidaudio source:

- `configureVocabularyBoosting` instantiates a `VocabularyRescorer` that internally references `ContextBiasingConstants` for CBW and threshold values
- The `updates` stream delivers bias-applied text for every confirmed chunk, allowing UI components to display domain-specific terms immediately upon detection
- The final `finish()` call guarantees that all boosted terms surviving the rescoring process appear in the complete transcript

## Summary

- **Context biasing** in FluidAudio operates as a shallow-fusion rescoring layer that modifies CTC log-probabilities during streaming decoding, not during model training.
- The system combines three components: `CustomVocabularyContext` for data management, `ContextBiasingConstants` for algorithmic tuning, and `VocabularyRescorer` for execution.
- **Adaptive CBW scaling** automatically adjusts boost strength based on term length, compensating for CTC's multi-token penalty.
- **Hierarchical similarity guards** (Levenshtein-based thresholds) prevent false positives while allowing fuzzy matches for domain terms.
- **Vocabulary-size-aware configuration** automatically tightens precision parameters when large custom dictionaries are loaded.
- The implementation requires no model retraining—users simply load a JSON vocabulary and call `configureVocabularyBoosting(...)` on their `StreamingAsrManager` instance.

## Frequently Asked Questions

### What is context biasing in ASR?

Context biasing is a decoding-time technique that increases the probability of specific words or phrases appearing in transcription output. In FluidAudio, the system adds a **context biasing weight (CBW)** to the log-probability of domain-specific terms during CTC rescoring, effectively telling the decoder "prefer these words when acoustic evidence is ambiguous." This happens without modifying the underlying acoustic model weights, making it suitable for runtime customization.

### How does FluidAudio handle large custom vocabularies?

FluidAudio addresses large-vocabulary challenges through `ContextBiasingConstants.rescorerConfig(forVocabSize:)`. As vocabulary size increases, the system automatically reduces the CBW and raises the `minSimilarity` threshold to maintain precision. Additionally, when `ContextBiasingConstants.useBkTree` is enabled, the `VocabularyRescorer` uses a BK-tree data structure for O(log V) fuzzy matching instead of linear scanning, keeping latency low even with thousands of domain terms.

### What are the similarity thresholds for?

The hierarchical similarity thresholds—`minSimilarityFloor`, `shortWordSimilarity`, and `stopwordSpanSimilarity`—act as Levenshtein-based guards in `VocabularyRescorer.ctcTokenRescore(...)`. They ensure that a boosted term only replaces the original hypothesis if the strings are sufficiently alike. This prevents false substitutions where acoustically similar but semantically different words (like "right" and "write") might otherwise trigger incorrect bias injections.

### Does context biasing require retraining the acoustic model?

No. FluidAudio’s context biasing is a **post-processing rescoring** technique applied during the CTC decoding phase. The acoustic model remains unchanged; instead, the `VocabularyRescorer` manipulates the token log-probabilities emitted by the model. This architecture allows users to swap domain vocabularies instantly without the computational cost or data requirements of fine-tuning or retraining the neural network.