How Inverse Text Normalization (ITN) Works in FluidAudio: A Deep Dive into the Swift-NeMo Integration

FluidAudio implements inverse text normalization through a Swift wrapper that dynamically loads the NeMo native library, providing context-aware text transformation with ambiguous word disambiguation via NLTagger.

Inverse text normalization (ITN) converts spoken-form text generated by automatic speech recognition (ASR) systems into written-form text suitable for display or downstream processing. In the FluidAudio repository, this functionality is implemented as a thin Swift layer over NVIDIA's NeMo text normalization engine, offering both high-performance native operations and intelligent ambiguity resolution.

Architecture of the ITN Engine

The core ITN implementation resides in Sources/FluidAudio/ITN/TextNormalizer.swift, which exposes a Sendable singleton interface via TextNormalizer.shared. Rather than reimplementing normalization logic in Swift, this class serves as a Foreign Function Interface (FFI) wrapper around the NeMo library compiled as a native binary.

The architecture follows a defensive design pattern: if the native library fails to load or required symbols are missing, the system gracefully degrades by returning the original unnormalized text rather than crashing or producing errors.

Dynamic Native Library Loading

At initialization, the TextNormalizer class attempts to dynamically bind to the NeMo library using dlopen(nil, RTLD_NOW) to access the current process's symbol table (lines 75‑87). It then resolves four critical C function pointers through unsafeBitCast:

• nemo_normalize – for single-expression normalization
• nemo_normalize_sentence – for sentence-level processing
• nemo_add_rule and nemo_remove_rule – for custom rule management

If any symbol lookup fails (lines 90‑118), the isNativeAvailable property returns false, and all subsequent normalization calls bypass the native layer, returning the input string unchanged. This ensures the library remains functional even in environments where the NeMo binary is not present.

Normalization Entry Points and Methods

The public API exposes three distinct normalization modes, each delegating to specific native functions:

normalize(_:) handles single-expression mode, forwarding the entire input string directly to nemo_normalize. This is optimal for isolated spoken-form tokens like "two hundred thirty two" → "232".

normalizeSentence(_:) implements sentence-level processing (lines 73‑87). This method scans full transcripts, identifies normalizable spans, and uses an NLTagger-based filter to avoid false positives on ambiguous words like "period" or "dash" that could be either punctuation commands or ordinary nouns.

normalizeSentence(_:maxSpanTokens:) extends sentence mode with a configurable maximum span length (lines 89‑102). This prevents excessive memory usage or latency when processing extremely long sentences by limiting the number of tokens considered in a single normalization pass.

Each method first attempts to use the sentence-mode FFI functions, falling back to single-expression normalize(_:) when sentence-mode symbols are unavailable.

Handling Ambiguous Words with NLTagger

A critical challenge in ITN is distinguishing between homophones that function as spoken punctuation commands versus ordinary vocabulary. FluidAudio addresses this through the filterAmbiguousWords(in:) method (lines 123‑168).

The system maintains a static set ambiguousWords (lines 30‑35) containing tokens like "period", "dash", "colon", "comma", and "point". When processing a sentence, the method instantiates NLTagger with the lexical class scheme (NSLinguisticTagSchemeLexicalClass in Objective-C terms, or .lexicalClass in Swift).

For each ambiguous token, the tagger determines its part of speech. If the token is tagged as noun, verb, adjective, or adverb, it is treated as ordinary vocabulary and left unnormalized. If tagged as any other category (or untagged), it is passed to the native normalizer for potential conversion to punctuation. This contextual disambiguation drastically reduces false positives in transcripts like "I need a period of rest" versus "Add a period at the end".

Custom Rule Management

Beyond the built-in NeMo normalization rules, FluidAudio exposes an API for runtime customization of the spoken-to-written mapping:

• addRule(spoken:written:) (lines 58‑64) invokes nemo_add_rule to register new mappings
• removeRule(spoken:) (lines 66‑71) invokes nemo_remove_rule to delete specific rules
• clearRules() (lines 73‑75) resets all custom rules
• ruleCount (lines 77‑80) returns the current number of active custom rules

These methods allow applications to domain-specific vocabulary, such as adding "gee pee tee" → "GPT" for technical transcripts, without modifying the underlying NeMo library.

Practical Implementation Examples

The following Swift code demonstrates the complete ITN workflow using the TextNormalizer API:

import FluidAudio

// Basic single-expression normalization
let spoken = "two hundred thirty two"
let written = TextNormalizer.shared.normalize(spoken)
// Result: "232"

// Sentence-level processing with ambiguity resolution
let sentence = "I need a period of five minutes."
let normalized = TextNormalizer.shared.normalizeSentence(sentence)
// Result: "I need a . of five minutes."
// Note: "period" preserved as noun, "." used for punctuation command

// Constrained span length for long transcripts
let longSentence = "january fifth twenty twenty five is the date"
let limited = TextNormalizer.shared.normalizeSentence(
    longSentence,
    maxSpanTokens: 3
)
// Result: "January 5, 2025 is the date"

// Custom rule injection
TextNormalizer.shared.addRule(spoken: "gee pee tee", written: "GPT")
let custom = TextNormalizer.shared.normalizeSentence("I love gee pee tee")
// Result: "I love GPT"

// Library status verification
if TextNormalizer.shared.isNativeAvailable,
   let version = TextNormalizer.shared.version {
    print("NeMo ITN version: \(version)")
}

Summary

FluidAudio's inverse text normalization system combines native performance with Swift-level safety and linguistic intelligence:

• Dynamic FFI binding to the NeMo Rust library via dlopen and unsafeBitCast, with graceful degradation when the native layer is unavailable
• Three normalization modes—single-expression, sentence-level, and constrained-span—accessible through normalize(_:) and normalizeSentence(_:) methods in Sources/FluidAudio/ITN/TextNormalizer.swift
• Contextual disambiguation using NLTagger to filter ambiguous words (period, dash, colon) based on part-of-speech tagging, preventing false positives in ordinary vocabulary
• Runtime customization via addRule(spoken:written:), removeRule(spoken:), and clearRules() for domain-specific vocabulary without recompilation

Frequently Asked Questions

How does FluidAudio handle cases where the NeMo native library is not available?

If the NeMo library fails to load or required symbols cannot be resolved during the dlopen initialization in TextNormalizer.swift (lines 75-118), the isNativeAvailable property returns false. In this fallback state, all normalization methods return the original input string unchanged, ensuring the application continues to function without crashing or producing errors.

Why does FluidAudio use NLTagger for ambiguous word filtering?

The ITN system uses NLTagger with the lexical class scheme (lines 123-168) to disambiguate words like "period," "dash," and "colon" that can function either as spoken punctuation commands or as ordinary nouns/verbs. By checking the part-of-speech tag, the system leaves vocabulary words untouched while converting actual punctuation commands, drastically reducing false positives in transcripts.

What is the difference between normalize() and normalizeSentence() in FluidAudio?

The normalize(_:) method performs single-expression normalization, forwarding the entire string directly to nemo_normalize for straightforward conversions like "two hundred" → "200". In contrast, normalizeSentence(_:) processes full transcripts using sentence-mode scanning with span detection and ambiguous-word filtering, making it suitable for complex sentences containing mixed content that requires contextual analysis.

Can I add custom normalization rules without modifying the NeMo library source code?

Yes, FluidAudio exposes runtime rule management through addRule(spoken:written:), removeRule(spoken:), and clearRules() (lines 58-80 in TextNormalizer.swift). These methods call the underlying nemo_add_rule and nemo_remove_rule FFI functions, allowing you to register domain-specific mappings like "gee pee tee" → "GPT" at runtime without recompiling the native library.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}