How FluidAudio Handles Streaming vs Batch Processing for ASR: Architecture and Implementation

FluidAudio automatically routes audio transcription requests to either batch or streaming pipelines at runtime based on configurable duration thresholds, using disk-backed chunking with overlapping windows for large files and in-memory processing for shorter clips.

FluidAudio is an open-source Swift framework for automatic speech recognition (ASR) that optimizes memory utilization by dynamically selecting between batch and streaming processing modes. The system, implemented across AsrManager.swift and related components in the fluidinference/fluidaudio repository, evaluates audio file characteristics against ASRConfig thresholds to determine the most efficient processing path without requiring manual mode selection from developers.

Configuration-Driven Processing Mode Selection

The decision logic governing streaming vs batch processing for ASR resides in ASRConfig, defined in Sources/FluidAudio/ASR/AsrTypes.swift. This configuration struct exposes two critical properties that control runtime behavior:

• streamingEnabled: Boolean flag defaulting to true
• streamingThreshold: Sample count threshold defaulting to 480,000 samples (approximately 30 seconds at 16kHz)

When AsrManager.transcribe(_ url: URL, source: AudioSource) receives a file transcription request, it calculates the estimated sample count after accounting for resampling ratios (sampleRateRatio). If estimatedSamples exceeds streamingThreshold, the method delegates to transcribeStreaming; otherwise, it loads the entire audio vector into memory and invokes transcribe(_ audioSamples: [Float]) for single-pass batch processing.

The Batch Processing Path

For audio files below the threshold, FluidAudio employs batch processing that minimizes computational overhead through single-pass inference. The batch path executes the following sequence:

1. audioConverter.resampleAudioFile loads the complete audio file into memory as a [Float] array
2. The sample buffer is padded to ASRConstants.maxModelSamples to match model encoder requirements
3. executeMLInferenceWithTimings runs once over the entire padded buffer
4. processTranscriptionResult constructs the final ASRResult containing text, confidence scores, and token timestamps

Because the entire audio vector resides in RAM throughout processing, this path delivers minimal latency for short clips but scales poorly with file duration, making the threshold-based routing essential for memory-constrained environments.

The Streaming Processing Path

When file duration exceeds the configured threshold, FluidAudio activates its streaming architecture to constrain memory usage regardless of file size. This implementation centers on AsrManager.transcribeStreaming(_ url: URL, source: AudioSource), which orchestrates three specialized components:

StreamingAudioSampleSource (Sources/FluidAudio/VAD/VadManager+Streaming.swift) provides disk-backed random-access reads without loading the full file into memory, maintaining constant memory usage even for hours-long recordings.

ChunkProcessor (Sources/FluidAudio/ASR/ChunkProcessor.swift) implements stateless chunking logic that computes:

• chunkSamples: Window size (~14.96 seconds)
• overlapSamples: ~2 seconds of overlap to preserve encoder left-context
• strideSamples: Step size between consecutive windows

For each window, ChunkProcessor reads samples from the disk-backed source, applies padding, executes executeMLInferenceWithTimings, and collects token-timestamp-confidence triples. After processing all windows, mergeChunks resolves overlapping regions into a unified token stream.

Real-Time Microphone Streaming

For live audio capture scenarios, FluidAudio exposes AsrManager.transcribeStreamingChunk, defined in Sources/FluidAudio/ASR/AsrTranscription.swift (lines 86-124). This low-level API maintains decoder state continuity across chunks through:

• microphoneDecoderState: Persistent decoder cache for microphone input
• systemDecoderState: Separate cache for system audio sources

Each incoming buffer continues transcription where the previous chunk ended, avoiding the discontinuities typical of naive chunked processing. This architecture enables continuous streaming ASR with latency constrained only by the buffer size (typically 4096 samples) rather than utterance duration.

Implementation Examples

Simple Batch Transcription

For short files under the default 30-second threshold, FluidAudio automatically selects batch processing:

import FluidAudio

let manager = AsrManager()
await manager.initialize(models: myAsrModels)

let audioURL = URL(fileURLWithPath: "/path/to/short.wav")
let result = try await manager.transcribe(audioURL)
print(result.text)

This routes through AsrManager.transcribe(_:source:) and executes single-pass inference without chunking overhead.

Forced Streaming for Long Files

Override the threshold to force streaming mode for memory-efficient processing of large files:

import FluidAudio

var cfg = ASRConfig()
cfg.streamingThreshold = 100_000  // ~6 seconds at 16kHz

let manager = AsrManager(config: cfg)
await manager.initialize(models: myAsrModels)

let longFile = URL(fileURLWithPath: "/big/podcast.wav")
let result = try await manager.transcribe(longFile)  // Streaming path

The estimated sample count exceeds the reduced threshold, triggering transcribeStreaming with ChunkProcessor handling overlapping windows.

Real-Time Microphone Integration

Implement continuous transcription from live audio capture:

import FluidAudio
import AVFoundation

let manager = AsrManager()
await manager.initialize(models: myAsrModels)

let mic = AVAudioEngine()
mic.inputNode.installTap(onBus: 0, bufferSize: 4096, format: nil) { buffer, _ in
    Task {
        let (tokens, timestamps, confidences, _) = try await manager.transcribeStreamingChunk(
            buffer.floatChannelData!.pointee,
            source: .microphone,
            previousTokens: [],
            isLastChunk: false
        )
        // Process live tokens...
    }
}

transcribeStreamingChunk maintains decoder state internally, enabling seamless continuation across audio buffers.

Summary

• Dynamic Routing: AsrManager selects between batch and streaming modes at runtime based on ASRConfig.streamingThreshold (default 480,000 samples)
• Batch Path: Loads entire audio files into memory for single-pass inference via executeMLInferenceWithTimings, optimal for files under ~30 seconds
• Streaming Path: Uses disk-backed StreamingAudioSampleSource and ChunkProcessor to process audio in overlapping ~15-second windows with constant memory usage
• Live Streaming: transcribeStreamingChunk maintains per-source decoder state (microphoneDecoderState) for continuous real-time transcription without file backing
• Unified Core: Both processing modes share the identical inference pipeline (executeMLInferenceWithTimings), ensuring consistent accuracy across batch and streaming scenarios

Frequently Asked Questions

How does FluidAudio determine whether to use batch or streaming processing?

The AsrManager.transcribe(_:source:) method calculates the estimated sample count after resampling and compares it against ASRConfig.streamingThreshold. If the estimated samples exceed the threshold (default 480,000 samples or approximately 30 seconds at 16kHz), the system automatically routes to transcribeStreaming; otherwise, it processes the file in batch mode by loading the entire audio buffer into memory.

What is the memory advantage of FluidAudio's streaming implementation?

FluidAudio's streaming path creates a disk-backed StreamingAudioSampleSource that provides random-access reads without loading the full file into memory, significantly reducing RAM usage for large audio files. The ChunkProcessor then walks the file in overlapping windows (approximately 14.96 seconds per chunk with 2-second overlap) and processes each segment individually before merging results.

Can FluidAudio handle real-time microphone input for live transcription?

Yes, the transcribeStreamingChunk method in AsrTranscription.swift supports real-time microphone streaming by maintaining a per-source decoder state (microphoneDecoderState or systemDecoderState). This allows continuous transcription where each incoming buffer continues where the previous chunk left off, enabling low-latency live ASR without requiring intermediate file storage.

Does FluidAudio use the same model for both batch and streaming ASR?

Yes, both batch and streaming paths utilize the same core inference method executeMLInferenceWithTimings defined in AsrTranscription.swift. The only differences lie in memory handling (full buffer versus chunked disk reads) and post-processing (merging overlapping chunks via ChunkProcessor.mergeChunks), ensuring consistent recognition accuracy regardless of processing mode.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}