# How FluidAudio Handles Streaming vs Batch Processing for ASR: Architecture and Implementation

> FluidAudio intelligently routes ASR requests to streaming or batch processing using configurable thresholds. Discover its runtime architecture and implementation details.

- Repository: [Fluid Inference/fluidaudio](https://github.com/fluidinference/fluidaudio)
- Tags: architecture
- Published: 2026-03-02

---

**FluidAudio automatically routes audio transcription requests to either batch or streaming pipelines at runtime based on configurable duration thresholds, using disk-backed chunking with overlapping windows for large files and in-memory processing for shorter clips.**

FluidAudio is an open-source Swift framework for automatic speech recognition (ASR) that optimizes memory utilization by dynamically selecting between batch and streaming processing modes. The system, implemented across [`AsrManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/AsrManager.swift) and related components in the `fluidinference/fluidaudio` repository, evaluates audio file characteristics against `ASRConfig` thresholds to determine the most efficient processing path without requiring manual mode selection from developers.

## Configuration-Driven Processing Mode Selection

The decision logic governing streaming vs batch processing for ASR resides in **`ASRConfig`**, defined in [`Sources/FluidAudio/ASR/AsrTypes.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/AsrTypes.swift). This configuration struct exposes two critical properties that control runtime behavior:

- **`streamingEnabled`**: Boolean flag defaulting to `true`
- **`streamingThreshold`**: Sample count threshold defaulting to **480,000 samples** (approximately 30 seconds at 16kHz)

When **`AsrManager.transcribe(_ url: URL, source: AudioSource)`** receives a file transcription request, it calculates the estimated sample count after accounting for resampling ratios (`sampleRateRatio`). If `estimatedSamples` exceeds `streamingThreshold`, the method delegates to `transcribeStreaming`; otherwise, it loads the entire audio vector into memory and invokes `transcribe(_ audioSamples: [Float])` for single-pass batch processing.

## The Batch Processing Path

For audio files below the threshold, FluidAudio employs **batch processing** that minimizes computational overhead through single-pass inference. The batch path executes the following sequence:

1. **`audioConverter.resampleAudioFile`** loads the complete audio file into memory as a `[Float]` array
2. The sample buffer is padded to `ASRConstants.maxModelSamples` to match model encoder requirements
3. **`executeMLInferenceWithTimings`** runs once over the entire padded buffer
4. **`processTranscriptionResult`** constructs the final `ASRResult` containing text, confidence scores, and token timestamps

Because the entire audio vector resides in RAM throughout processing, this path delivers minimal latency for short clips but scales poorly with file duration, making the threshold-based routing essential for memory-constrained environments.

## The Streaming Processing Path

When file duration exceeds the configured threshold, FluidAudio activates its **streaming architecture** to constrain memory usage regardless of file size. This implementation centers on **`AsrManager.transcribeStreaming(_ url: URL, source: AudioSource)`**, which orchestrates three specialized components:

**`StreamingAudioSampleSource`** (`Sources/FluidAudio/VAD/VadManager+Streaming.swift`) provides disk-backed random-access reads without loading the full file into memory, maintaining constant memory usage even for hours-long recordings.

**`ChunkProcessor`** ([`Sources/FluidAudio/ASR/ChunkProcessor.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/ChunkProcessor.swift)) implements stateless chunking logic that computes:
- `chunkSamples`: Window size (~14.96 seconds)
- `overlapSamples`: ~2 seconds of overlap to preserve encoder left-context
- `strideSamples`: Step size between consecutive windows

For each window, `ChunkProcessor` reads samples from the disk-backed source, applies padding, executes `executeMLInferenceWithTimings`, and collects token-timestamp-confidence triples. After processing all windows, **`mergeChunks`** resolves overlapping regions into a unified token stream.

## Real-Time Microphone Streaming

For live audio capture scenarios, FluidAudio exposes **`AsrManager.transcribeStreamingChunk`**, defined in [`Sources/FluidAudio/ASR/AsrTranscription.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/ASR/AsrTranscription.swift) (lines 86-124). This low-level API maintains **decoder state continuity** across chunks through:
- **`microphoneDecoderState`**: Persistent decoder cache for microphone input
- **`systemDecoderState`**: Separate cache for system audio sources

Each incoming buffer continues transcription where the previous chunk ended, avoiding the discontinuities typical of naive chunked processing. This architecture enables continuous streaming ASR with latency constrained only by the buffer size (typically 4096 samples) rather than utterance duration.

## Implementation Examples

### Simple Batch Transcription

For short files under the default 30-second threshold, FluidAudio automatically selects batch processing:

```swift
import FluidAudio

let manager = AsrManager()
await manager.initialize(models: myAsrModels)

let audioURL = URL(fileURLWithPath: "/path/to/short.wav")
let result = try await manager.transcribe(audioURL)
print(result.text)

```

*This routes through `AsrManager.transcribe(_:source:)` and executes single-pass inference without chunking overhead.*

### Forced Streaming for Long Files

Override the threshold to force streaming mode for memory-efficient processing of large files:

```swift
import FluidAudio

var cfg = ASRConfig()
cfg.streamingThreshold = 100_000  // ~6 seconds at 16kHz

let manager = AsrManager(config: cfg)
await manager.initialize(models: myAsrModels)

let longFile = URL(fileURLWithPath: "/big/podcast.wav")
let result = try await manager.transcribe(longFile)  // Streaming path

```

*The estimated sample count exceeds the reduced threshold, triggering `transcribeStreaming` with `ChunkProcessor` handling overlapping windows.*

### Real-Time Microphone Integration

Implement continuous transcription from live audio capture:

```swift
import FluidAudio
import AVFoundation

let manager = AsrManager()
await manager.initialize(models: myAsrModels)

let mic = AVAudioEngine()
mic.inputNode.installTap(onBus: 0, bufferSize: 4096, format: nil) { buffer, _ in
    Task {
        let (tokens, timestamps, confidences, _) = try await manager.transcribeStreamingChunk(
            buffer.floatChannelData!.pointee,
            source: .microphone,
            previousTokens: [],
            isLastChunk: false
        )
        // Process live tokens...
    }
}

```

*`transcribeStreamingChunk` maintains decoder state internally, enabling seamless continuation across audio buffers.*

## Summary

- **Dynamic Routing**: `AsrManager` selects between batch and streaming modes at runtime based on `ASRConfig.streamingThreshold` (default 480,000 samples)
- **Batch Path**: Loads entire audio files into memory for single-pass inference via `executeMLInferenceWithTimings`, optimal for files under ~30 seconds
- **Streaming Path**: Uses disk-backed `StreamingAudioSampleSource` and `ChunkProcessor` to process audio in overlapping ~15-second windows with constant memory usage
- **Live Streaming**: `transcribeStreamingChunk` maintains per-source decoder state (`microphoneDecoderState`) for continuous real-time transcription without file backing
- **Unified Core**: Both processing modes share the identical inference pipeline (`executeMLInferenceWithTimings`), ensuring consistent accuracy across batch and streaming scenarios

## Frequently Asked Questions

### How does FluidAudio determine whether to use batch or streaming processing?

The `AsrManager.transcribe(_:source:)` method calculates the estimated sample count after resampling and compares it against `ASRConfig.streamingThreshold`. If the estimated samples exceed the threshold (default 480,000 samples or approximately 30 seconds at 16kHz), the system automatically routes to `transcribeStreaming`; otherwise, it processes the file in batch mode by loading the entire audio buffer into memory.

### What is the memory advantage of FluidAudio's streaming implementation?

FluidAudio's streaming path creates a disk-backed `StreamingAudioSampleSource` that provides random-access reads without loading the full file into memory, significantly reducing RAM usage for large audio files. The `ChunkProcessor` then walks the file in overlapping windows (approximately 14.96 seconds per chunk with 2-second overlap) and processes each segment individually before merging results.

### Can FluidAudio handle real-time microphone input for live transcription?

Yes, the `transcribeStreamingChunk` method in [`AsrTranscription.swift`](https://github.com/fluidinference/fluidaudio/blob/main/AsrTranscription.swift) supports real-time microphone streaming by maintaining a per-source decoder state (`microphoneDecoderState` or `systemDecoderState`). This allows continuous transcription where each incoming buffer continues where the previous chunk left off, enabling low-latency live ASR without requiring intermediate file storage.

### Does FluidAudio use the same model for both batch and streaming ASR?

Yes, both batch and streaming paths utilize the same core inference method `executeMLInferenceWithTimings` defined in [`AsrTranscription.swift`](https://github.com/fluidinference/fluidaudio/blob/main/AsrTranscription.swift). The only differences lie in memory handling (full buffer versus chunked disk reads) and post-processing (merging overlapping chunks via `ChunkProcessor.mergeChunks`), ensuring consistent recognition accuracy regardless of processing mode.