Streaming vs Offline VAD Processing in FluidAudio: Architecture and Implementation Guide

TLDR: Offline VAD processes entire audio files in a single pass returning per-chunk probabilities, while streaming VAD processes audio incrementally with explicit state management to generate real-time speech start/end events.

The fluidinference/fluidaudio repository provides a Swift implementation of Voice Activity Detection (VAD) using the Silero model, offering two distinct processing modes tailored to different application requirements. Understanding the architectural differences between streaming and offline VAD processing is critical for selecting the appropriate API and achieving optimal performance in either batch analysis or real-time audio pipelines.

Core Architectural Differences

Offline VAD Processing

Offline VAD operates as a stateless batch processor. When you invoke process(_ url:) or process(_ samples:) in Sources/FluidAudio/VAD/VadManager.swift, the method iterates through the entire audio buffer in 4096-sample chunks. The internal LSTM hidden and cell states advance automatically within the processAudioSamples function, but these remain invisible to the caller. The API returns a flat array of VadResult objects, each containing the voice probability, a boolean isVoiceActive flag, processing time, and the next hidden state reference for internal use.

Streaming VAD Processing

Streaming VAD exposes the underlying state machine to support incremental processing. The entry point makeStreamState() in Sources/FluidAudio/VAD/VadManager+Streaming.swift initializes a fresh VadStreamState object that the caller must retain. Each subsequent call to processStreamingChunk(_:state:config:returnSeconds:timeResolution:) consumes a single audio chunk, updates the LSTM state within the provided VadStreamState, and evaluates Silero-style hysteresis thresholds (positive and negative thresholds with speech padding and minimum silence requirements). The method returns a VadStreamResult containing the updated state, the raw probability, and an optional VadStreamEvent indicating speechStart or speechEnd with precise timestamps.

API Implementation and State Management

The fundamental distinction lies in state ownership. In VadManager.swift, the processAudioSamples function manages a local LSTMState variable that persists only for the duration of the batch loop. Conversely, VadManager+Streaming.swift requires the caller to maintain the VadStreamState across asynchronous chunk deliveries, enabling real-time pipelines to process live microphone input without buffering entire files.

The streaming implementation incorporates a dedicated state machine (lines 57-88 of VadManager+Streaming.swift) that tracks transition counters and timing buffers to generate events. This logic is absent from the offline path, which simply returns raw probabilities and leaves event detection to post-processing.

Configuration and Performance Characteristics

Offline Configuration

Offline processing utilizes VadConfig exclusively, specifying model parameters and basic thresholds. The 4096-sample chunk size (approximately 256ms at 16kHz) represents the only processing granularity. Performance is optimized for throughput rather than latency, making it ideal for transcribing recorded meetings or analyzing podcast archives.

Streaming Configuration

Streaming processing requires VadSegmentationConfig (defined in Sources/FluidAudio/VAD/VadTypes.swift), which exposes granular hysteresis controls:

• Positive threshold: Probability required to trigger speech detection
• Negative threshold: Probability below which speech ends
• Speech padding: Frames to include before/after detected speech
• Minimum silence: Duration of silence required to declare speech end

These parameters enable sub-second latency and early-exit handling, critical for live transcription or voice-controlled interfaces.

Summary

• Offline VAD provides batch processing via process(_:) in VadManager.swift, returning complete [VadResult] arrays with per-chunk probabilities but no built-in event generation.
• Streaming VAD enables real-time processing through processStreamingChunk in VadManager+Streaming.swift, requiring explicit VadStreamState management and emitting speechStart/speechEnd events via configurable hysteresis.
• Both modes utilize the same underlying Silero model; the distinction resides in API state exposure and event generation capabilities.
• Select offline processing for file-based analysis and streaming for live audio pipelines requiring immediate voice activity notifications.

Frequently Asked Questions

When should I use streaming VAD versus offline VAD?

Use streaming VAD when processing live audio from microphones or network streams where you need immediate notification of speech start and end events with minimal latency. Use offline VAD when analyzing recorded files or bulk audio archives where you can afford higher latency in exchange for simpler stateless API usage and complete probability arrays for post-processing.

How does state management differ between the two modes?

In offline VAD, the VadManager internally manages LSTM hidden and cell states during the processAudioSamples call, discarding them after returning the [VadResult] array. In streaming VAD, you must instantiate and persist a VadStreamState object via makeStreamState(), passing it to each processStreamingChunk call so the LSTM state and hysteresis counters survive across chunk boundaries.

Can I switch from offline to streaming processing mid-application?

Yes, the VadManager instance supports both modes simultaneously. You can call process(_:) for batch files while maintaining separate VadStreamState instances for live streams. The underlying Silero model weights are shared, but ensure thread-safe access if processing multiple streams concurrently, as the LSTM state is not thread-safe across different VadStreamState objects.

What configuration options are unique to streaming VAD?

Streaming VAD accepts a VadSegmentationConfig parameter (defined in VadTypes.swift) that exposes Silero-style hysteresis parameters: positive and negative probability thresholds, speech padding frames, and minimum silence duration. These control when speechStart and speechEnd events fire. Offline VAD uses only VadConfig for basic model parameters and lacks segmentation-specific tuning options.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}