# How FluidAudio Handles Audio Format Conversion for Inference

> FluidAudio ensures consistent model-ready audio data for inference by normalizing all inputs to 16 kHz mono Float32 using its AudioConverter class. Learn how it works.

- Repository: [Fluid Inference/fluidaudio](https://github.com/fluidinference/fluidaudio)
- Tags: how-to-guide
- Published: 2026-03-02

---

**FluidAudio normalizes every audio input to 16 kHz, mono, Float32 using the central `AudioConverter` class that wraps Apple's `AVAudioConverter`, ensuring consistent model-ready data for ASR, VAD, diarization, and TTS pipelines.**

The `fluidinference/fluidaudio` repository enforces strict audio normalization before any inference occurs. Regardless of source format—whether stereo 48 kHz Int16 PCM or mono 8 kHz files—the pipeline converts inputs to a standardized format consumable by all machine learning models.

## The 16 kHz Mono Float32 Target Standard

All inference components in FluidAudio expect audio in **16 kHz sample rate, single-channel mono, and 32-bit floating point (Float32)** format. This standard eliminates variability across ASR, VAD, diarization, and TTS modules, allowing models to assume consistent input characteristics.

The conversion guarantee is implemented in [`Sources/FluidAudio/Shared/AudioConverter.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Shared/AudioConverter.swift), where the `AudioConverter` type orchestrates three transformations: sample-rate conversion to 16 kHz, channel mixing to mono, and sample-format conversion to Float32.

## The AudioConverter Architecture

### Format Detection and Validation

Before processing, `AudioConverter.isTargetFormat` inspects the source `AVAudioFormat` to determine if conversion is necessary. This method validates sample rate, channel count, sample format, and interleaving status by comparing them against the target specification.

```swift
// From Sources/FluidAudio/Shared/AudioConverter.swift#L31-L37
// Checks if the format already matches 16kHz, mono, Float32
if format.sampleRate == targetSampleRate && 
   format.channelCount == 1 && 
   format.commonFormat == .pcmFormatFloat32 {
    return true
}

```

### Channel Mixing Strategy

Multi-channel audio undergoes downmixing to mono. For stereo sources, `AudioConverter.extractMonoFloat32` leverages `AVAudioConverter` to perform the mix. When handling sources with more than two channels, the implementation falls back to `linearResample`, which applies a manual linear mix to collapse channels into a single mono stream before resampling.

```swift
// Fallback linear mix for >2 channels in Sources/FluidAudio/Shared/AudioConverter.swift#L39-L46
private func linearResample(_ buffer: AVAudioPCMBuffer) -> [Float] {
    // Manual channel averaging for multi-channel inputs
    let channelCount = Int(buffer.format.channelCount)
    let frameLength = Int(buffer.frameLength)
    // ... linear mixing implementation
}

```

### Sample-Rate and Sample-Format Conversion

Rather than implementing custom DSP algorithms, FluidAudio delegates resampling to Apple's native `AVAudioConverter`. The `resampleWithAVAudio` method constructs a target `AVAudioFormat` with `sampleRate: outputRate` (16000) and invokes `convertBuffer` to handle both frequency conversion and bit-depth transformation (e.g., Int16 → Float32) in a single operation.

According to the source code comments in `Sources/FluidAudio/Shared/AudioConverter.swift#L11-L13`, this approach "Uses `AVAudioConverter` for all sample-rate, sample-format, and channel-count conversions," preserving audio quality through Apple's optimized signal processing.

## Stateless Design and Thread Safety

Each conversion operation instantiates a fresh `AVAudioConverter` instance, making the `AudioConverter` class stateless and thread-safe. As documented in the public initializer comments (`Sources/FluidAudio/Shared/AudioConverter.swift#L13-L15`), this design allows a single `AudioConverter` instance to be reused safely across concurrent file processing or streaming pipelines without locking or state corruption.

## Usage Examples

### File-Based Conversion

For batch processing or CLI workflows, the `resampleAudioFile(_:)` method reads any file supported by `AVAudioFile` and returns normalized Float32 samples.

```swift
import AVFoundation
import FluidAudio

public func loadSamples16kMono(path: String) async throws -> [Float] {
    let converter = AudioConverter()               // Stateless, can be reused
    return try await converter.resampleAudioFile(path: path)
}

```

This pattern appears throughout `Sources/FluidAudioCLI/Commands/*.swift`, where CLI entry points invoke this method before passing data to transcription or analysis models.

### Streaming Buffer Conversion

Real-time inference pipelines use `resampleBuffer(_:)` to process captured audio chunks without file I/O overhead.

```swift
import AVFoundation
import FluidAudio

let converter = AudioConverter()

func processChunk(_ pcmBuffer: AVAudioPCMBuffer) async throws {
    let samples = try await converter.resampleBuffer(pcmBuffer) // 16 kHz mono Float32
    // Feed `samples` to ASR/VAD/diarization pipelines
}

```

The `VadManager` implementation in `Sources/FluidAudio/VAD/VadManager.swift#L44-L60` demonstrates this streaming approach, calling either `resampleAudioFile(url)` for file inputs or `resampleBuffer(audioBuffer)` for live stream processing.

## Integration with Inference Components

All high-level inference modules—**VAD**, **ASR**, **diarization**, and **TTS**—consume audio exclusively through the `AudioConverter` interface. This centralized routing ensures that model inputs never contain unexpected sample rates or channel configurations.

As detailed in `Documentation/Guides/AudioConversion.md#L7-L10`, the design philosophy emphasizes that conversion uses `AVAudioConverter` for "sample-rate conversion, sample-format conversion (e.g., Int16 → Float32), and channel mixing (stereo → mono)." This unified preprocessing layer prevents format-related errors from reaching the inference engines.

## Summary

- **FluidAudio enforces a strict 16 kHz, mono, Float32 standard** for all model inputs through the `AudioConverter` class.
- **Format conversion wraps `AVAudioConverter`** for high-quality resampling, channel mixing, and bit-depth conversion without custom DSP code.
- **Stateless architecture** creates a fresh converter per operation, ensuring thread safety across concurrent file and stream processing.
- **Dual API surface** supports both file-based (`resampleAudioFile`) and buffer-based (`resampleBuffer`) workflows used by VAD, ASR, and CLI commands.
- **Multi-channel handling** automatically downmixes stereo to mono, with a linear mix fallback for sources exceeding two channels.

## Frequently Asked Questions

### What target format does FluidAudio require for model inference?

FluidAudio requires all audio inputs to be **16 kHz sample rate, single-channel (mono), and 32-bit floating point (Float32)**. This standard is enforced by the `AudioConverter` class before data reaches any ASR, VAD, diarization, or TTS component.

### How does FluidAudio handle stereo or multi-channel audio?

Stereo sources are downmixed to mono using `AVAudioConverter`. For audio with more than two channels, `AudioConverter` falls back to a manual linear mixing algorithm in `linearResample` that averages all channels into a single mono stream before resampling to 16 kHz.

### Is the AudioConverter thread-safe for concurrent processing?

Yes. `AudioConverter` is designed as a stateless utility that creates a fresh `AVAudioConverter` instance for each operation. As noted in `Sources/FluidAudio/Shared/AudioConverter.swift#L13-L15`, this allows the same converter instance to process multiple files or streams concurrently without synchronization issues.

### Why does FluidAudio use AVAudioConverter instead of custom resamplers?

The implementation relies exclusively on Apple's `AVAudioConverter` to ensure broadcast-quality sample-rate conversion and format transformation. According to [`Documentation/Guides/AudioConversion.md`](https://github.com/fluidinference/fluidaudio/blob/main/Documentation/Guides/AudioConversion.md), this choice avoids hand-rolled resampling algorithms, leveraging Apple's optimized signal processing for consistent, artifact-free conversion across all supported audio formats.