# How to Configure the Offline Diarization Pipeline with VBx Clustering in FluidAudio

> Learn to configure the offline diarization pipeline with VBx clustering in FluidAudio. Set custom thresholds and warm-start parameters for efficient speaker diarization.

- Repository: [Fluid Inference/fluidaudio](https://github.com/fluidinference/fluidaudio)
- Tags: how-to-guide
- Published: 2026-03-02

---

**Configure the offline diarization pipeline with VBx clustering by initializing `OfflineDiarizerConfig` with custom thresholds, warm-start parameters, and optional speaker constraints, then passing it to `OfflineDiarizerManager` for processing.**

FluidAudio provides a high-performance offline diarization system that uses Variational Bayes (VBx) clustering to group speaker embeddings into distinct clusters. The pipeline is fully configurable through the `OfflineDiarizerConfig` struct defined in [`Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerTypes.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerTypes.swift). When you configure the offline diarization pipeline with VBx clustering, you control everything from the initial Agglomerative Hierarchical Clustering (AHC) warm-start to the final Expectation-Maximization (EM) iteration limits.

## Pipeline Architecture and VBx Integration

The offline diarization pipeline consists of three sequential stages executed by `OfflineDiarizerManager.process(...)`:

1. **Segmentation** – detects speech activity and creates short audio chunks.
2. **Embedding extraction** – converts each chunk into a 256‑dimensional speaker embedding plus a 128‑dimensional PLDA "rho" vector.
3. **Speaker clustering** – groups embeddings using the **VBx** (Variational Bayes) algorithm implemented in `VBxClustering.refine(...)`.

According to the fluidinference/fluidaudio source code, the `VBxClustering` class in [`Sources/FluidAudio/Diarizer/Offline/Clustering/VBxClustering.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Clustering/VBxClustering.swift) performs the full VBx EM loop, returning a `VBxOutput` containing the gamma matrix, mixture weights (`pi`), hard assignments, and ELBO history.

## Core Configuration Parameters

All VBx-related knobs are exposed through `OfflineDiarizerConfig`. The following properties control clustering behavior:

| Property | Description | Default (community‑1) |
|----------|-------------|----------------------|
| `clustering.threshold` | Euclidean distance threshold for the initial AHC warm-start | `0.6` |
| `clustering.warmStartFa` | VBx warm-start precision parameter | `0.07` |
| `clustering.warmStartFb` | VBx warm-start recall parameter | `0.8` |
| `vbx.maxIterations` | Maximum EM iterations for VBx | `20` |
| `vbx.convergenceTolerance` | ELBO-based convergence epsilon | `1e‑4` |
| `clustering.minSpeakers` | Minimum speaker constraint | `nil` |
| `clustering.maxSpeakers` | Maximum speaker constraint | `nil` |
| `clustering.numSpeakers` | Exact speaker count override | `nil` |

## Configuration Flow Through the Pipeline

### CLI Entry Point

The `ProcessCommand` in [`Sources/FluidAudioCLI/Commands/ProcessCommand.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudioCLI/Commands/ProcessCommand.swift) (lines 52‑63) parses `--mode offline` and constructs an `OfflineDiarizerConfig` with user-supplied thresholds and optional embedding export paths.

### Manager Orchestration

`OfflineDiarizerManager.process(...)` in [`Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Core/OfflineDiarizerManager.swift) (lines 241‑257) validates the configuration via `config.validate()`, loads CoreML models, runs segmentation and embedding extraction in parallel, and invokes `VBxClustering.refineWithConstraints(...)`.

### VBx Clustering Implementation

The `VBxClustering.refine(...)` method at lines 40‑64 of [`VBxClustering.swift`](https://github.com/fluidinference/fluidaudio/blob/main/VBxClustering.swift) executes the VBx EM loop. If speaker constraints are present, `refineWithConstraints` performs an additional K-Means re-clustering step using `KMeansClustering`, marking the output as adjusted via `VBxOutput.wasAdjusted`.

### Speaker Count Constraints

The `SpeakerCountConstraints` struct in [`Sources/FluidAudio/Diarizer/Offline/Clustering/SpeakerCountConstraints.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Clustering/SpeakerCountConstraints.swift) handles logic for min/max/exact speaker enforcement, automatically triggering K-Means when VBx output falls outside specified ranges.

## Practical Configuration Examples

**Use default VBx settings**

```swift
let cfg = OfflineDiarizerConfig()

```

**Adjust clustering threshold**

Lower values create more initial clusters for finer separation:

```swift
let cfg = OfflineDiarizerConfig(clusteringThreshold: 0.5)

```

**Tune VBx warm-start parameters**

Higher `Fa` improves precision; higher `Fb` improves recall:

```swift
var cfg = OfflineDiarizerConfig()
cfg.Fa = 0.1
cfg.Fb = 0.6

```

**Increase EM iterations**

Allow more convergence steps at the cost of CPU time:

```swift
var cfg = OfflineDiarizerConfig()
cfg.maxVBxIterations = 30

```

**Enforce speaker range**

Force re-clustering if detected speakers fall outside [2, 5]:

```swift
let cfg = OfflineDiarizerConfig().withSpeakers(min: 2, max: 5)

```

**Force exact speaker count**

Override AHC/VBx estimates entirely:

```swift
let cfg = OfflineDiarizerConfig().withSpeakers(exactly: 3)

```

**Export embeddings for debugging**

```swift
let cfg = OfflineDiarizerConfig(embeddingExportPath: "/tmp/embeddings.json")

```

**Complete integration example**

```swift
let manager = OfflineDiarizerManager(config: cfg)
let models = try await OfflineDiarizerModels.load(from: modelDir)
manager.initialize(models: models)
let result = try await manager.process(audioSource: diskSource,
                                      audioLoadingSeconds: loadDuration)

```

## Summary

- Configure the offline diarization pipeline with VBx clustering using the `OfflineDiarizerConfig` struct in [`OfflineDiarizerTypes.swift`](https://github.com/fluidinference/fluidaudio/blob/main/OfflineDiarizerTypes.swift).
- Adjust `clustering.threshold`, `Fa`, and `Fb` to control the AHC warm-start and VBx precision/recall trade-offs.
- Set `maxVBxIterations` and `convergenceTolerance` to balance accuracy against computational cost.
- Use `withSpeakers()` constraints to enforce minimum, maximum, or exact speaker counts via `SpeakerCountConstraints`.
- Pass the configuration to `OfflineDiarizerManager.process(...)` to execute the full pipeline including segmentation, embedding extraction, and VBx clustering.

## Frequently Asked Questions

### What is the default VBx clustering threshold in FluidAudio?

The default Euclidean distance threshold for the initial Agglomerative Hierarchical Clustering warm-start is `0.6`, defined in `OfflineDiarizerConfig`. Lowering this value creates more initial clusters and can improve speaker separation for difficult audio.

### How do I force FluidAudio to detect exactly 3 speakers?

Use the `withSpeakers(exactly:)` convenience method when building your configuration: `let cfg = OfflineDiarizerConfig().withSpeakers(exactly: 3)`. This overrides the VBx estimate and applies K-Means clustering to enforce the exact count.

### Where does the VBx clustering logic live in the source code?

The core VBx algorithm is implemented in [`Sources/FluidAudio/Diarizer/Offline/Clustering/VBxClustering.swift`](https://github.com/fluidinference/fluidaudio/blob/main/Sources/FluidAudio/Diarizer/Offline/Clustering/VBxClustering.swift), specifically in the `refine(...)` method at lines 40‑64. The orchestration happens in [`OfflineDiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/OfflineDiarizerManager.swift) at lines 241‑257.

### Can I export speaker embeddings for external analysis?

Yes. Set the `embeddingExportPath` property in `OfflineDiarizerConfig` to a file path (e.g., [`/tmp/embeddings.json`](https://github.com/fluidinference/fluidaudio/blob/main//tmp/embeddings.json)). After processing, `OfflineDiarizerManager` writes a JSON file containing each embedding, chunk indices, speaker indices, and final cluster assignments.