# Multi-Speaker Diarization with Overlapping Speech: Best Practices in FluidAudio > Master multi-speaker diarization with overlapping speech. FluidAudio offers best practices for accurate speaker separation even with complex audio. Learn how. - Repository: [Fluid Inference/fluidaudio](https://github.com/fluidinference/fluidaudio) - Tags: best-practices - Published: 2026-03-02 --- **FluidAudio handles multi-speaker diarization with overlapping speech by processing audio in overlapping chunks, filtering ambiguous frames where multiple speakers are active, and merging speaker embeddings across chunk boundaries using cosine-distance thresholds.** Multi-speaker diarization with overlapping speech remains one of the most challenging scenarios in audio processing, requiring sophisticated chunk management and speaker tracking to maintain consistent identities. The FluidAudio open-source framework implements a robust pipeline that addresses these challenges through configurable overlap parameters and intelligent masking strategies. This guide examines the source code architecture in `fluidinference/fluidaudio` and provides actionable best practices for configuring the diarization system when speakers talk simultaneously. ## Chunk Overlap Strategy for Boundary Stability The diarization pipeline processes long recordings in overlapping segments to ensure speech crossing chunk boundaries is captured consistently. In [`DiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/DiarizerManager.swift), the `chunkOverlap` value defined in `DiarizerConfig` is converted to samples and used to calculate the stride between consecutive chunks. The step size calculation guarantees that each successive chunk contains a shared region with the previous chunk: ```swift let overlapDuration = Int(config.chunkOverlap.rounded()) let stepSize = chunkSize - (sampleRate * overlapDuration) ``` As implemented in [`DiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/DiarizerManager.swift) (lines 24–27), this approach ensures that any speech segment appearing at a chunk boundary is processed twice, providing redundant embedding extraction that stabilizes speaker identity assignment across transitions. ## Mask Cleaning for Overlapping Frames To prevent noisy embeddings from frames containing multiple simultaneous speakers, the pipeline implements a cleaning mask in `processChunkWithSpeakerTracking`. After segmentation, the system generates per-speaker binary masks and explicitly filters frames where the sum of speaker probabilities exceeds a single-speaker threshold. In [`DiarizerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/DiarizerManager.swift) (lines 74–79), the logic discards ambiguous frames: ```swift let isClean: Float = speakerSum < 2.0 ? 1.0 : 0.0 speakerMask.append(speakerProb * isClean) ``` This **mask cleaning** step ensures that only frames containing a single active speaker contribute to the embedding extraction, eliminating the false speaker switches typically caused by overlapping speech artifacts. ## Speaker Assignment and Embedding Merging The `SpeakerManager` class maintains an in-memory database that assigns stable speaker IDs and resolves conflicts when speakers overlap across chunks. According to [`SpeakerManager.swift`](https://github.com/fluidinference/fluidaudio/blob/main/SpeakerManager.swift) (lines 21–33), the system stores speaker embeddings and updates them gradually to preserve identity consistency. When processing each segment, the `assignSpeaker` method (lines 35–50) implements the following logic: 1. Attempts to match the new embedding to an existing speaker within `speakerThreshold` cosine distance 2. If matched, updates the existing speaker's embedding (subject to `embeddingThreshold` to prevent overwriting high-confidence vectors) 3. If no match is found and the segment exceeds `minSpeechDuration`, creates a **new speaker ID** This threshold-based merging naturally handles overlapping speech scenarios because the clean mask ensures only one speaker's embedding is generated per frame, while the overlap strategy guarantees redundant processing of boundary-crossing speech. ## Recommended Configuration for Overlap-Heavy Recordings Based on the source code analysis, these parameters optimize performance for recordings with frequent speaker overlap: - **chunkDuration**: 10.0 seconds (default) — Balances memory usage with segmentation model context requirements - **chunkOverlap**: 2.0 seconds (minimum) — Guarantees boundary-crossing speech is captured twice; increase to 3.0 seconds for very dense overlapping speech - **speakerThreshold**: 0.60–0.70 — Tighter distance threshold prevents the same speaker from being split into multiple IDs during overlap - **embeddingThreshold**: 0.40–0.45 — Allows gradual embedding refinement without destabilizing high-confidence vectors - **minSpeechDuration**: 1.0 second — Filters out short bursts often caused by overlap artifacts - **debugMode**: true (development only) — Exposes the full `speakerDatabase` for inspecting speaker tracking decisions As defined in `DiarizerConfig` ([`DiarizerTypes.swift`](https://github.com/fluidinference/fluidaudio/blob/main/DiarizerTypes.swift), lines 29–34), these values control the chunking behavior and clustering sensitivity throughout the pipeline. ## Implementation Examples ### Basic Diarization with Overlap Handling This example demonstrates the standard configuration for processing recordings with overlapping speech: ```swift import FluidAudio let config = DiarizerConfig( chunkDuration: 10.0, chunkOverlap: 2.0, speakerThreshold: 0.65, embeddingThreshold: 0.45, debugMode: true ) let diarizer = DiarizerManager(config: config) let samples: [Float] = try AudioLoader.loadWavFile(at: url) if let models = try? await DiarizerModels.download() { diarizer.initialize(models: models) } let result = try diarizer.performCompleteDiarization(samples) for segment in result.segments { print("Speaker \(segment.speakerId): \(segment.startTimeSeconds)s - \(segment.endTimeSeconds)s") } if let db = result.speakerDatabase { print("Total speakers discovered: \(db.count)") } ``` ### Tuning Parameters for Dense Overlap Increase overlap duration and tighten matching thresholds for recordings with heavy simultaneous speech: ```swift let customConfig = DiarizerConfig( chunkOverlap: 3.0, // Larger overlap for redundancy speakerThreshold: 0.58, // Tighter matching to prevent splits embeddingThreshold: 0.42, minSpeechDuration: 0.8, debugMode: false ) let manager = DiarizerManager(config: customConfig) ``` The 3-second overlap provides additional redundancy when speech is dense, while the reduced `speakerThreshold` minimizes the risk of assigning multiple IDs to the same person during overlapping sections. ### Post-Processing to Collapse Near-Duplicate Speakers When automatic tracking produces separate IDs for the same speaker (rare but possible with heavy overlap), apply post-hoc merging based on embedding similarity: ```swift let minDistanceForMerge: Float = 0.25 var mergedSegments = result.segments for i in 0..