fluidaudio
Frontier CoreML audio models in your apps — text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source.
Understand streaming vs offline VAD processing in FluidAudio. Explore architecture and implementation for real-time or batch VAD with this technical guide.
How FluidAudio Handles Audio Format Conversion for InferenceFluidAudio ensures consistent model-ready audio data for inference by normalizing all inputs to 16 kHz mono Float32 using its AudioConverter class. Learn how it works.
How to Implement Speaker Identification with Known Speaker Embeddings in FluidAudioImplement speaker identification with known speaker embeddings using FluidAudio. Learn how the SpeakerManager class efficiently matches vectors with cosine distance and EMA updates.
Limitations of Sortformer for Speaker Diarization: Constraints and WorkaroundsExplore the limitations of Sortformer for speaker diarization including fixed speaker slots and cross-session identity issues. Learn about workarounds for large conferences.
How FluidAudio Handles Model Downloads and CachingDiscover how FluidAudio manages model downloads and caching with its centralized subsystem. Learn how it fetches, caches, and verifies Core ML models for efficient use.
How to Customize VAD Thresholds for Different Audio Environments in FluidAudioCustomize VAD thresholds in FluidAudio for any audio environment. Adjust defaultThreshold and VadSegmentationConfig for precise voice activity detection and segmentation.
FluidAudio Offline Diarization Pipeline Architecture: Core ML Segmentation and VBx Clustering in SwiftExplore the FluidAudio offline diarization pipeline architecture. Discover how Core ML models, Swift concurrency, and VBx clustering create on-device speaker segmentation for your audio projects.
How to Handle the Last Chunk in Streaming ASR Processing in FluidAudioLearn how to handle the last chunk in streaming ASR processing by setting the isLastChunk flag to true in FluidAudio. Trigger final decoding and emit all pending tokens.
Thread Safety Guarantees in FluidAudio Managers: Swift Actor Implementation ExplainedDiscover FluidAudio manager thread safety guarantees. Swift actors eliminate data races by serializing access to mutable state. Learn more about this compiler-enforced protection.
How Context Biasing Improves ASR Accuracy for Domain-Specific Terms in FluidAudioEnhance ASR accuracy for domain-specific terms with context biasing in FluidAudio. Learn how this technique boosts recognition by weighting rare words during CTC decoding.
How to Debug Diarization Issues with Speaker Embedding Validation in FluidAudioDebug diarization issues in FluidAudio by validating speaker embeddings. Enable debug logging and check embedding magnitudes for common errors. Resolve inconsistencies and improve speaker recognition accuracy.
FluidAudio Performance Metrics: ASR Timing and Diarization Quality ExplainedUnderstand FluidAudio performance metrics including ASR timing and diarization quality. Learn how FluidAudio's PerformanceMonitor and DiarizationMetricsCalculator measure real-time factors memory usage and error rates.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →