How Meetily Integrates Voice Activity Detection (VAD) into Its Real-Time Audio Pipeline

Meetily employs a three-layer VAD architecture that wraps the Silero model in a ContinuousVadProcessor, processes mixed audio through a real-time pipeline, and forwards only speech segments to Whisper, reducing transcription workload by approximately 70%.

Meetily is an open-source meeting transcription application by Zackriya-Solutions that relies on Voice Activity Detection to isolate speech before sending audio to OpenAI's Whisper. The integration spans three distinct Rust modules in the Tauri backend, creating a low-latency pipeline that bridges natural speech pauses while filtering silence. This design ensures the transcription worker receives only active speech at 16 kHz, preserving computational resources.

VAD Engine: ContinuousVadProcessor Implementation

The core Voice Activity Detection logic resides in frontend/src-tauri/src/audio/vad.rs, implemented as the ContinuousVadProcessor struct. This component wraps the silero-rs library and manages sample rate conversion, buffering, and segment extraction.

Configuration and Initialization

When the pipeline initializes the VAD processor at pipeline.rs lines 22-31, it passes the capture device sample rate and a redemption time (defaulting to 400 ms). Inside vad.rs lines 46-57, the processor configures the Silero model with a fixed 16 kHz processing rate and sets a minimum speech duration of 250 ms to eliminate false positives from brief noises.

// Configuration from vad.rs L46-L57
config.redemption_time = Duration::from_millis(redemption_time_ms as u64);
config.min_speech_time = Duration::from_millis(250);

Resampling and Chunk Processing

If the input audio does not match 16 kHz, the process_audio method resamples the stream before buffering. The implementation accumulates samples until a complete 30 ms window (480 samples) is available, then feeds it to the Silero session. Valid speech transitions are emitted as SpeechSegment structs.

// Usage pattern based on vad.rs L32-L33 and L85-L110
let mut vad_processor = ContinuousVadProcessor::new(input_sample_rate, 400)?;
let segments = vad_processor.process_audio(&mixed_audio)?;

Audio Pipeline: Real-Time Mixing and Segmentation

The AudioPipeline in frontend/src-tauri/src/audio/pipeline.rs orchestrates the flow between capture devices, the VAD engine, and transcription channels.

Initialization and VAD Binding

During construction at pipeline.rs lines 22-31, the pipeline instantiates the ContinuousVadProcessor and stores it for the processing loop:

// From pipeline.rs L22-L31
let vad_processor = ContinuousVadProcessor::new(sample_rate, redemption_time)?;

Processing Loop and Segment Filtering

Inside the run method (lines 35-67), the pipeline mixes microphone and system audio streams, then passes the result to the VAD processor. Returned segments undergo length filtering to ensure only substantial speech (≥ 800 samples, approximately 50 ms at 16 kHz) is forwarded.

// Core processing from pipeline.rs L35-L67
let speech_segments = self.vad_processor.process_audio(&mixed)?;
for seg in speech_segments {
    if seg.samples.len() >= 800 {
        let chunk = AudioChunk {
            data: seg.samples,
            sample_rate: 16000,
            timestamp: seg.start_timestamp_ms / 1000.0,
            chunk_id: self.chunk_id_counter,
            device_type: DeviceType::Microphone,
        };
        self.transcription_sender.send(chunk)?;
        self.chunk_id_counter += 1;
    }
}

Shutdown Flushing

To prevent speech loss during termination, the pipeline implements flush_remaining_audio at lines 100-128. This method calls vad_processor.flush(), forcing any open speech segment to close and emitting final chunks before the pipeline shuts down.

Transcription Worker: Bypassing Redundant VAD

The TranscriptionWorker in frontend/src-tauri/src/audio/transcription/worker.rs receives AudioChunk objects via channel and forwards them directly to Whisper. Because the pipeline already performed Voice Activity Detection, the worker explicitly skips redundant processing.

As documented in the source at lines 420-422:

// Skip VAD processing here since the pipeline already extracted speech using VAD

This architectural separation ensures Whisper receives only clean, pre-filtered speech segments, eliminating the computational overhead of processing silence while maintaining the full mixed audio for recording files.

Practical VAD Integration Example

The following example demonstrates direct usage of the VAD processor, including proper initialization, processing, and flushing:

use meetily::frontend::src_tauri::src::audio::vad::{ContinuousVadProcessor, SpeechSegment};

fn extract_speech_segments(
    audio_samples: &[f32], 
    input_rate: u32
) -> anyhow::Result<Vec<SpeechSegment>> {
    // Initialize with 400ms redemption time to bridge natural pauses
    let mut vad = ContinuousVadProcessor::new(input_rate, 400)?;
    
    // Process audio buffer
    let mut segments = vad.process_audio(audio_samples)?;
    
    // Flush remaining speech on stream end to capture trailing words
    segments.extend(vad.flush()?);
    
    Ok(segments)
}

Summary

• Voice Activity Detection is implemented via ContinuousVadProcessor in vad.rs, wrapping Silero with 16 kHz resampling and configurable redemption thresholds.
• The audio pipeline at pipeline.rs mixes multiple streams and filters VAD segments to a minimum of 800 samples before dispatching to transcription channels.
• A flush mechanism in flush_remaining_audio (lines 100-128) guarantees complete speech capture during pipeline shutdown.
• The transcription worker at worker.rs bypasses secondary VAD processing, as the pipeline already isolated speech, reducing Whisper computational load by approximately 70%.

Frequently Asked Questions

How does Meetily handle different input sample rates?

The ContinuousVadProcessor automatically resamples audio to 16 kHz if the capture device operates at a different rate, as implemented in frontend/src-tauri/src/audio/vad.rs (lines 88-94). This ensures the Silero VAD model always receives the expected sample rate regardless of hardware capabilities.

What is the redemption time and why is it set to 400ms?

The redemption time is a configurable buffer that keeps the VAD channel open after speech stops to bridge natural pauses in conversation. Meetily defaults to 400 ms to prevent over-segmentation of thoughts while maintaining responsiveness, configurable during ContinuousVadProcessor initialization in pipeline.rs (lines 22-31).

Why does the transcription worker skip VAD processing?

As explicitly commented in worker.rs (lines 420-422), the worker skips VAD because the upstream AudioPipeline already extracted speech segments using the ContinuousVadProcessor. This architectural separation prevents double-processing and ensures Whisper receives only clean, pre-filtered audio chunks.

How does Meetily ensure no speech is lost when stopping a recording?

The pipeline calls flush_remaining_audio during shutdown (defined in pipeline.rs lines 100-128), which triggers vad_processor.flush(). This forces the VAD engine to close any active speech segments and emit final chunks, ensuring complete transcription of the audio stream.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}