How Meetily's Voice Activity Detection (VAD) Implementation Filters Non-Speech Audio

Meetily's VAD implementation uses a three-layer pipeline architecture that wraps the silero-rs library to isolate speech segments before Whisper transcription, reducing audio processing volume by approximately 70% while preserving natural conversation flow through configurable redemption time buffering.

Meetily, the open-source real-time meeting transcription tool from Zackriya-Solutions/meetily, processes audio through a sophisticated Voice Activity Detection system that ensures only speech reaches the Whisper engine. The implementation centers on the ContinuousVadProcessor struct in frontend/src-tauri/src/audio/vad.rs, which resamples incoming audio to 16 kHz and buffers it into 30 ms chunks for analysis. By filtering non-speech audio at the pipeline stage rather than during transcription, the system significantly reduces computational overhead and latency.

How the VAD Engine Processes Audio in Real-Time

The core Voice Activity Detection logic resides in frontend/src-tauri/src/audio/vad.rs, where the ContinuousVadProcessor wraps the Silero VAD library to analyze audio streams. This engine handles resampling, chunking, and segment boundary detection to extract clean speech from mixed audio inputs.

ContinuousVadProcessor Initialization and Configuration

When the audio pipeline starts, it instantiates a ContinuousVadProcessor with the capture device's sample rate and a redemption time parameter set to 400 ms across all platforms. According to the source code at lines 32-50 of vad.rs, the processor configures the underlying Silero VAD with specific thresholds to prevent over-segmentation:

config.redemption_time = Duration::from_millis(redemption_time_ms as u64);
config.min_speech_time = Duration::from_millis(250);

The redemption time setting bridges natural pauses in speech, preventing the system from fragmenting sentences into separate segments during brief hesitations. The minimum speech time of 250 ms filters out short noise bursts that could be mistaken for speech.

Audio Resampling and Chunk Buffering

If the capture device does not operate at 16 kHz, the process_audio method resamples incoming samples to meet the Silero VAD requirements (source: lines 88-94 of vad.rs). The processor buffers audio until it accumulates a full 30 ms window—exactly 480 samples at 16 kHz—before feeding it to the Silero session (source: lines 99-108 of vad.rs).

This chunked approach ensures the VAD receives consistently sized inputs while maintaining real-time processing capabilities. Completed speech transitions are converted into SpeechSegment structs and queued for dispatch to the transcription pipeline.

Flushing Remaining Audio on Shutdown

When the pipeline terminates, the system calls flush_remaining_audio (lines 100-128 of pipeline.rs), which triggers vad_processor.flush() to force-close any open speech segments. This ensures that trailing speech captured just before shutdown is not lost but instead emitted as final SpeechSegment instances.

Audio Pipeline Integration and Speech Segmentation

The AudioPipeline struct in frontend/src-tauri/src/audio/pipeline.rs orchestrates the flow between raw audio capture and VAD-filtered output. It mixes microphone and system audio streams before passing them to the VAD processor for speech extraction.

Mixing and VAD Processing Loop

During pipeline initialization (lines 22-31 of pipeline.rs), the system creates the VAD processor:

let vad_processor = ContinuousVadProcessor::new(sample_rate, redemption_time)?;

Inside the main processing loop (AudioPipeline::run, lines 35-67), the pipeline receives raw audio chunks, mixes them using ProfessionalAudioMixer, and submits the result to the VAD engine:

let speech_segments = self.vad_processor.process_audio(&mixed_with_gain)?;

Segment Filtering and Dispatch

For each returned SpeechSegment, the pipeline applies a minimum length filter of 800 samples (approximately 50 ms at 16 kHz) to eliminate transient noise artifacts (source: lines 38-56 of pipeline.rs). Valid segments are repackaged as AudioChunk instances with a fixed 16 kHz sample rate and dispatched to the transcription channel:

if seg.samples.len() >= 800 {
    let transcription_chunk = AudioChunk {
        data: seg.samples,
        sample_rate: 16000,
        timestamp: seg.start_timestamp_ms / 1000.0,
        chunk_id: self.chunk_id_counter,
        device_type: DeviceType::Microphone,
    };
    self.transcription_sender.send(transcription_chunk)?;
}

While the pipeline forwards speech-only segments to transcription, it simultaneously sends the full mixed audio to the recording manager, ensuring that saved WAV files contain complete meeting audio while transcripts contain only speech.

Transcription Worker Consumption

The transcription worker in frontend/src-tauri/src/audio/transcription/worker.rs receives pre-filtered AudioChunk instances directly from the pipeline. As noted in the source code comment at lines 420-422, the worker deliberately avoids duplicate VAD processing:

// Skip VAD processing here since the pipeline already extracted speech using VAD

This design choice streamlines the transcription path, allowing the worker to focus solely on forwarding audio to the Whisper engine without redundant analysis.

Configuration Parameters and Performance Impact

Meetily's VAD implementation balances latency and accuracy through carefully tuned parameters. The 400 ms redemption time prevents over-segmentation during natural speech pauses, while the 250 ms minimum speech duration filters non-speech audio events. By extracting speech segments before transcription, the system reduces the volume of audio processed by Whisper by approximately 70%, significantly lowering computational requirements and API costs.

Code Examples

Using ContinuousVadProcessor Directly

The following example demonstrates how to instantiate and use the VAD processor for standalone speech detection:

use meetily::frontend::src_tauri::src::audio::vad::{ContinuousVadProcessor, SpeechSegment};

fn demo_vad(samples: &[f32], input_rate: u32) -> anyhow::Result<()> {
    // Create a VAD with a 2-second redemption window
    let mut vad = ContinuousVadProcessor::new(input_rate, 2000)?;
    // Process the whole audio buffer
    let mut segments: Vec<SpeechSegment> = vad.process_audio(samples)?;
    // Flush any remaining speech
    segments.extend(vad.flush()?);
    // Print segment timestamps
    for seg in segments {
        println!(
            "Speech: {} ms → {} ms, {} samples",
            seg.start_timestamp_ms,
            seg.end_timestamp_ms,
            seg.samples.len()
        );
    }
    Ok(())
}

See frontend/src-tauri/src/audio/vad.rs for the ContinuousVadProcessor API—creation at lines 32-33 and process_audio at lines 85-110.

Implementing Pipeline VAD Integration

This excerpt from AudioPipeline::run shows how mixed audio flows through the VAD processor:

// Inside AudioPipeline::run (excerpt)
while let Some(chunk) = self.receiver.recv().await {
    // Mix mic + system audio
    self.ring_buffer.add_samples(chunk.device_type.clone(), chunk.data);
    while self.ring_buffer.can_mix() {
        let (mic_win, sys_win) = self.ring_buffer.extract_window().unwrap();
        let mixed = self.mixer.mix_window(&mic_win, &sys_win);

        // VAD segmentation
        let speech_segments = self.vad_processor.process_audio(&mixed)?;
        for seg in speech_segments {
            if seg.samples.len() >= 800 {
                let transcription_chunk = AudioChunk {
                    data: seg.samples,
                    sample_rate: 16000,
                    timestamp: seg.start_timestamp_ms / 1000.0,
                    chunk_id: self.chunk_id_counter,
                    device_type: DeviceType::Microphone,
                };
                self.transcription_sender.send(transcription_chunk)?;
                self.chunk_id_counter += 1;
            }
        }
    }
}

Core VAD usage at lines 35-67 of pipeline.rs; segment filtering at lines 38-56.

Flushing VAD State During Shutdown

To ensure no speech is lost during application termination, force-flush the VAD processor:

// Force-flush during pipeline termination
self.flush_remaining_audio()?; // Calls vad_processor.flush()

Implementation in flush_remaining_audio—lines 100-128 of pipeline.rs.

Summary

• Three-layer architecture: Meetily implements VAD at the engine level (vad.rs), pipeline level (pipeline.rs), and consumes filtered results in the transcription worker (worker.rs).
• Silero-rs integration: The ContinuousVadProcessor wraps the Silero VAD library, resampling all audio to 16 kHz and processing it in 30 ms chunks.
• Smart filtering: A 400 ms redemption time bridges natural speech pauses, while a 250 ms minimum speech duration and 800-sample segment threshold eliminate noise.
• Performance optimization: Pre-filtering audio reduces Whisper processing volume by approximately 70%, preserving computational resources for actual speech transcription.
• Dual output: The pipeline maintains separate audio paths—speech-only for transcription and full mixed audio for recording archival.

Frequently Asked Questions

How does Meetily's VAD handle different microphone sample rates?

Meetily's ContinuousVadProcessor automatically resamples incoming audio to 16 kHz using the resample_to_16k function before feeding it to the Silero VAD engine (source: lines 88-94 of frontend/src-tauri/src/audio/vad.rs). This ensures consistent VAD performance regardless of whether the capture device operates at 44.1 kHz, 48 kHz, or other standard rates.

What is the redemption time in Meetily's VAD implementation?

The redemption time is a 400 ms buffering period that bridges natural pauses in human speech, preventing the system from splitting continuous sentences into separate transcription segments when speakers hesitate briefly. This parameter is configured during ContinuousVadProcessor initialization and applied through the Silero VAD configuration (source: lines 46-57 of vad.rs).

Why does Meetily filter audio segments by length before transcription?

The pipeline discards speech segments shorter than 800 samples (approximately 50 ms at 16 kHz) to eliminate transient noise artifacts and false positives from the VAD engine. This filtering occurs in AudioPipeline::run (source: lines 38-56 of pipeline.rs) and ensures that only substantial speech content reaches the Whisper transcription worker.

Does the transcription worker perform additional VAD processing?

No, the transcription worker deliberately skips VAD processing because the pipeline has already extracted speech segments. As documented in the source code comment at lines 420-422 of frontend/src-tauri/src/audio/transcription/worker.rs, the worker assumes all received AudioChunk instances contain pre-filtered speech and forwards them directly to Whisper without further analysis.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}