How Meetily's Voice Activity Detection (VAD) Filter Reduces Whisper Processing Load by 70%

The Meetily desktop application implements a custom VAD-driven audio pipeline that discards silence and background noise before audio reaches the Whisper transcription engine, eliminating approximately 70% of raw audio data and dramatically reducing computational overhead.

Meetily, an open-source meeting transcription tool, leverages a custom Voice Activity Detection (VAD) filter to optimize real-time speech recognition. By preprocessing audio streams to identify and remove non-speech segments before they reach OpenAI's Whisper model, the system achieves roughly a 70% reduction in processing load. This architecture minimizes GPU memory pressure and prevents the transcription engine from wasting cycles on silence and background noise.

Architectural Overview of the VAD Pipeline

The audio processing pipeline follows a six-stage flow that progressively filters non-speech content before transcription occurs.

Audio Capture and Resampling

The pipeline begins in frontend/src-tauri/src/audio/pipeline.rs, where raw microphone and system audio are captured at 48 kHz. Before VAD analysis, the mixed audio is down-sampled to 16 kHz, defined by the constant VAD_SAMPLE_RATE = 16000 in frontend/src-tauri/src/audio/vad.rs (line 34). This resampling reduces the sample volume by 3× before any speech detection occurs, aligning with the Silero VAD library's requirements.

Chunk-Based Silence Detection

After resampling, the audio is segmented into 30 ms chunks (480 samples), calculated as vad_chunk_size = (VAD_SAMPLE_RATE as f32 * 0.03) as usize (see vad.rs, line 64). For each chunk, the system computes RMS and peak energy. If both metrics fall below the VAD_SILENCE_THRESHOLD of approximately 0.003 (defined in frontend/src-tauri/src/lib_old_complex.rs, line 66), the chunk is immediately discarded with a log entry: log_debug!("🔇 VAD: Genuine silence detected …").

Speech Segment Accumulation

Consecutive speech chunks are merged using a configurable redemption time of 2000 ms (VAD_REDEMPTION_TIME_MS in frontend/src-tauri/src/audio/retranscription.rs, line 48). This bridges natural pauses between sentences to create contiguous speech segments. Only these consolidated segments are forwarded to Whisper, as implemented in pipeline.rs (lines 841-847), eliminating the need for time-based buffering of silent periods.

Implementation Details in the Meetily Codebase

The core VAD logic resides in frontend/src-tauri/src/audio/vad.rs within the VADProcessor struct. Key implementation details include:

• Initialization: VADProcessor::new() accepts the input sample rate (e.g., 48000) and redemption time, creating a resampler to 16 kHz (info!("VAD processor created: input={}Hz, vad=16000Hz …") at line 67).
• Processing: The process_chunk() method feeds 480-sample windows to the Silero VAD session, logging extraction metrics: debug!("VAD: Processed {} samples, extracted {} speech samples …") (line 321).
• Segment Finalization: When speech ends, the processor creates a SpeechSegment with precise timestamps: info!("VAD: Completed speech segment …") (line 266).

Short segments below the duration threshold are filtered out, as shown in logs: ⏭️ Dropping short VAD segment: 15.0ms (240 samples < 800), ensuring only substantial speech content reaches the transcription engine.

Performance Impact and Benefits

The VAD filter achieves the 70% reduction by removing typical meeting silence—including gaps between speakers, pause fillers, and background noise—from the Whisper input stream. This filtering delivers three primary benefits:

• Reduced Input Size: Only approximately 30% of original audio frames require transcription.
• Lower Resource Consumption: Fewer frames decrease GPU memory pressure and CPU inference cycles.
• Higher Accuracy: Eliminating silence prevents Whisper from hallucinating non-existent speech in quiet audio segments.

When speech is detected, segments are transmitted immediately to Whisper, enabling near-real-time transcription updates without waiting for silence-filled buffers to process.

Practical Implementation Example

The following Rust pattern mirrors the integration found in frontend/src-tauri/src/audio/pipeline.rs (lines 722-748):

// Initialize the VAD processor for a 48 kHz input stream
let vad = VADProcessor::new(
    48000,              // input_sample_rate
    2000,               // redemption_time_ms
    Some(callback),     // optional progress callback
)?;

// Process captured audio chunks (Vec<i16>)
for chunk in audio_buffer {
    vad.process_chunk(&chunk)?;
}

// Flush remaining speech segments after capture ends
let final_segments = vad.flush()?;

// Transmit only speech segments to Whisper
for segment in final_segments {
    whisper_engine.transcribe(segment.samples).await?;
}

This implementation ensures that whisper_engine.transcribe() receives only valid speech data, logging transmissions like: 📤 Sending VAD segment: 350.2ms, 5600 samples.

Summary

• VAD preprocessing in frontend/src-tauri/src/audio/vad.rs resamples audio to 16 kHz and analyzes 30 ms chunks to detect speech.
• Silence thresholds (RMS < 0.003) defined in lib_old_complex.rs filter out non-speech content before it reaches the transcription engine.
• Redemption time (2000 ms) merges nearby speech segments to create natural transcription boundaries.
• Pipeline integration at pipeline.rs lines 841-847 forwards only speech segments, achieving approximately 70% reduction in Whisper processing load.
• Performance gains include lower GPU memory usage, reduced latency, and fewer transcription hallucinations.

Frequently Asked Questions

How does the VAD filter achieve a 70% reduction in processing load?

The filter removes silence and background noise by analyzing 30 ms audio chunks and discarding those with RMS energy below 0.003. Since typical meetings contain large portions of silence between speakers, this eliminates roughly 70% of raw audio frames before they reach Whisper, according to the implementation in frontend/src-tauri/src/audio/vad.rs.

What is the redemption time in Meetily's VAD implementation?

The redemption time is a configurable buffer (default 2000 ms or VAD_REDEMPTION_TIME_MS in retranscription.rs, line 48) that bridges gaps between consecutive speech chunks. This prevents over-segmentation by merging natural pauses into single continuous speech segments, reducing the number of separate Whisper requests.

Why does the VAD processor resample audio to 16 kHz?

The Silero VAD library requires a 16 kHz sample rate for accurate voice detection. The VAD_SAMPLE_RATE = 16000 constant (defined in vad.rs, line 34) ensures compatibility while also reducing data volume by 66% when downsampling from 48 kHz capture sources, optimizing both detection accuracy and processing efficiency.

Which source files contain the core VAD logic?

The primary implementation resides in frontend/src-tauri/src/audio/vad.rs, containing the VADProcessor struct and chunk analysis logic. The pipeline orchestration occurs in frontend/src-tauri/src/audio/pipeline.rs (lines 841-847), while silence thresholds are referenced in frontend/src-tauri/src/lib_old_complex.rs (line 66) and redemption timing in frontend/src-tauri/src/audio/retranscription.rs (line 48).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}