How Meetily's Voice Activity Detection (VAD) Filters Audio Before Transcription

Meetily uses a Silero VAD-based pipeline that resamples 48 kHz mixed audio to 16 kHz, processes it in 30 ms chunks with a 2000 ms redemption window, and forwards only speech segments to Whisper, reducing transcription load by approximately 70%.

Meetily, an open-source meeting transcription assistant from Zackriya-Solutions, implements a sophisticated Voice Activity Detection (VAD) pipeline to ensure that only speech reaches the Whisper transcription engine. By filtering silence and background noise at the capture stage, the system eliminates hallucination-prone transcriptions while significantly reducing computational overhead. The entire workflow is implemented in Rust within the Tauri backend, specifically in the frontend/src-tauri/src/audio/ directory.

The VAD Pipeline Architecture

The audio processing flow follows a strict sequence from capture to transcription, orchestrated by pipeline.rs and the dedicated VadProcessor in vad.rs.

Audio Capture and Mixing

The pipeline begins by capturing audio from both the microphone and system output at 48 kHz. These streams are mixed together to create a unified audio source that represents the full meeting context. According to the source code in frontend/src-tauri/src/audio/pipeline.rs, this mixed stream serves as the input for the VAD processor.

Resampling for Silero VAD

Before processing, the VadProcessor (defined in frontend/src-tauri/src/audio/vad.rs) internally resamples the 48 kHz mixed stream down to 16 kHz, the required sample rate for the Silero VAD model. This resampling happens transparently within the processor initialization:

let vad = VadProcessor::new(
    input_sample_rate,          // 48 kHz mixed audio
    VAD_REDEMPTION_TIME_MS,     // 2000 ms bridge
)?;

Chunking and Speech Classification

Once resampled, the audio is split into 30 ms chunks (480 samples at 16 kHz). The Silero VAD model evaluates each chunk individually, classifying it as either speech or silence. This granular approach allows the system to react quickly to changes in conversation dynamics while maintaining low latency.

Redemption Time Handling

To prevent fragmentation of natural speech patterns, Meetily implements a redemption time mechanism defined by the constant VAD_REDEMPTION_TIME_MS = 2000. This 2000 ms window bridges brief pauses between sentences, ensuring that a short breath or thoughtful pause does not split a continuous thought into separate transcription segments. The redemption logic operates within frontend/src-tauri/src/audio/vad.rs, maintaining continuity across momentary silence.

Speech Segmentation and Forwarding

When the VAD determines that speech is occurring, the pipeline extracts the relevant samples and prepares them for transcription.

RMS Threshold Protection

To avoid dropping quiet but legitimate speech (such as whispered comments or distant speakers), the system checks the RMS (Root Mean Square) energy of apparently silent chunks. If a chunk registers as silent by VAD standards but still contains audio energy above a low RMS threshold, the segment is retained. This safety check prevents the loss of low-volume speech that might otherwise be discarded.

Direct Whisper Integration

Speech segments are forwarded immediately to the Whisper engine without additional time-based accumulation. In frontend/src-tauri/src/audio/pipeline.rs (lines 831-842), the pipeline sends validated speech segments directly to the transcription channel:

let speech_segments = vad.process_samples(&mixed_chunk)?;
for seg in speech_segments {
    // Forward only speech to Whisper
    whisper_sender.send(seg)?;
    // Emit UI event for real-time feedback
    app.emit("speech-detected", seg.duration_ms)?;
}

This direct forwarding ensures minimal latency between speech detection and transcription start.

Recording Storage with VAD Filtering

The same VAD-filtered audio used for transcription is persisted to disk for later replay. In frontend/src-tauri/src/audio/recording_saver_old.rs (line 182), the system starts accumulation with "processed (VAD-filtered) audio," ensuring that saved meeting recordings contain only spoken content without silent gaps. This approach reduces storage requirements and makes playback more efficient.

Frontend Integration

The frontend receives real-time notifications about speech detection events through the Tauri event system. In frontend/src/services/recordingService.ts (line 160), the application listens for speech-detected events to provide visual feedback in the UI:

invoke('listen_for_event', { name: 'speech-detected' })
  .then((payload) => {
    console.log(`Speech segment: ${payload.durationMs} ms`);
  });

This integration allows users to see when the system is actively capturing speech, creating transparency about the recording state.

Summary

• Resampling pipeline: 48 kHz mixed audio is resampled to 16 kHz to meet Silero VAD requirements (frontend/src-tauri/src/audio/vad.rs).
• Chunk-based processing: Audio is analyzed in 30 ms segments (480 samples) for precise speech detection.
• Redemption window: A 2000 ms (VAD_REDEMPTION_TIME_MS) bridge prevents over-segmentation during natural pauses.
• Quality safeguards: RMS threshold checks retain low-volume speech that might otherwise be discarded as silence.
• Direct forwarding: Validated speech segments are sent immediately to Whisper (lines 831-842 in pipeline.rs), reducing processing load by ~70%.
• Filtered storage: Saved recordings contain only VAD-filtered speech, eliminating silent periods from disk.

Frequently Asked Questions

Why does Meetily resample audio to 16 kHz specifically for VAD processing?

The Silero VAD model requires a 16 kHz sample rate to operate correctly, as defined by the VAD_SAMPLE_RATE constant in frontend/src-tauri/src/audio/vad.rs. While the system captures audio at 48 kHz for quality mixing purposes, the VadProcessor internally handles the conversion to 16 kHz before processing chunks. This ensures compatibility with the neural network's training parameters while allowing the rest of the pipeline to maintain higher fidelity audio where needed.

What happens if someone pauses for longer than the 2000 ms redemption time?

If silence exceeds the 2000 ms redemption window (VAD_REDEMPTION_TIME_MS), the VAD processor finalizes the current speech segment and stops accumulating audio. When speech resumes, a new segment begins. This creates a natural boundary between distinct conversation turns while preventing excessive fragmentation during brief hesitations. The recording_saver_old.rs module stores these as separate segments in the final recording file.

Does VAD filtering affect the quality of saved meeting recordings?

The saved recordings contain only VAD-filtered audio, meaning silent periods are removed entirely from the stored files. However, the actual speech content is preserved at full quality. The system applies the RMS threshold check to ensure quiet speech is not discarded, so whispered or distant speech remains in the recording. According to recording_manager.rs (line 108), the pipeline ensures that "processed (VAD-filtered) audio" is what gets stored, optimizing storage space without sacrificing intelligibility.

How does the RMS threshold prevent the VAD from cutting off quiet speech?

When the Silero VAD marks a chunk as silent, Meetily performs a secondary RMS energy check before discarding it. If the chunk contains audio energy above a low threshold—indicating quiet but present speech—the system treats it as speech regardless of the VAD classification. This prevents the system from dropping the beginning or end of words spoken softly, ensuring that whispered comments or distant speakers are still captured and forwarded to the Whisper transcription engine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}