How Voice Activity Detection (VAD) Works in Meetily's Real-Time Transcription Pipeline

In Meetily, Voice Activity Detection (VAD) acts as a speech gatekeeper that filters mixed 48 kHz audio into discrete voice segments using 16 kHz Silero-based detection with a configurable 2000 ms redemption time, ensuring only validated speech reaches the Whisper transcription engine.

Meetily's transcription system captures parallel audio streams from both microphone input and system audio, merging them into a single mixed feed before analysis. This dual-stream architecture requires precise filtering to isolate actual speech from background noise and silence. Voice Activity Detection (VAD) serves as the critical preprocessing stage in the audio pipeline that determines what content advances to the Whisper engine, directly impacting transcription accuracy and computational efficiency.

Dual-Stream Audio Mixing and VAD Input

Meetily captures audio from two simultaneous sources: user speech via microphone and shared system audio from meeting participants. These streams are combined into a continuous 48 kHz mixed audio flow. Before transcription occurs, this mixed stream enters the VAD processor located in frontend/src-tauri/src/audio/vad.rs (lines 33-65), which resamples the audio to 16 kHz for analysis. The VAD operates on discrete 30 ms chunks containing 480 samples each, creating a granular detection window that identifies speech boundaries with millisecond precision.

The Three-Stage VAD Pipeline

The VAD implementation in frontend/src-tauri/src/audio/pipeline.rs (lines 678-735) performs three core functions that optimize the transcription workflow.

Speech-Only Segmentation

The VAD processor converts the continuous mixed audio stream into discrete speech segments, stripping silence and non-speech audio. This segmentation prevents the Whisper model from processing empty audio that could trigger hallucinations or inaccurate transcriptions. By filtering at the input stage, the system reduces the data volume sent to the GPU/CPU transcription worker.

Dynamic Chunking with Redemption Time

Unlike traditional time-based chunking that processes fixed 2-second blocks, Meetily's VAD-driven approach uses speech detection to dictate chunk boundaries. The target_chunk_duration_ms setting is bypassed in favor of VAD redemption time, which defaults to 2000 ms. This redemption window bridges natural pauses in speech, allowing the system to concatenate related utterances into longer, more coherent segments rather than fragmenting sentences at arbitrary time intervals. The redemption logic appears in frontend/src-tauri/src/audio/pipeline.rs (lines 722-731).

Direct Whisper Pre-filtering

Once the VAD confirms a speech segment, the pipeline dispatches it directly to the Whisper engine without additional silence pruning. The system logs segment metadata—duration and sample count—before transmission, as implemented in frontend/src-tauri/src/audio/pipeline.rs (lines 841-858). This ensures the transcription worker receives only validated audio content, improving both accuracy and computational efficiency.

Implementation Details and Code Examples

The VAD processor initialization requires specific sample rate configurations to handle the mixed input stream. The implementation uses a Silero-based detection model that requires 16 kHz input, necessitating resampling from the native 48 kHz capture rate.

Initializing the VAD processor:

let vad = VadProcessor::new(
    input_sample_rate,            // 48 kHz mixed audio
    VAD_SAMPLE_RATE,              // 16 kHz required by Silero VAD
    VAD_REDEMPTION_TIME_MS,       // 2000 ms bridges natural pauses
)?;

Running the VAD-driven pipeline:

info!("VAD-driven audio pipeline started");
pipeline.run_vad().await?;
info!("VAD-driven audio pipeline ended");

Sending validated segments to the transcription engine:

info!(
    "📤 Sending VAD segment: {:.1}ms, {} samples",
    segment.duration_ms(),
    segment.sample_count()
);
whisper_engine.transcribe(segment).await?;

Performance and Accuracy Benefits

VAD integration delivers measurable improvements to Meetily's transcription pipeline. By filtering silence and background noise at the preprocessing stage, the system reduces GPU/CPU load significantly, allowing the Whisper engine to focus computational resources solely on speech content. This pre-filtering eliminates transcription artifacts that commonly occur when speech recognition models process non-speech audio, resulting in cleaner output and reduced hallucination rates.

The VAD system also appears in frontend/src-tauri/src/audio/retranscription.rs, where it provides identical speech detection capabilities for processing recorded meeting files, ensuring consistent behavior between real-time and replay transcription modes.

Summary

• Meetily combines microphone and system audio into a 48 kHz mixed stream before VAD processing
• The VAD resamples audio to 16 kHz and processes it in 30 ms (480-sample) chunks using Silero detection
• Redemption time of 2000 ms bridges natural pauses to create coherent speech segments, replacing fixed-duration chunking
• Filtered segments pass directly to the Whisper engine without additional pruning, reducing computational overhead and preventing model hallucinations
• Key implementation files include frontend/src-tauri/src/audio/vad.rs and frontend/src-tauri/src/audio/pipeline.rs

Frequently Asked Questions

What sample rates does Meetily's VAD use for audio processing?

Meetily captures mixed audio at 48 kHz but resamples it to 16 kHz for VAD analysis, as required by the Silero detection model. The VAD processes this resampled audio in 30 ms segments containing 480 samples each, as defined in frontend/src-tauri/src/audio/vad.rs (lines 33-65).

How does VAD redemption time affect transcription segments?

The redemption time setting, defaulting to 2000 ms, bridges gaps between speech utterances to prevent fragmenting natural pauses. Unlike fixed time-based chunking, this dynamic approach concatenates related speech into longer, more contextually complete segments before sending them to Whisper, improving transcription coherence.

Why does Meetily use VAD before sending audio to Whisper?

VAD acts as a computational gatekeeper that removes silence and non-speech audio before transcription. This pre-filtering prevents the Whisper model from hallucinating during quiet periods and reduces GPU/CPU load by ensuring the engine processes only validated speech segments, as implemented in frontend/src-tauri/src/audio/pipeline.rs (lines 841-858).

Can VAD settings be adjusted for different meeting environments?

Yes, the VAD_REDEMPTION_TIME_MS parameter is configurable during VadProcessor initialization, allowing adjustment of how aggressively the system bridges speech pauses. The input sample rate and target VAD sample rate are also parameterized, though the Silero model requires 16 kHz input for accurate detection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}