Performance Optimization Techniques That Reduce Meetily Whisper Load by ~70% with VAD Filtering

Meetily reduces Whisper transcription load by approximately 70% using Voice Activity Detection (VAD) preprocessing that strips silent audio segments before they reach the transcription engine.

The open-source meeting assistant Meetily (Zackriya-Solutions/meetily) implements an intelligent audio pipeline that dramatically cuts computational overhead. By applying VAD filtering before Whisper processing, the system ensures the transcription engine receives only active speech segments. According to the repository's documentation in CLAUDE.md at lines 44-49, this optimization slashes the volume of audio data processed by the local Whisper engine while maintaining real-time transcription accuracy.

How VAD Filtering Reduces Whisper Processing Load

Traditional transcription pipelines feed continuous audio streams—including silence, background noise, and non-speech sections—directly into Whisper. This wastes CPU and GPU cycles processing audio that contains no translatable content.

Meetily's pipeline takes a different approach. It intercepts audio at frontend/src-tauri/src/audio/pipeline.rs and applies Silero VAD to identify speech boundaries. The system resamples raw audio to 16 kHz (the required rate for Silero VAD), then processes it in 30 ms chunks to distinguish speech from silence. Only segments containing actual voice activity proceed to transcription, effectively eliminating up to 70% of the audio volume that would otherwise burden the Whisper engine.

The Meetily Audio Pipeline Architecture

Resampling and Chunk Processing

The VAD processor requires specific input parameters to function correctly. The pipeline resamples incoming mixed microphone and system audio from its native rate to 16 kHz, then divides the stream into uniform chunks for analysis.

This resampling happens before any Whisper interaction, ensuring the VAD model receives data in its expected format. The chunk-based processing allows real-time classification without introducing significant latency.

Speech Segment Aggregation

To prevent fragmented transcription of natural speech patterns, Meetily configures the VAD with a redemption time of 2,000 ms (2 seconds). This parameter, set in VadConfig at lines 722-724 of pipeline.rs, merges short pauses between words or sentences into a single continuous speech segment.

The configuration also specifies a minimum speech duration of 250 ms, filtering out accidental noises and brief sounds that don't constitute meaningful speech. These thresholds ensure the system captures complete thoughts without forwarding unnecessary audio.

Implementation in pipeline.rs

The core VAD logic resides in frontend/src-tauri/src/audio/pipeline.rs. The following Rust code initializes the VAD processor with the redemption time that enables the 70% load reduction:

let vad = VadProcessor::new(VadConfig {
    redemption_time_ms: VAD_REDEMPTION_TIME_MS, // 2000 ms
    min_speech_ms: 250,
    // other params…
})?;
info!(
    "VAD processor created: input={}Hz, vad={}Hz, chunk_size={} samples",
    input_sample_rate, VAD_SAMPLE_RATE, vad_chunk_size
);

The pipeline runs asynchronously, continuously feeding chunks to the VAD processor while maintaining the audio receiver loop:

pub async fn run_vad_pipeline(&mut self) -> anyhow::Result<()> {
    self.vad_processor = self.create_vad_processor()?;
    while let Some(audio_chunk) = self.audio_receiver.recv().await {
        self.vad_processor.process_chunk(audio_chunk)?;
    }
    self.flush_remaining_segments().await?;
    Ok(())
}

Real-Time Segment Streaming to Whisper

Once the VAD identifies a speech segment, the pipeline immediately streams it to the Whisper engine without batching or delay. This direct handoff occurs at lines 766-770, where the system validates segment length before transcription.

The following implementation shows how the pipeline drops short segments and forwards only viable speech data, complete with logging that tracks the reduced sample counts:

if segment.samples.len() >= MIN_SAMPLES_FOR_WHISPER {
    // Directly hand the speech segment to Whisper
    whisper_engine.transcribe(segment.samples.clone()).await?;
    info!(
        "📤 Sending VAD segment: {:.1}ms, {} samples",
        segment.duration_ms(),
        segment.samples.len()
    );
} else {
    debug!("⏭️ Dropping short VAD segment: {:.1}ms (< 800 samples)", segment.duration_ms());
}

The logging statements at lines 841-858 explicitly track these metrics, demonstrating the volume difference between raw input and filtered output.

Performance Impact and Metrics

The VAD-based optimization delivers measurable improvements across three dimensions:

• Computational Load: Whisper processes approximately 70% less audio data, proportionally reducing CPU and GPU utilization.
• Latency: By skipping silence, the transcription engine focuses resources on active speech, decreasing time-to-text for spoken content.
• Quality Preservation: The 2-second redemption time and 250ms minimum speech thresholds maintain natural language boundaries without cutting off speaker transitions.

These gains are achieved without model quantization or approximation—the system simply eliminates redundant processing of non-speech audio.

Summary

• VAD preprocessing in frontend/src-tauri/src/audio/pipeline.rs filters silence before Whisper receives data.
• 16 kHz resampling and 30ms chunk processing prepare audio for Silero VAD analysis.
• A 2,000 ms redemption time merges short pauses to create complete, natural speech segments.
• Real-time streaming of filtered segments ensures Whisper processes only viable speech, cutting load by ~70%.
• The optimization is documented in CLAUDE.md lines 44-49 and implemented across the Rust audio pipeline.

Frequently Asked Questions

What is the redemption time in Meetily's VAD configuration?

The redemption time is set to 2,000 ms (2 seconds) in the VadConfig struct within frontend/src-tauri/src/audio/pipeline.rs at lines 722-724. This parameter merges pauses shorter than 2 seconds into adjacent speech segments, preventing the transcription engine from receiving fragmented audio that would split natural sentences.

Why does Meetily resample audio to 16kHz before VAD processing?

Silero VAD requires 16 kHz input to accurately classify voice activity. The pipeline resamples raw audio from its original capture rate to this target frequency before processing 30ms chunks, ensuring the VAD model receives data in its expected format for reliable speech detection.

How does Meetily prevent sending silence to the Whisper engine?

Before calling whisper_engine.transcribe(), the pipeline validates that segment.samples.len() meets the MIN_SAMPLES_FOR_WHISPER threshold. Segments failing this length check—indicating insufficient speech content—are logged and discarded, ensuring only meaningful audio reaches the transcription engine.

Where is the ~70% load reduction documented in the Meetily repository?

The performance gain is explicitly stated in CLAUDE.md at lines 44-49, which notes that VAD filtering reduces the amount of audio processed by the local Whisper engine by approximately 70 percent. The implementation supporting this metric is found in frontend/src-tauri/src/audio/pipeline.rs at lines 722-858.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}