Meetily Voice Activity Detection (VAD) Pipeline Architecture

Meetily's real-time transcription system employs a tightly-coupled, three-layer VAD architecture that preprocesses audio through the ContinuousVadProcessor before reaching Whisper, reducing computational load by filtering silence while preserving full mixed audio recordings.

The open-source Meetily project (Zackriya-Solutions/meetily) implements a sophisticated Voice Activity Detection (VAD) pipeline to enable efficient real-time transcription. By integrating the silero-rs library within a Rust-based audio processing layer, Meetily isolates speech segments from mixed microphone and system audio before forwarding them to the Whisper transcription engine. This architecture minimizes unnecessary processing by ensuring only detected speech reaches the transcription worker.

Core VAD Engine Implementation

The VAD engine resides in frontend/src-tauri/src/audio/vad.rs and centers on the ContinuousVadProcessor struct, which wraps the Silero VAD library with Meetily-specific configuration and buffering logic.

Processor Initialization and Configuration

When the audio pipeline starts, ContinuousVadProcessor::new accepts the capture device's sample rate and a redemption time parameter (400ms on all platforms). The implementation configures the underlying Silero VAD to operate at a fixed 16kHz sample rate while setting min_speech_time to 250ms to prevent fragmentation of short speech bursts.

// From vad.rs [L32-L50] and [L46-L57]
let vad_processor = ContinuousVadProcessor::new(sample_rate, redemption_time)?;
config.redemption_time = Duration::from_millis(redemption_time_ms as u64);
config.min_speech_time = Duration::from_millis(250);

Resampling and Chunk Processing

The processor enforces a strict 16kHz requirement for the Silero model. If the input source differs, process_audio invokes resample_to_16k to convert the stream. Audio buffers accumulate until a complete 30ms window (480 samples at 16kHz) is available, at which point the VAD session analyzes the chunk for speech presence.

// Resampling logic from vad.rs [L88-L94]
if input_rate != 16000 {
    samples = resample_to_16k(&samples, input_rate);
}

Speech Segment Extraction and Flushing

Completed speech transitions are converted into SpeechSegment objects and queued for dispatch. During pipeline shutdown, the flush() method forces any open speech segment to close and emits the final pieces, ensuring no trailing audio is lost.

Audio Pipeline Integration

The AudioPipeline struct in frontend/src-tauri/src/audio/pipeline.rs orchestrates the flow between capture devices, the VAD processor, and downstream consumers.

Real-Time Mixing and Segmentation

Raw microphone and system audio streams enter an AudioMixerRingBuffer, where ProfessionalAudioMixer combines them into unified windows. The pipeline passes this mixed audio to vad_processor.process_audio(), which returns SpeechSegment objects. Only segments exceeding 800 samples (approximately 50ms at 16kHz) are converted to AudioChunk structures and dispatched to the transcription channel.

// From pipeline.rs [L35-L67]
let speech_segments = self.vad_processor.process_audio(&mixed)?;
for seg in speech_segments {
    if seg.samples.len() >= 800 {
        let chunk = AudioChunk {
            data: seg.samples,
            sample_rate: 16000,
            timestamp: seg.start_timestamp_ms / 1000.0,
            chunk_id: self.chunk_id_counter,
            device_type: DeviceType::Microphone,
        };
        self.transcription_sender.send(chunk)?;
        self.chunk_id_counter += 1;
    }
}

Graceful Shutdown Handling

During pipeline termination, flush_remaining_audio ([L100-L128] of pipeline.rs) invokes vad_processor.flush() to force-close any open speech segments and emit final audio pieces. This ensures no trailing speech is lost when the user stops recording.

Transcription Worker Consumption

Located in frontend/src-tauri/src/audio/transcription/worker.rs, the transcription worker receives pre-filtered AudioChunk objects via a channel. The worker deliberately bypasses additional VAD processing, as confirmed by an explicit code comment: "Skip VAD processing here since the pipeline already extracted speech using VAD" ([L420-L422]). This design eliminates redundant computation and reduces latency.

Configuration Parameters and Performance Impact

Meetily's VAD configuration balances responsiveness with accuracy:

• Redemption Time: 400ms bridges natural pauses between sentences, preventing over-segmentation
• Minimum Speech Duration: 250ms filters out transient noises and short vocalizations
• Chunk Size: 30ms windows (480 samples) provide granular detection without excessive overhead

This architecture reduces the volume of audio processed by Whisper by approximately 70%, transmitting only detected speech while the recording_manager.rs preserves the complete mixed audio stream for local WAV files.

Summary

• Meetily's VAD pipeline consists of three layers: the ContinuousVadProcessor engine, the AudioPipeline mixer/orchestrator, and the transcription worker consumer.
• The ContinuousVadProcessor in vad.rs wraps silero-rs, resamples audio to 16kHz, and buffers 30ms chunks while using a 400ms redemption time to bridge natural pauses.
• Mixed microphone and system audio flows through pipeline.rs, where the VAD extracts speech segments and filters out silence before forwarding chunks to transcription.
• The transcription worker in worker.rs receives pre-processed speech segments and skips redundant VAD checking, as indicated by source code comments at lines 420-422.
• A dedicated flush mechanism ensures all pending speech segments are emitted during pipeline shutdown, preventing data loss.

Frequently Asked Questions

What VAD library does Meetily use?

Meetily integrates the silero-rs Rust implementation of the Silero VAD model. The ContinuousVadProcessor struct in frontend/src-tauri/src/audio/vad.rs provides a wrapper that handles resampling, configuration, and segment buffering while exposing a simplified API to the rest of the application.

Why does Meetily resample audio to 16kHz for VAD processing?

The Silero VAD model requires a fixed 16kHz sample rate for accurate speech detection. Meetily's ContinuousVadProcessor automatically detects when the capture device uses a different rate (such as 48kHz) and invokes resample_to_16k to convert the audio before feeding it to the VAD session, ensuring consistent detection accuracy regardless of input hardware.

How does Meetily prevent losing speech during natural pauses?

The pipeline implements a redemption time of 400ms in the VAD configuration. This parameter keeps the speech segment open during brief silences between words or sentences, preventing the fragmentation of continuous utterances into separate chunks while still accurately detecting when speech has truly ended.

Where does the actual speech transcription happen after VAD processing?

After the AudioPipeline identifies and extracts speech segments in pipeline.rs, it sends AudioChunk objects to the transcription worker defined in frontend/src-tauri/src/audio/transcription/worker.rs. This worker forwards the audio directly to the Whisper model without additional VAD processing, as the pipeline has already isolated the relevant speech content.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}