How Meetily's Audio Pipeline Separates Recording and Transcription Paths with VAD Filtering

Meetily captures microphone and system audio in parallel, mixes them into a single stream, then forks that stream into two independent paths: one for full recording (saved to disk as WAV) and one for transcription (filtered through Voice Activity Detection to extract only speech segments).

Meetily is an open-source meeting assistant built with a Rust-based audio backend. Its architecture cleanly isolates recording (archival quality) from transcription (speech-only) by processing a single mixed audio stream through dual dispatch channels. This design ensures Whisper receives minimal non-speech audio while maintaining complete recordings for playback.

Parallel Audio Capture and Buffering

Dual-Device Input Handling

The pipeline originates in frontend/src-tauri/src/audio/pipeline.rs, where the AudioCapture struct manages independent capture threads for microphone and system audio. Raw chunks from each device are converted to mono, optionally resampled, and forwarded via process_audio_data (lines 624-645). The capture stage never writes to disk; it only feeds raw data to the central pipeline.

Ring Buffer Accumulation

Incoming audio is staged in an AudioMixerRingBuffer, which maintains separate internal buffers (mic_buffer and system_buffer). The add_samples method (lines 49-65) appends chunks until sufficient data exists for a fixed 50ms processing window. The pipeline checks readiness via can_mix (lines 87-90) before extracting synchronized windows with extract_window (lines 92-140).

The Professional Mixing Stage

Once both buffers contain aligned samples, AudioPipeline::run invokes ProfessionalAudioMixer::mix_window (lines 154-185) inside pipeline.rs. This mixer applies soft-scaling to combine streams without aggressive ducking, producing a single normalized audio stream that serves as the source for both downstream paths.

// From AudioPipeline::run (pipeline.rs, lines 821-827)
self.ring_buffer.add_samples(chunk.device_type.clone(), chunk.data);

while self.ring_buffer.can_mix() {
    let (mic_window, sys_window) = self.ring_buffer.extract_window().unwrap();
    let mixed_clean = self.mixer.mix_window(&mic_window, &sys_window);
    // mixed_clean is now the single source for both paths
}

Dual-Path Dispatch Architecture

After mixing, the pipeline splits the audio at line 822 of pipeline.rs, creating two distinct data flows from the identical source buffer.

Transcription Path with VAD Filtering

The mixed audio flows into ContinuousVadProcessor::process_audio (defined in frontend/src-tauri/src/audio/vad.rs, lines 16-45). This real-time Voice Activity Detection segments speech at 16kHz, discarding silence and non-speech noise. Only VAD-positive chunks are wrapped in AudioChunk structs and dispatched via transcription_sender (lines 834-862), drastically reducing Whisper's computational load.

// VAD processing and transcription dispatch (pipeline.rs, lines 834-862)
match self.vad_processor.process_audio(&mixed_with_gain) {
    VadResult::Speech(segment) => {
        let transcription_chunk = AudioChunk {
            data: segment.to_vec(),
            sample_rate: 16000,
            device_type: DeviceType::Mixed,
            timestamp: std::time::Instant::now(),
        };
        let _ = self.transcription_sender.send(transcription_chunk);
    }
    VadResult::Silence => { /* Discarded from transcription path */ }
}

Recording Path (Full Mixed Audio)

Simultaneously, the unfiltered mixed audio is cloned and sent to the file writer via recording_sender_for_mixed (lines 868-877). This path bypasses VAD entirely, ensuring the saved WAV contains microphone and system audio exactly as heard in the meeting, regardless of speech detection accuracy.

// Recording dispatch (pipeline.rs, lines 868-877)
if let Some(ref sender) = self.recording_sender_for_mixed {
    let recording_chunk = AudioChunk {
        data: mixed_with_gain.clone(), // Full audio, not VAD-filtered
        sample_rate: 48000,            // Original mixed rate
        device_type: DeviceType::Mixed,
        timestamp: std::time::Instant::now(),
    };
    let _ = sender.send(recording_chunk);
}

Frontend Integration

The separation is exposed to the TypeScript frontend through distinct event channels. The recording-manager.rs orchestrator (invoked via lib.rs command handlers) connects the pipeline to the UI.

// Starting capture (frontend invocation)
await invoke('start_recording', {
  mic_device_name: "Built‑in Microphone",
  system_device_name: "BlackHole 2ch",
  meeting_name: "Team Sync"
});

// Listening for VAD-filtered transcription
listen<TranscriptUpdate>('transcript-update', event => {
  // Contains only speech segments detected by VAD
  console.log('Transcribed:', event.payload.text);
});

// Listening for recording progress (full audio)
listen<RecordingUpdate>('recording-update', event => {
  // Contains complete mixed audio, no VAD applied
  console.log('Chunk saved, duration:', event.payload.duration);
});

Summary

• Single Source of Truth: The ProfessionalAudioMixer creates one mixed stream that feeds both paths, eliminating synchronization issues between mic and system recordings.
• Zero-Loss Recording: The recording_sender_for_mixed channel receives raw mixed audio without VAD processing, guaranteeing archival quality.
• Efficient Transcription: The ContinuousVadProcessor filters the same mixed stream, forwarding only speech segments (16kHz) to Whisper via transcription_sender.
• Buffer Management: AudioMixerRingBuffer handles alignment and windowing (50ms chunks) before mixing occurs.
• Clean Architecture: AudioCapture, AudioPipeline, and RecordingManager operate in distinct layers, with pipeline.rs serving as the sole integration point for VAD and dispatch logic.

Frequently Asked Questions

Why does Meetily mix audio before splitting it into paths?

Mixing microphone and system audio into a single stream in pipeline.rs ensures perfect synchronization between speakers. The ProfessionalAudioMixer handles soft-scaling (lines 154-185) to prevent audio ducking, creating a natural-sounding combined track. This single mixed source eliminates drift between the two capture devices before the VAD filter is applied.

How does VAD filtering affect transcription accuracy?

The ContinuousVadProcessor in vad.rs (lines 16-45) operates at 16kHz and removes silent periods and background noise. By forwarding only VadResult::Speech segments to the transcription channel (lines 834-862), Whisper receives higher signal-to-noise audio, improving word-error rates while reducing API costs or local compute time.

What happens if the VAD incorrectly filters speech?

Since the recording path receives the full mixed audio via recording_sender_for_mixed (lines 868-877) regardless of VAD decisions, no speech is permanently lost. If VAD misses a quiet utterance, it remains in the saved WAV file even if it never reaches the transcription worker, allowing for manual review or re-processing if needed.

Can I disable VAD filtering for specific meetings?

According to the source code in pipeline.rs, the VAD processor is instantiated within AudioPipeline::run and always processes the mixed stream before transcription dispatch. There is no runtime flag in the current implementation to bypass ContinuousVadProcessor::process_audio; disabling it would require modifying the channel logic at lines 834-862 to send all mixed_with_gain data instead of only VAD-positive segments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}