Recording Path vs Transcription Path in Meetily's Audio Pipeline: Architecture and Implementation

The recording path writes the complete mixed audio stream to a WAV file for later playback, while the transcription path filters the same audio through Voice Activity Detection (VAD) and downsamples it to 16 kHz to feed Whisper for live speech-to-text conversion.

Meetily's audio processing engine handles microphone and system audio through a unified pipeline defined in frontend/src-tauri/src/audio/pipeline.rs. Understanding the distinction between the recording path and transcription path is essential for developers customizing the application's audio behavior or debugging pipeline issues.

Core Audio Pipeline Architecture

The pipeline begins with AudioCapture grabbing raw streams from both microphone and system audio devices. After optional resampling, noise suppression, and loudness normalization, samples accumulate in an AudioMixerRingBuffer. When a 50 ms window is ready, the ProfessionalAudioMixer::mix_window method blends the two synchronized buffers.

At this point, the pipeline diverges into two distinct paths that process the same mixed audio data differently.

The Recording Path: Full Mixed Audio Persistence

The recording path ensures users get a complete, faithful recording of their meeting. After mixing, the pipeline wraps the raw mixed buffer in an AudioChunk and sends it to recording_sender_for_mixed.

Key characteristics:

• Goal: Produce a single mixed WAV file containing both microphone and system audio
• Sample Rate: Preserves the original capture rate (typically 48 kHz)
• Data Flow: AudioPipeline::run → ring_buffer.extract_window → mixer.mix_window → recording_sender_for_mixed.send(recording_chunk)
• Destination: RecordingSaver writes chunks to disk as a .wav file

The recording sender is wired during pipeline initialization in AudioPipelineManager::start at lines 94-97 of pipeline.rs:

// From AudioPipelineManager::start
pipeline.recording_sender_for_mixed = recording_sender;

Inside the main loop at lines 68-78, the mixed audio flows directly to the recording path:

// Inside AudioPipeline::run – Recording path
if let Some((mic_win, sys_win)) = self.ring_buffer.extract_window() {
    let mixed = self.mixer.mix_window(&mic_win, &sys_win);
    
    // Recording path: Send full mixed audio to file
    if let Some(ref rec_sender) = self.recording_sender_for_mixed {
        let rec_chunk = AudioChunk {
            data: mixed.clone(),
            sample_rate: self.sample_rate,  // Original rate (e.g., 48000)
            timestamp: chunk.timestamp,
            chunk_id: self.chunk_id_counter,
            device_type: DeviceType::Microphone, // Mixed audio
        };
        let _ = rec_sender.send(rec_chunk);
    }
}

The Transcription Path: Optimized Speech Extraction

The transcription path optimizes for Whisper's requirements by extracting only speech segments and converting to the required 16 kHz sample rate. This path uses ContinuousVadProcessor to discard silence and background noise before sending data to the transcription engine.

Key characteristics:

• Goal: Feed only speech segments to Whisper for live transcription
• Sample Rate: Downsampled to 16 kHz (Whisper's optimal input rate)
• Processing: VAD analysis removes non-speech audio to reduce processing load
• Data Flow: AudioPipeline::run → mix_window → vad_processor.process_audio → transcription_sender.send(transcription_chunk)

The implementation at lines 85-108 of pipeline.rs handles the VAD processing:

// Inside AudioPipeline::run – Transcription path
if let Ok(segments) = self.vad_processor.process_audio(&mixed) {
    for seg in segments {
        let trans_chunk = AudioChunk {
            data: seg.samples,
            sample_rate: 16_000,  // Downsampled for Whisper
            timestamp: seg.start_timestamp_ms / 1_000.0,
            chunk_id: self.chunk_id_counter,
            device_type: DeviceType::Microphone,
        };
        let _ = self.transcription_sender.send(trans_chunk);
    }
}

The transcription receiver (typically in frontend/src-tauri/src/audio/transcription/engine.rs) emits these chunks via Tauri events to the frontend, where Whisper converts them to text.

Critical Differences Between the Two Paths

Aspect	Recording Path	Transcription Path
Audio Content	Full mixed audio (speech + silence + system sounds)	Speech-only segments after VAD filtering
Sample Rate	Original capture rate (48 kHz)	Downsampled to 16 kHz
Processing Stage	Direct from mixer	Post-VAD processing
Output Destination	RecordingSaver → .wav file	Whisper Engine → UI transcript
Sender Channel	recording_sender_for_mixed	transcription_sender

How the Paths Work Together

Both paths originate from the same mixed audio window, ensuring synchronization between what gets recorded and what gets transcribed. The AudioPipelineManager initializes both senders during startup:

let mut manager = AudioPipelineManager::new();
manager.start(
    recording_state.clone(),
    transcription_sender.clone(),
    /*target_chunk_duration_ms=*/ 0,
    /*sample_rate=*/ 48000,
    Some(recording_sender.clone()),          // Enables recording path
    mic_name.clone(),
    mic_kind,
    system_name.clone(),
    system_kind,
)?;

This design allows Meetily to simultaneously generate a complete audio archive while providing real-time transcription without duplicating the audio capture logic. The recording path preserves the natural conversation flow, while the transcription path optimizes for accuracy and latency by preprocessing audio for Whisper.

Summary

• Recording Path captures the complete mixed audio stream at original quality (48 kHz) and writes it to a WAV file via RecordingSaver.
• Transcription Path passes the same mixed audio through ContinuousVadProcessor to extract speech segments, downsample to 16 kHz, and stream them to Whisper for live transcription.
• Both paths diverge inside AudioPipeline::run in frontend/src-tauri/src/audio/pipeline.rs after the audio mixing stage.
• The separation allows optimal file storage (full audio) while minimizing transcription processing (speech-only, reduced sample rate).

Frequently Asked Questions

Why does the transcription path use 16 kHz while recording uses 48 kHz?

Whisper models are trained and optimized for 16 kHz audio input. Downsampling from the original capture rate (typically 48 kHz) reduces processing overhead and matches Whisper's expected input format. The recording path maintains 48 kHz to preserve audio fidelity for playback purposes, as documented in the AudioChunk configuration within pipeline.rs.

Can I disable the recording path while keeping transcription active?

Yes. The recording path is optional and controlled by the recording_sender_for_mixed parameter in AudioPipelineManager::start. Passing None disables the recording path while allowing the transcription path to function normally via the transcription_sender channel.

What happens to system audio in the transcription path?

Both microphone and system audio are mixed together by ProfessionalAudioMixer::mix_window before reaching either path. The transcription path processes this mixed audio through VAD, meaning speech from both sources (meeting participants and system audio containing voices) will be detected and sent to Whisper, while pure silence or noise is filtered out.

Where does the VAD processing occur in the codebase?

Voice Activity Detection happens in the transcription path within frontend/src-tauri/src/audio/pipeline.rs at lines 85-108. The ContinuousVadProcessor analyzes the mixed audio window and returns only speech segments, which are then packaged into AudioChunk structures and sent to the transcription engine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}