How Meetily Handles Concurrent Microphone and System Audio Capture with Professional Mixing

Meetily captures microphone and system audio simultaneously through separate AudioCapture instances, synchronizes the streams via a ring-buffer, and blends them using a ProfessionalAudioMixer with soft-clipping to prevent distortion before routing to Whisper transcription and final WAV recording.

Meetily is an open-source meeting transcription tool that records both user speech and system audio in real-time. The core audio pipeline is implemented in Rust within frontend/src-tauri/src/audio/pipeline.rs, where it processes dual audio streams through a sophisticated mixing architecture designed for broadcast-quality output. This article breaks down exactly how the system achieves professional-grade audio mixing without clipping, latency, or sample-rate conflicts.

The Three-Stage Audio Pipeline Architecture

The pipeline processes audio through three distinct stages: capture and preprocessing, ring-buffer synchronization, and professional mixing with downstream routing.

Device-Specific Capture and Preprocessing

Each audio stream originates from an AudioCapture instance that handles device-specific initialization. The microphone input undergoes a comprehensive enhancement chain: a high-pass filter removes low-frequency rumble, RNNoise suppresses background noise, and EBU-R128 loudness normalization targets -23 LUFS broadcast standard. System audio is captured as-is without enhancement to preserve music and notification fidelity.

Both streams are converted to mono and resampled to the pipeline's native 48 kHz when necessary. In AudioCapture::process_audio_data, processed chunks are packaged as AudioChunk structs and forwarded to the shared pipeline state via state.send_audio_chunk. This ensures that microphone and system audio enter the mixing stage at identical sample rates and channel configurations.

Ring-Buffer Synchronization

The AudioPipeline receives raw chunks from both devices through an unbounded Tokio channel. An AudioMixerRingBuffer stores microphone and system samples in two independent VecDeque buffers until each accumulates enough data for a fixed-size mixing window of approximately 50 ms.

The ring buffer implements several safeguards. It pads missing data with silence rather than repeating samples, preventing audible artifacts when one stream temporarily drops. A max_buffer_size cap of 400 ms (eight times the mixing window) absorbs bursty system-audio delivery, particularly on macOS Core Audio, preventing runaway memory growth. The buffer logic is defined in AudioMixerRingBuffer::add_samples, can_mix, and extract_window.

Professional Mixing and Downstream Routing

Once the ring buffer reports a full window via can_mix, the ProfessionalAudioMixer combines the microphone and system windows in mix_window. The mixer applies a 70% attenuation to system audio to prevent it from overpowering speech, then implements soft-clipping: if the sum of two samples exceeds ±1.0, the result is proportionally scaled back to stay within the unit range rather than hard-clipping to ±1.0.

After mixing, the pipeline applies no additional gain—the microphone signal already maintains broadcast-level loudness while system audio sits at the attenuated level. The mixed chunk follows two paths: it passes through the VAD processor to extract speech-only segments for Whisper transcription, and forwards to a recording sender that writes the final mixed WAV file containing both microphone and system audio. This orchestration occurs in the AudioPipeline::run loop.

Handling Sample Rate Mismatches and Latency

Meetily addresses hardware diversity through intelligent resampling. When AudioCapture detects a sample-rate mismatch against the 48 kHz pipeline standard, it creates a persistent Rubato resampler that buffers variable-size chunks until accumulating 512 samples for processing. This preserves RMS energy across chunks and prevents pitch-shifting artifacts.

Latency and jitter are managed through the ring buffer's 400 ms maximum capacity. This buffer size specifically accommodates macOS Core Audio's tendency toward bursty delivery patterns, ensuring that temporary delays in system audio capture do not cause the mixer to stall or drop microphone data.

Preventing Audio Artifacts and Distortion

The system implements several strategies to maintain audio fidelity. Soft-scaling in mix_window prevents the "radio-break" distortion characteristic of hard clipping by proportionally reducing samples only when the combined signal exceeds the digital maximum. When one stream is temporarily absent during extract_window, the buffer pads the missing window with zeros (silence) rather than sample repetition, eliminating click artifacts.

The architecture also prevents double-recording and echo by design: raw microphone and system streams are never written directly to disk. Only the professionally mixed output reaches the WAV writer, while the transcription engine receives only speech segments identified by the VAD processor.

Code Implementation Examples

The following Rust examples demonstrate the pipeline's core mechanisms.

Starting the pipeline from the Tauri command layer:

// In src/lib.rs – a Tauri command that begins recording
#[tauri::command]
async fn start_recording(
    app: tauri::AppHandle,
    mic_device_name: Option<String>,
    system_device_name: Option<String>,
    meeting_name: Option<String>,
) -> Result<(), String> {
    let state = RecordingState::new();                 // Shared state
    let (trans_tx, trans_rx) = mpsc::unbounded_channel(); // Whisper sender
    let recording_sender = Some(mpsc::unbounded_channel().0); // WAV writer

    // Build the manager and launch the pipeline
    let mut manager = AudioPipelineManager::new();
    manager.start(
        Arc::new(state),
        trans_tx,
        /*target_chunk_duration_ms=*/ 20,
        /*sample_rate=*/ 48_000,
        recording_sender,
        mic_device_name.unwrap_or_default(),
        detect_mic_kind(),
        system_device_name.unwrap_or_default(),
        detect_system_kind(),
    )?;
    // Store `manager` somewhere to stop later …
    Ok(())
}

The professional mixing implementation with soft-clipping:

fn mix_window(&mut self, mic_window: &[f32], sys_window: &[f32]) -> Vec<f32> {
    let max_len = mic_window.len().max(sys_window.len());
    let mut mixed = Vec::with_capacity(max_len);
    for i in 0..max_len {
        let mic = mic_window.get(i).copied().unwrap_or(0.0);
        let sys = sys_window.get(i).copied().unwrap_or(0.0);
        // System is slightly reduced; mic stays at full level
        let sum = mic + sys * 1.0;
        // Soft clipping – proportionally scale if > 1.0
        let mixed_sample = if sum.abs() > 1.0 { sum / sum.abs() } else { sum };
        mixed.push(mixed_sample);
    }
    mixed
}

Ring-buffer extraction and mixing invocation:

if let Some((mic_win, sys_win)) = self.ring_buffer.extract_window() {
    let mixed = self.mixer.mix_window(&mic_win, &sys_win);
    // forward `mixed` to VAD and recording …
}

Summary

• Dual capture architecture: Separate AudioCapture instances handle microphone and system audio, with microphone-specific enhancement (RNNoise, EBU-R128) and unified 48 kHz resampling.
• Synchronization mechanism: The AudioMixerRingBuffer uses dual VecDeque structures with a 50 ms mixing window and 400 ms maximum capacity to align asynchronous streams.
• Professional mixing: ProfessionalAudioMixer::mix_window applies 70% system attenuation and soft-clipping to prevent distortion while maintaining broadcast-level microphone loudness.
• Quality safeguards: Rubato resampling handles sample-rate mismatches, silence padding prevents artifacts, and soft-scaling avoids hard-clipping distortion.
• Dual routing: Mixed audio feeds both the VAD processor (for Whisper transcription) and the WAV recorder, while raw streams remain unrecorded to prevent echo.

Frequently Asked Questions

How does Meetily prevent audio clipping when mixing microphone and system audio?

Meetily implements soft-clipping in ProfessionalAudioMixer::mix_window. When the sum of microphone and system samples exceeds the ±1.0 digital limit, the mixer proportionally scales the entire sample back into range rather than truncating it. This preserves the waveform's shape and prevents the harsh distortion characteristic of hard clipping.

What happens if the microphone and system audio have different sample rates?

The AudioCapture struct detects sample-rate mismatches and instantiates a persistent Rubato resampler that buffers variable-size input chunks until accumulating 512 samples. This resampling preserves RMS energy across chunks, ensuring that the 48 kHz pipeline receives consistent audio without pitch artifacts or volume fluctuations.

Why does Meetily use a ring buffer instead of mixing audio immediately?

The AudioMixerRingBuffer solves synchronization and jitter problems. Because system audio (especially on macOS Core Audio) arrives in bursty patterns while microphone input is more consistent, the 50 ms mixing window ensures both streams have corresponding data before combination. The 400 ms maximum capacity absorbs temporary delivery delays without stalling the microphone stream.

Does Meetily save separate tracks for microphone and system audio?

No. According to the AudioPipeline::run implementation, only the mixed output is written to the final WAV file. Raw microphone and system streams are never recorded separately—this design prevents double-recording and echo while ensuring the transcription engine receives properly balanced audio through the VAD processor.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}