How Meetily Handles Concurrent Microphone and System Audio Streams in Its Rust Pipeline

Meetily captures microphone and system audio concurrently, synchronizes them in a dual-ring buffer with 50 ms windows, mixes them using soft-limiting to prevent clipping, and routes the combined stream to both Whisper transcription and high-fidelity WAV recording.

The open-source meeting assistant Zackriya-Solutions/meetily solves the complex challenge of handling concurrent microphone and system audio streams through a sophisticated Rust-based pipeline. Built inside the Tauri framework’s Rust core (frontend/src-tauri/src/audio/), the system captures independent audio sources, aligns them temporally, and produces a professional-grade mix that feeds both the AI transcription engine and the local recording file.

Capturing Concurrent Audio Streams from Separate Devices

Meetily spawns independent capture threads for each audio source in frontend/src-tauri/src/audio/pipeline.rs. The AudioCapture struct reads raw samples from the microphone and system audio devices simultaneously.

Key preprocessing steps applied before buffering include:

• Resampling: Devices with mismatched sample rates are converted to the pipeline’s uniform 48 kHz target using a persistent rubato::SincFixedIn resampler, preserving RMS energy across chunks.
• Microphone conditioning: In frontend/src-tauri/src/audio/capture/microphone.rs, the stream passes through high-pass filtering, noise suppression, and loudness normalization to ensure clean voice capture.
• System audio capture: frontend/src-tauri/src/audio/capture/system.rs handles platform-specific implementations (ScreenCaptureKit on macOS, WASAPI on Windows) to grab computer audio without feedback loops.

Synchronizing Streams in the Ring Buffer

The AudioMixerRingBuffer struct manages synchronization between the two independent capture threads. It maintains two VecDeque<f32> queues—mic_buffer and system_buffer—that accumulate samples until a fixed 50 ms window (configurable via window_ms) is filled.

Buffer management follows strict constraints to prevent jitter and memory bloat:

• 400 ms cap: Each buffer holds a maximum of eight windows (approximately 400 ms) to guard against Bluetooth latency or macOS Core Audio delays. When either buffer exceeds this cap, the oldest samples are dropped and a warning is logged.
• Zero-padding: When extract_window() drains a window from each queue, if one stream contains fewer samples than the window size, the remaining space is zero-padded with silence. This prevents audible artifacts that would arise from sample repetition or truncation.

Professional Mixing with Soft Scaling

Once both queues contain sufficient data, ProfessionalAudioMixer::mix_window combines the microphone and system audio windows into a single buffer. The mixing algorithm implements intelligent gain staging to prevent distortion:

// System audio is attenuated to approximately 70% to leave headroom
let sys_scaled = sys * 0.7;
let sum = mic + sys_scaled;

// Soft scaling prevents hard clipping
let mixed_sample = if sum.abs() > 1.0 {
    sum / sum.abs()  // Proportional reduction
} else {
    sum
};

Soft scaling (dividing by the absolute value when exceeding ±1.0) is applied instead of hard clipping. This "radio-break" prevention technique proportionally reduces the amplitude of transient peaks, maintaining audio fidelity when both sources are loud simultaneously.

Routing Mixed Audio to Transcription and Recording

The mixed buffer feeds two independent downstream channels to ensure parity between what is transcribed and what is saved:

1. Transcription path: The mixed audio passes through ContinuousVadProcessor (defined in frontend/src-tauri/src/audio/vad.rs). Voice Activity Detection segments the stream at 16 kHz and emits only speech sections via self.transcription_sender.send(transcription_chunk), dramatically reducing Whisper’s processing load.
2. Recording path: The full-resolution 48 kHz mixed stream is sent via self.recording_sender_for_mixed.send(recording_chunk) to generate the final WAV file.

Both paths receive the same mixed audio signal, guaranteeing that the saved recording faithfully reflects the transcribed content.

Pipeline Lifecycle and Graceful Shutdown

The AudioPipeline::run method drives the continuous processing loop. It reads AudioChunk structs from a crossbeam channel, routes them to the appropriate ring buffer based on device_type, and triggers mixing whenever can_mix() returns true.

Lifecycle management ensures no audio is lost during termination:

• Drain-on-close: The loop continues processing until the input channel closes, flushing all buffered audio even after the user stops the recording.
• Forced flush: AudioPipelineManager::force_flush_and_stop injects a special "flush" chunk with an ID of u64::MAX into the pipeline. This forces immediate processing of any residual samples in the ring buffer before the application shuts down, preventing truncation of final words or system sounds.

// Simplified pipeline initialization
let mut manager = AudioPipelineManager::new();
manager.start(
    recording_state.clone(),
    transcription_sender,
    200,               // target chunk duration (VAD controls actual segmentation)
    48000,             // pipeline sample rate
    Some(recording_sender),
    mic_name,
    mic_kind,
    sys_name,
    sys_kind,
)?;

// Core mixing loop inside AudioPipeline::run
self.ring_buffer.add_samples(chunk.device_type.clone(), chunk.data);
while self.ring_buffer.can_mix() {
    let (mic_win, sys_win) = self.ring_buffer.extract_window().unwrap();
    let mixed = self.mixer.mix_window(&mic_win, &sys_win);
    
    // Send to VAD for transcription segmentation
    self.process_for_transcription(&mixed);
    
    // Send to WAV writer
    self.recording_sender_for_mixed.send(mixed).ok();
}

Summary

• Dual capture: Independent threads handle microphone and system audio with device-specific resampling to 48 kHz.
• Synchronization: The AudioMixerRingBuffer aligns streams using 50 ms windows with a 400 ms jitter cap and zero-padding for underflows.
• Clean mixing: ProfessionalAudioMixer applies approximately 70% attenuation to system audio and uses soft scaling to prevent digital clipping.
• Parallel routing: Identical mixed audio feeds both VAD-driven Whisper transcription (16 kHz) and full-quality WAV recording (48 kHz).
• Robust lifecycle: The pipeline drains completely on shutdown, with a forced flush mechanism ensuring no audio is truncated.

Frequently Asked Questions

How does Meetily prevent audio drift between the microphone and system capture?

The AudioMixerRingBuffer decouples the capture threads from the mixing thread. Each source accumulates samples independently, and the extract_window() function only proceeds when both queues contain a full 50 ms window. If one stream lags due to Bluetooth jitter or OS scheduling, the ring buffer accommodates up to 400 ms of differential delay before dropping old samples, ensuring temporal alignment without requiring complex timestamp interpolation.

What happens if one audio source is silent while the other is active?

When extract_window() encounters a queue with insufficient samples, it zero-pads the remainder of the window. This means silent sections are treated as digital zeroes (silence) rather than repeating the last known sample, preventing audible clicking or buzzing artifacts in the final mix. The additive mixing algorithm naturally preserves the active stream’s amplitude since adding zero does not alter the signal.

Why does Meetily reduce system audio volume to 70% before mixing?

Attenuating system audio prevents the combined signal from exceeding the ±1.0 floating-point range when both sources are at full volume. By leaving approximately 3 dB of headroom, the mixer can sum the signals additively without immediately triggering the soft-scaling limiter. This preserves dynamic range for the microphone—the primary transcription source—while still capturing clear computer audio.

Can the mixing window size be adjusted for lower latency?

Yes, the window_ms parameter in AudioMixerRingBuffer is configurable, though it defaults to 50 ms. Reducing this value decreases latency between capture and transcription but increases CPU overhead due to more frequent mixing operations. The 400 ms maximum buffer size (eight windows) remains constant regardless of window duration to maintain jitter tolerance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}