How Meetily Synchronizes Microphone and System Audio Using a Ring Buffer

Meetily aligns asynchronous microphone and system audio streams by buffering them in separate ring buffers and extracting fixed-size windows, zero-padding the shorter stream to maintain synchronization for mixing and transcription.

The open-source meeting assistant Meetily captures both microphone input and system output simultaneously, but these streams arrive at different rates with varying latency. To produce a single, synchronized audio track for Whisper transcription and recording, the Rust-based audio pipeline implements a specialized ring buffer synchronizer in frontend/src-tauri/src/audio/pipeline.rs.

Ring Buffer Architecture

The synchronization logic centers on AudioMixerRingBuffer, a struct that maintains independent FIFO queues for each audio source.

struct AudioMixerRingBuffer {
    mic_buffer: VecDeque<f32>,
    system_buffer: VecDeque<f32>,
    window_size_samples: usize,   // e.g. 50 ms worth of samples
    max_buffer_size: usize,       // safety limit (≈ 400 ms)
}

mic_buffer and system_buffer store raw PCM samples as VecDeque<f32> to enable efficient push and pop operations from opposite ends. The window_size_samples field defines the mixing granularity—typically 50 milliseconds of audio calculated from the device sample rate. To prevent memory exhaustion during jitter spikes, max_buffer_size caps storage at roughly 400 milliseconds, a limit specifically tuned to accommodate macOS Core Audio latency variations.

Feeding Samples into the Buffer

When audio chunks arrive from the capture threads, the pipeline calls add_samples to enqueue data into the appropriate buffer.

match device_type {
    DeviceType::Microphone => self.mic_buffer.extend(samples),
    DeviceType::System => self.system_buffer.extend(samples),
}

This method accepts a DeviceType enum variant to distinguish between sources. It automatically trims overflow beyond max_buffer_size and emits diagnostic warnings when buffers approach capacity, allowing developers to monitor stream health in real time.

Detecting Ready Windows

Before mixing, the pipeline verifies that sufficient data exists using can_mix.

fn can_mix(&self) -> bool {
    self.mic_buffer.len() >= self.window_size_samples ||
    self.system_buffer.len() >= self.window_size_samples
}

The function returns true when either buffer contains a full window of samples. This greedy approach prevents one slow stream from blocking the entire pipeline while ensuring the mixing loop progresses deterministically.

Extracting Synchronized Windows

The extract_window method pulls aligned audio segments from both buffers. If one source lacks sufficient samples, the implementation zero-pads the remainder rather than repeating the last sample, which eliminates audible artifacts like stuttering or echo.

fn extract_window(&mut self) -> Option<(Vec<f32>, Vec<f32>)> {
    // Drain window_size_samples from each buffer
    // Zero-pad if underfilled
    // Return None only when both buffers are empty
}

The function returns Option<(Vec<f32>, Vec<f32>)> containing the microphone and system windows respectively. When both queues are exhausted, it yields None, signaling the pipeline to await new capture data.

Mixing Strategy

Once windows are extracted, ProfessionalAudioMixer::mix_window combines them into a single PCM buffer. The mixer applies soft scaling to prevent hard clipping: it calculates the absolute sum of overlapping samples and attenuates the result to maintain the [-1.0, +1.0] range. System audio receives a configurable gain factor (default 1.0) while microphone audio remains at full level, preserving voice clarity over background audio.

End-to-End Pipeline Flow

The complete synchronization workflow operates as follows:

1. Capture: AudioCapture streams concurrently from microphone and system devices.
2. Enqueue: Each chunk invokes ring_buffer.add_samples(device_type, samples).
3. Check: The recording manager loops, polling ring_buffer.can_mix().
4. Extract: extract_window() returns aligned vectors, zero-padding any deficit.
5. Mix: ProfessionalAudioMixer blends the windows with soft-clipping protection.
6. Emit: The mixed buffer writes to disk via RecordingSaver and feeds the Whisper transcription engine.

Because the ring buffer preserves FIFO ordering and drops only the oldest excess samples, the system automatically corrects for clock drift between devices without introducing phase distortion.

Implementation Example

To integrate the synchronizer directly:

use meetily::audio::pipeline::{AudioMixerRingBuffer, ProfessionalAudioMixer};
use meetily::audio::recording_state::DeviceType;

// Initialize with 48 kHz sample rate
let mut ring = AudioMixerRingBuffer::new(48_000);
let mut mixer = ProfessionalAudioMixer::new(48_000);

// Feed captured chunks
ring.add_samples(DeviceType::Microphone, mic_chunk);
ring.add_samples(DeviceType::System, system_chunk);

// Process when ready
while let Some((mic_win, sys_win)) = ring.extract_window() {
    let mixed = mixer.mix_window(&mic_win, &sys_win);
    // Write to file or transmit to transcription service
    recording_saver.save_chunk(mixed).await;
}

This pattern ensures that even if the system audio driver stalls temporarily, the microphone stream continues processing with silence substituted for the missing frames, maintaining perfect alignment for downstream consumers.

Summary

• Dual-ring architecture: Separate VecDeque buffers for microphone and system audio prevent cross-contamination while allowing independent drift compensation.
• Deterministic windows: Fixed 50 ms extraction intervals guarantee uniform chunk sizes required by Whisper transcription models.
• Zero-padding safety: Missing samples are filled with silence rather than repeated frames, eliminating looping artifacts.
• Soft-clipping mixer: ProfessionalAudioMixer prevents digital distortion when combining multiple audio sources.
• Jitter resilience: The 400 ms max_buffer_size absorbs temporary latency spikes common in macOS Core Audio and Windows WASAPI capture.

Frequently Asked Questions

Why does Meetily use zero-padding instead of sample repetition when a buffer underflows?

Zero-padding inserts digital silence for missing samples, which creates a clean gap in the audio waveform. Repeating the last sample would generate a DC offset or "stuck" audio effect that sounds like a stutter or click. According to the extract_window implementation in pipeline.rs, the pipeline prioritizes artifact-free output over strict sample preservation.

How does the ring buffer handle clock drift between microphone and system audio?

The FIFO design automatically compensates for drift by allowing the faster stream to accumulate samples while the slower stream catches up. If the microphone consistently delivers more samples, the excess is eventually trimmed by max_buffer_size limits during add_samples. During extraction, the deficit is zero-padded, ensuring the mixing window always maintains the exact window_size_samples length required for synchronization.

What is the purpose of the 400 ms max_buffer_size limit?

The max_buffer_size parameter acts as a safety valve against unbounded memory growth during network or driver jitter. Set to approximately 400 milliseconds, this value accommodates latency variations observed in macOS Core Audio without wasting RAM. When buffers exceed this threshold, the oldest samples are discarded to maintain real-time processing constraints.

Can the mixing window size be adjusted for different transcription models?

Yes. The window_size_samples field is calculated from the sample rate and a target duration—defaulting to 50 ms—passed during AudioMixerRingBuffer::new(). Developers can instantiate the buffer with custom parameters to match alternative chunk sizes required by different Whisper model configurations or other speech-to-text engines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}