How Meetily Uses Ring Buffer Synchronization to Mix Microphone and System Audio

Meetily synchronizes asynchronous microphone and system audio streams using a dual VecDeque ring buffer structure that aligns samples into fixed windows, zero-padding missing data to ensure continuous, time-aligned mixing suitable for real-time transcription.

Meetily, an open-source meeting transcription tool by Zackriya-Solutions, must merge two independent audio sources—microphone input and system audio—into a single synchronized track before sending data to Whisper for transcription. The ring buffer synchronization mechanism implemented in frontend/src-tauri/src/audio/pipeline.rs ensures that audio from both sources remains time-aligned and gap-free, even when the streams arrive at different rates or experience temporary silence.

Ring Buffer Architecture in AudioMixerRingBuffer

At the core of Meetily’s audio pipeline lies the AudioMixerRingBuffer struct, which isolates incoming samples from each device into separate FIFO queues before synchronization.

Dual VecDeque Structure

The struct maintains two independent buffers defined at lines 16-23 of pipeline.rs:

struct AudioMixerRingBuffer {
    mic_buffer: VecDeque<f32>,        // Samples from the microphone
    system_buffer: VecDeque<f32>,     // Samples from the system audio
    window_size_samples: usize,       // Fixed mixing window (≈50 ms)
    max_buffer_size: usize,           // Safety cap (≈400 ms)
}

VecDeque<f32> provides O(1) push and pop operations on both ends, making it ideal for real-time audio processing. The window_size_samples field defines the mixing granularity—currently configured to approximately 600 milliseconds of audio—while max_buffer_size acts as a safety cap set to eight times the window size (roughly 400-4800 ms depending on configuration) to tolerate platform-specific jitter, particularly from macOS Core Audio.

Ingesting Asynchronous Audio Streams

Microphone and system audio arrive on different schedules and chunk sizes. The ring buffer handles this asynchrony through the add_samples method implemented at lines 49-85.

The add_samples Method

This method routes incoming audio chunks to the appropriate buffer:

fn add_samples(&mut self, device_type: DeviceType, samples: Vec<f32>) {
    match device_type {
        DeviceType::Microphone => self.mic_buffer.extend(samples),
        DeviceType::System => self.system_buffer.extend(samples),
    }
    // Overflow protection – drop oldest samples if the buffer exceeds max size
    while self.mic_buffer.len() > self.max_buffer_size { self.mic_buffer.pop_front(); }
    while self.system_buffer.len() > self.max_buffer_size { self.system_buffer.pop_front(); }
}

The extend operation appends entire chunks efficiently, while the overflow protection loops ensure memory usage remains bounded by discarding the oldest samples when buffers exceed max_buffer_size. This prevents unbounded growth during network delays or processing spikes.

Synchronization and Window Extraction

Synchronization occurs when the pipeline extracts aligned windows of identical sample counts from both buffers, ready for mixing.

Readiness Checking with can_mix

The can_mix method at lines 87-90 determines when sufficient data exists to proceed:

fn can_mix(&self) -> bool {
    self.mic_buffer.len() >= self.window_size_samples ||
    self.system_buffer.len() >= self.window_size_samples
}

This logic triggers mixing whenever either buffer contains a full window, allowing the pipeline to continue processing even if one stream is temporarily silent—the missing side will be handled via zero-padding rather than blocking the entire pipeline.

Zero-Padding in extract_window

The extract_window method at lines 92-141 performs the actual synchronization:

fn extract_window(&mut self) -> Option<(Vec<f32>, Vec<f32>)> {
    if !self.can_mix() { return None; }

    // Mic window – zero‑pad if not enough samples
    let mic_window = if self.mic_buffer.len() >= self.window_size_samples {
        self.mic_buffer.drain(0..self.window_size_samples).collect()
    } else {
        let mut padded = self.mic_buffer.drain(..).collect::<Vec<_>>();
        padded.resize(self.window_size_samples, 0.0);
        padded
    };

    // System window – same zero‑padding strategy
    let sys_window = if self.system_buffer.len() >= self.window_size_samples {
        self.system_buffer.drain(0..self.window_size_samples).collect()
    } else {
        let mut padded = self.system_buffer.drain(..).collect::<Vec<_>>();
        padded.resize(self.window_size_samples, 0.0);
        padded
    };

    Some((mic_window, sys_window))
}

When extracting, the method drains exactly window_size_samples from each deque. If a buffer contains fewer samples than required, the method pads the remaining space with 0.0 (silence) rather than repeating samples or waiting indefinitely. This zero-padding strategy avoids "pumping" artifacts while maintaining strict time alignment between the two streams.

Pipeline Integration

The AudioPipeline orchestrates the mixing process in its main processing loop around lines 172-179:

while self.ring_buffer.can_mix() {
    if let Some((mic_window, sys_window)) = self.ring_buffer.extract_window() {
        let mixed_clean = self.mixer.mix_window(&mic_window, &sys_window);
        // mixed_clean → VAD → Whisper OR → recording sender
    }
}

Here, the pipeline repeatedly checks can_mix() and extracts synchronized pairs via extract_window(). The ProfessionalAudioMixer::mix_window method combines the two streams using soft scaling without aggressive ducking, producing a single mixed buffer. This buffer then flows to the Voice Activity Detection (VAD) processor for transcription and simultaneously to the recording channel for persistent WAV file storage.

Why This Design Works

Meetily’s ring buffer synchronization solves several critical real-time audio challenges:

• Asynchronous arrival rates: Separate VecDeque instances allow each stream to enqueue at its own rate without blocking the other.
• Jitter tolerance: The max_buffer_size cap (eight times the window size) absorbs latency spikes from macOS Core Audio and other platform-specific delays.
• Handling missing data: Zero-padding guarantees constant-length mixed windows without introducing audible clicks or gaps when one source is silent.
• Real-time performance: All VecDeque operations are O(1), and the pipeline only consumes CPU cycles when sufficient data is present, keeping resource usage low during idle periods.

Summary

• Dual VecDeque structure in AudioMixerRingBuffer isolates microphone and system audio streams while maintaining O(1) insertion and removal performance.
• 600ms mixing windows with zero-padding ensure time-aligned output even when one stream lacks data.
• Overflow protection bounds memory usage by dropping aged samples when buffers exceed the safety cap.
• Non-blocking extraction via can_mix() allows continuous processing of available audio without waiting for slow or silent sources.

Frequently Asked Questions

Why does Meetily use ring buffers instead of simple concatenation for audio mixing?

Ring buffers provide FIFO ordering with O(1) operations at both ends, which is essential for real-time audio where samples must be consumed in strict chronological order while new data arrives asynchronously. Simple concatenation would require O(n) reallocation or complex index management, whereas VecDeque in frontend/src-tauri/src/audio/pipeline.rs offers predictable performance characteristics necessary for live transcription.

How does the mixer handle temporary silence in one audio stream?

When extract_window detects that one buffer contains fewer samples than window_size_samples, it drains the available samples and resizes the vector to the full window length using 0.0 values. This zero-padding strategy, implemented at lines 106-111 and 118-123, inserts silence rather than causing delays or artifacts, ensuring the mixed track maintains consistent timing.

What prevents memory leaks if audio chunks arrive faster than they are processed?

The add_samples method includes overflow protection that continuously removes the oldest samples via pop_front() whenever either buffer exceeds max_buffer_size. This safety mechanism, found at lines 56-59, bounds memory consumption to approximately 400-4800 milliseconds of audio per stream regardless of input rate.

How is the mixing window size determined and why does it matter?

The window size is calculated from the sample rate using the formula sample_rate * window_ms / 1000, currently configured to roughly 600 milliseconds. This granularity balances latency (shorter windows enable faster transcription response) with mixing efficiency (longer windows amortize processing overhead), ensuring Whisper receives sufficiently large chunks for accurate speech recognition while maintaining real-time responsiveness.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}