How Meetily Synchronizes Async Audio Streams with Ring Buffer Mixing

Meetily uses a lock-free AudioMixerRingBuffer to accumulate independent microphone and system audio samples, aligns them into fixed 50 ms windows, and mixes them only when synchronized pairs are available, padding with silence when one stream temporarily lags.

Capturing microphone speech and system output simultaneously in a desktop application creates a classic asynchronous programming challenge. In the Zackriya-Solutions/meetily codebase, these two audio sources run on separate async tasks and produce samples at unpredictable rates. To deliver glitch-free mixed recordings and accurate transcription, the app implements a deterministic ring buffer synchronization strategy that aligns disparate streams before they reach the mixer.

The Ring Buffer Architecture

The synchronization logic centers on AudioMixerRingBuffer, defined in frontend/src-tauri/src/audio/pipeline.rs at lines 16–23. This structure maintains two independent sample queues using VecDeque<f32>: one for microphone samples (mic_buffer) and one for system audio (system_buffer).

Window Sizing and Alignment

Each queue accumulates raw PCM samples until they reach window_size_samples, a value calculated from the sample rate to approximate 50 ms of audio. This fixed window size is critical for downstream processing because the Voice Activity Detection (VAD) and Whisper transcription engines expect consistent temporal chunks. By buffering both streams independently and only extracting when aligned windows are available, the pipeline guarantees temporal coherence even when the underlying capture tasks drift.

Memory Safety Limits

To prevent unbounded memory growth during network congestion or CPU pressure, the implementation caps each queue with max_buffer_size (approximately 400 ms of audio). When a queue exceeds this limit, the oldest samples are automatically trimmed, ensuring the application remains responsive without dropping the entire stream.

Adding Samples and Flow Control

Audio chunks arrive from separate capture threads via the add_samples method (lines 49–84 in pipeline.rs). This method accepts a DeviceType enum flag—either Microphone or System—and a Vec<f32> of raw samples.

// Inside AudioMixerRingBuffer
pub fn add_samples(&mut self, device_type: DeviceType, samples: Vec<f32>) {
    match device_type {
        DeviceType::Microphone => self.mic_buffer.extend(samples),
        DeviceType::System => self.system_buffer.extend(samples),
    }
    
    // Trim if exceeding max_buffer_size (≈400ms)
    if self.mic_buffer.len() > self.max_buffer_size {
        self.mic_buffer.drain(0..self.mic_buffer.len() - self.max_buffer_size);
    }
    // Similar trim for system_buffer...
}

Each call appends the new samples to the appropriate queue, logs buffer health metrics, and applies the overflow safeguard. This design allows the pipeline to accept data as fast as the OS audio APIs provide it, without blocking the async runtime.

Synchronization and Window Extraction

The actual synchronization happens in two stages: readiness checking and aligned extraction.

The can_mix Check

Before attempting to mix, the pipeline queries can_mix (lines 87–90), which returns true when either queue contains enough samples for a full window:

pub fn can_mix(&self) -> bool {
    self.mic_buffer.len() >= self.window_size_samples || 
    self.system_buffer.len() >= self.window_size_samples
}

This asynchronous approach is deliberate—if the system audio lags behind the microphone, the pipeline does not stall. It continues processing available microphone windows, mixing them with silence, until the system stream catches up.

Zero-Padding Strategy

When the pipeline calls extract_window (lines 92–141), it attempts to pull exactly window_size_samples from each queue. If one queue lacks sufficient data, the method pads the missing portion with zeros (silence). This preserves window length consistency, prevents audible artifacts like clicks or pops, and maintains alignment with the transcription engine's expectations.

pub fn extract_window(&mut self) -> Option<(Vec<f32>, Vec<f32>)> {
    if !self.can_mix() { return None; }
    
    let mut mic_window = Vec::with_capacity(self.window_size_samples);
    let mut sys_window = Vec::with_capacity(self.window_size_samples);
    
    // Drain available mic samples, pad with zeros if insufficient
    for _ in 0..self.window_size_samples {
        mic_window.push(self.mic_buffer.pop_front().unwrap_or(0.0));
    }
    
    // Same for system audio...
    Some((mic_window, sys_window))
}

Integration with the Audio Pipeline

The main processing loop in AudioPipeline::run (lines 818–828) orchestrates the synchronization. For each incoming AudioChunk, it calls add_samples, then enters a drain loop:

// Inside the main async loop
self.ring_buffer.add_samples(chunk.device_type, chunk.data);

while self.ring_buffer.can_mix() {
    if let Some((mic_win, sys_win)) = self.ring_buffer.extract_window() {
        let mixed = self.mixer.mix_window(&mic_win, &sys_win);
        // Send to VAD/transcription and recording sink
        self.process_mixed_audio(mixed).await;
    }
}

This while loop ensures that if multiple windows are ready (perhaps because the UI thread was briefly paused), the pipeline catches up immediately rather than processing one window per chunk.

Implementation Example

To utilize the ring buffer synchronization in your own Tauri-based Rust audio pipeline:

use meetily::audio::pipeline::{AudioMixerRingBuffer, DeviceType};

// Initialize with 48kHz sample rate
let mut ring = AudioMixerRingBuffer::new(48_000);

// Simulate receiving asynchronous chunks
let mic_chunk = vec![0.1_f32; 2400];  // 50ms at 48kHz
let sys_chunk = vec![0.05_f32; 2400];

ring.add_samples(DeviceType::Microphone, mic_chunk);
ring.add_samples(DeviceType::System, sys_chunk);

// Process only when synchronized
if ring.can_mix() {
    if let Some((mic, sys)) = ring.extract_window() {
        // mic and sys are guaranteed to be the same length
        let mixed: Vec<f32> = mic.iter()
            .zip(sys.iter())
            .map(|(m, s)| m * 0.7 + s * 0.3)  // Simple mix
            .collect();
        // Forward to transcription...
    }
}

From the TypeScript frontend, raw audio flows into this pipeline via Tauri's invoke system:

// frontend/src/components/AudioCapture.tsx
await invoke('audio_chunk', {
  device_type: 'Microphone',
  data: Float32Array.from(micSamples)  // Raw PCM from Web Audio API
});

Summary

• Lock-free architecture: Two VecDeque<f32> queues in AudioMixerRingBuffer handle asynchronous input without blocking the async runtime.
• Fixed-window alignment: 50 ms windows ensure compatibility with VAD and transcription engines while accommodating drift between streams.
• Zero-padding fallback: extract_window pads incomplete buffers with silence to maintain temporal consistency and prevent audio artifacts.
• Memory protection: max_buffer_size caps queues at approximately 400 ms, auto-trimming excess to prevent overflow.
• Pipeline integration: The main AudioPipeline::run loop continuously drains ready windows, enabling catch-up processing when the UI thread resumes.

Frequently Asked Questions

What happens when one audio stream is slower than the other?

The asynchronous design allows the pipeline to continue processing the faster stream. When can_mix detects that one buffer has reached window_size_samples, extract_window pulls the available data from that buffer and pads the slower buffer with zeros. This produces a complete mixed window without stalling the entire pipeline, ensuring the recording remains continuous even when system audio temporarily lags behind the microphone.

How does Meetily prevent memory overflow in the ring buffers?

Each queue enforces a max_buffer_size limit of approximately 400 ms of audio samples. In add_samples, after extending the queue with new data, the method checks if the length exceeds this cap and drains the oldest samples from the front of the VecDeque. This safeguard prevents unbounded growth if the downstream transcription pipeline stalls due to CPU load or network latency.

What is the mixing window size and why was it chosen?

The window size defaults to roughly 50 ms of audio, calculated dynamically from the sample rate (sample_rate * 0.05). This duration balances latency and processing efficiency: it is short enough to minimize delay between speech and transcription, yet long enough to provide sufficient context for the VAD (Voice Activity Detection) algorithm to distinguish speech from background noise in frontend/src-tauri/src/audio/vad.rs.

How does the frontend send audio data to the Rust backend?

The TypeScript frontend captures raw PCM samples from the Web Audio API and transmits them via Tauri's invoke command to the audio_chunk handler. The data is sent as a Float32Array along with a device_type discriminator ('Microphone' or 'System'), which the Rust runtime routes to the appropriate queue in AudioMixerRingBuffer within the Tauri command handler.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}