How Meetily's Audio Pipeline Synchronizes Microphone and System Audio Buffers

Meetily uses a window-based ring buffer with dual VecDeque<f32> queues and zero-padding logic to align microphone and system audio streams before mixing, ensuring drift-free synchronization regardless of input latency variations.

Meetily is an open-source meeting recording application that captures both microphone input and system audio simultaneously. Synchronizing these independent audio streams presents a significant engineering challenge, as operating system audio callbacks arrive with different latencies and jitter characteristics. The project solves this through a sophisticated audio pipeline buffer synchronization mechanism implemented in Rust, centered around the AudioMixerRingBuffer struct in frontend/src-tauri/src/audio/pipeline.rs.

The AudioMixerRingBuffer Architecture

At the core of the synchronization logic sits the AudioMixerRingBuffer struct defined in frontend/src-tauri/src/audio/pipeline.rs (lines 16-23). This structure maintains two independent VecDeque<f32> queues—one for microphone samples and one for system audio.

The buffer allocates space for a fixed-size mixing window of approximately 600 milliseconds plus a 400-millisecond safety cap to absorb platform-specific jitter from APIs like Core Audio on macOS. By separating the streams into distinct queues while maintaining a unified extraction logic, the pipeline preserves source isolation for downstream Voice Activity Detection (VAD) processing while preparing for synchronized mixing.

The Six-Step Synchronization Workflow

The pipeline follows a deterministic ingestion and extraction cycle that guarantees aligned output.

1. Buffer Initialization and Capacity Management

When AudioCapture starts recording, it initializes the ring buffer with the target sample rate. The constructor establishes the window size in samples (based on the 600ms target) and sets hard limits to prevent unbounded memory growth during capture stalls.

2. Ingesting Samples via add_samples()

Each platform-specific capture callback—whether from microphone.rs or system_audio_stream.rs—invokes add_samples() (lines 49-84 of pipeline.rs). This method appends raw PCM floats to the appropriate queue (mic_buffer or system_buffer).

If either queue exceeds the safety limit (400ms), the function drops the oldest samples and emits a warning, preventing memory exhaustion during system stress.

3. Readiness Detection with can_mix()

Before extraction, can_mix() (lines 87-90) verifies synchronization feasibility. The function returns true if either queue contains a full window of samples. This "or" logic keeps the pipeline moving when one source lags, ensuring continuous recording without blocking on the slower stream.

4. Window Extraction and Zero-Padding

The extract_window() method (lines 92-140) pulls exactly window_size_samples from each queue. When a queue contains fewer samples than required—due to temporary dropout or latency differences—the method drains the available data and zero-pads the remainder with silence.

This prevents audible clicks or phase discontinuities while maintaining strict alignment between the two extracted windows.

5. Professional Audio Mixing

With two equal-length windows guaranteed, the ProfessionalAudioMixer::mix_window() function (lines 54-88) receives the paired buffers. The mixer applies per-sample scaling coefficients and soft clipping algorithms to produce a composite stream without distortion. Because synchronization happened upstream in the ring buffer, the mixer operates on guaranteed-aligned data without drift compensation logic.

6. Global Timestamp Correlation

Every mixed chunk carries a global recording timestamp (referenced at line 13 of pipeline.rs). This metadata allows downstream transcription services and Voice Activity Detection (in vad.rs) to correlate mixed audio frames with the original capture timeline, maintaining temporal accuracy for speaker diarization.

Why This Approach Eliminates Audio Drift

Several architectural decisions ensure robust audio pipeline buffer synchronization:

• Fixed-size windows create deterministic latency characteristics. The 600ms window defines the exact interval between mixed output chunks.
• Zero-padding guarantees continuous output even during temporary source silence or delay, eliminating the need for complex resampling or time-stretching algorithms.
• Independent queues preserve the original streams for parallel processing paths, such as running VAD exclusively on the microphone channel while mixing both sources.
• Safety buffers absorb real-world jitter from operating system schedulers without dropping active recording data.

Implementation Example

The following Rust pattern mirrors the production flow inside recording_manager.rs:

// Initialize with target sample rate (e.g., 48000 Hz)
let mut ring_buffer = AudioMixerRingBuffer::new(sample_rate);

// In microphone capture callback
ring_buffer.add_samples(DeviceType::Microphone, mic_pcm_data);

// In system audio capture callback  
ring_buffer.add_samples(DeviceType::System, system_pcm_data);

// Processing thread loop
if let Some((mic_window, sys_window)) = ring_buffer.extract_window() {
    let mut mixer = ProfessionalAudioMixer::new(sample_rate);
    let mixed_chunk = mixer.mix_window(&mic_window, &sys_window);
    // Forward to encoder or transcription service
}

Key Source Files

Understanding the synchronization architecture requires familiarity with these components:

• frontend/src-tauri/src/audio/pipeline.rs: Contains AudioMixerRingBuffer, ProfessionalAudioMixer, and the core synchronization logic at lines 16-140.
• frontend/src-tauri/src/audio/recording_manager.rs: Orchestrates capture tasks and instantiates AudioCapture instances.
• frontend/src-tauri/src/audio/capture/microphone.rs and system_audio_stream.rs: Platform-specific capture implementations that feed raw samples to the pipeline.
• frontend/src-tauri/src/audio/vad.rs: Consumes the isolated microphone stream for voice detection.
• frontend/src-tauri/src/audio/audio_processing.rs: Applies noise suppression and loudness normalization before mixing.

Summary

• Meetily implements audio pipeline buffer synchronization through a dual-queue ring buffer architecture in pipeline.rs.
• The AudioMixerRingBuffer maintains separate VecDeque<f32> queues for microphone and system audio with 600ms windows and 400ms safety caps.
• extract_window() uses zero-padding to ensure aligned output even when sources drift or drop temporarily.
• can_mix() uses "or" logic to prevent pipeline stalls when one source experiences latency.
• Global timestamps preserve temporal correlation for downstream transcription and analysis.

Frequently Asked Questions

How does Meetily handle clock drift between different audio hardware devices?

Meetily does not rely on hardware clock synchronization. Instead, the extract_window() method pulls samples based on the fixed 600ms window boundary. If one device delivers samples faster than the other, the excess accumulates in the 400ms safety buffer until the slower device catches up. If the buffer exceeds its limit, older samples drop. This software-based approach compensates for hardware clock variance without requiring low-level driver adjustments.

What happens if the system audio temporarily stops producing samples?

When extract_window() encounters an underrun in either queue, it zero-pads the missing samples to maintain the fixed window size. This produces silence in the mixed output for that source rather than shifting the audio stream or causing pipeline stalls. The zero-padding logic (lines 92-140 of pipeline.rs) ensures the mixer always receives equal-length buffers, preventing array index errors or audio artifacts.

Can the buffer window size be adjusted for lower latency recording?

The 600ms window represents a balance between latency tolerance and CPU efficiency. While the constant derives from the sample rate calculation in AudioMixerRingBuffer::new(), reducing the window would decrease latency but increase the risk of dropout on systems with high audio callback jitter. The 400ms safety cap provides headroom for most desktop environments, including macOS Core Audio and Windows WASAPI.

Why maintain separate queues instead of interleaving samples immediately after capture?

Separate queues preserve the integrity of each source stream for independent processing. The Voice Activity Detector (vad.rs) analyzes only the microphone channel to identify speaker segments, while the mixing stage requires both channels. Interleaving at capture time would force the VAD to process mixed audio, reducing speaker isolation accuracy. The dual-queue design allows parallel processing paths before final composition in ProfessionalAudioMixer::mix_window().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}