Ring Buffer Mixing Architecture in Meetily's Audio Pipeline

Meetily synchronizes asynchronous microphone and system audio streams using a dual-deque ring buffer that aligns samples into 50ms windows, applies proportional soft-clipping to prevent distortion, and routes the mixed output to both Whisper transcription and WAV recording.

Meetily's audio pipeline solves the challenge of combining two independent capture sources—user speech and system audio—into a single professional-quality stream. This ring buffer mixing architecture, implemented in Rust within the Tauri backend at frontend/src-tauri/src/audio/pipeline.rs, ensures synchronized playback despite differing capture rates and hardware latencies.

Core Ring Buffer Components

The AudioMixerRingBuffer struct serves as the central synchronization primitive, maintaining separate VecDeque<f32> buffers for each audio source until a complete mixing window is available.

Buffer Structure and Constraints

Defined at lines 16-23 in pipeline.rs, the ring buffer maintains two independent queues: one for microphone samples and one for system audio. The implementation enforces strict memory bounds to prevent unbounded growth during asynchronous capture.

The buffer operates with two critical timing parameters initialized in the new() constructor (lines 27-30):

• Mixing window: Approximately 50ms of samples, ensuring both streams align before processing
• Maximum buffer size: Approximately 400ms (line 35), acting as a safety cap against jitter or lagging streams

Sample Ingestion and Overflow Protection

The add_samples method (lines 49-85) receives chunks from either capture device and pushes them onto the appropriate deque. When buffer occupancy exceeds the 400ms threshold, the implementation trims older samples while emitting diagnostic warnings to the logging system. This protects against memory exhaustion when one stream temporarily stalls, particularly on macOS Core Audio where system audio bursts can be irregular.

The Mixing Workflow

Window Availability Detection

Before mixing occurs, the can_mix method (lines 87-90) checks if either buffer contains sufficient samples for a full 50ms window. Unlike strict synchronization schemes that wait for both streams, this approach allows the pipeline to begin processing as soon as one side is ready, reducing latency.

Zero-Padding and Window Extraction

The extract_window method (lines 92-140) pulls equal-length vectors from both buffers. When one stream lacks sufficient samples—such as during microphone silence or system audio gaps—the implementation applies zero-padding (silence) rather than repeating the last sample. This design choice prevents audible repetition artifacts when streams pause temporarily.

Professional Audio Mixing

Once windows are extracted, ProfessionalAudioMixer::mix_window (lines 54-88) performs the actual combination:

1. Attenuation: System audio is reduced to approximately 70% of full scale to prevent overwhelming speech
2. Summation: Microphone and attenuated system samples are added together
3. Soft clipping: If the summed sample exceeds ±1.0, it is proportionally scaled back rather than hard-clipped, preserving natural dynamics and avoiding the "radio-breaks" distortion mentioned in the project changelog

Pipeline Integration

Audio Capture to Ring Buffer

During the pipeline's run method, AudioCapture::process_audio_data pushes raw or resampled mono samples into the ring buffer via ring_buffer.add_samples(...). This occurs at lines 20-21 of the main processing loop, where samples are pre-processed through audio_processing.rs for mono conversion and noise suppression before entering the buffer.

Dual Output Routing

After mixing, the resulting buffer feeds two destinations simultaneously:

• Transcription path: The mixed audio flows through the Voice Activity Detection (VAD) processor in vad.rs before reaching Whisper for transcription
• Recording path: The same buffer is sent via recording_sender_for_mixed to recording_manager.rs, which coordinates writing the final mix to disk as a WAV file

Design Trade-offs and Robustness

The architecture prioritizes low latency and fidelity through specific engineering decisions:

Zero-padding vs. Last-sample hold: Choosing silence over sample repetition avoids artificial-sounding loops during stream interruptions, though it introduces momentary gaps that are less perceptible in practice.

Soft clipping implementation: Proportional scaling (soft clipping) maintains audio dynamics better than hard truncation, preventing the harsh digital distortion that would compromise transcription accuracy and listening comfort.

Asymmetric readiness: By allowing mixing when either buffer is ready (rather than requiring both), the system maintains responsiveness even when one capture device experiences driver delays or hardware latency variations.

Code Example

The following Rust snippet demonstrates the complete mixing cycle:

// 1. Initialise the ring buffer (sample_rate = 48 kHz)
let mut ring_buffer = AudioMixerRingBuffer::new(48_000);

// 2. Add a chunk from the microphone
ring_buffer.add_samples(DeviceType::Microphone, mic_samples);

// 3. Add a chunk from system audio
ring_buffer.add_samples(DeviceType::System, sys_samples);

// 4. When enough data exists, extract windows and mix
while ring_buffer.can_mix() {
    if let Some((mic_win, sys_win)) = ring_buffer.extract_window() {
        // Mix using the professional mixer
        let mixed = ProfessionalAudioMixer::new(48_000)
                       .mix_window(&mic_win, &sys_win);
        // `mixed` can now be sent to VAD/Whisper or written to a file
    }
}

Summary

• Ring buffer location: AudioMixerRingBuffer in frontend/src-tauri/src/audio/pipeline.rs (lines 16-23) manages dual VecDeque<f32> queues for microphone and system audio
• Timing parameters: 50ms mixing windows ensure synchronization while 400ms maximum buffers prevent memory overflow during jitter
• Mixing strategy: ProfessionalAudioMixer::mix_window applies 70% system attenuation and soft-clipping to combine streams without distortion
• Zero-padding approach: Silence insertion during window extraction prevents repetition artifacts when one stream pauses
• Dual output: Mixed audio simultaneously feeds Whisper transcription via vad.rs and WAV recording via recording_manager.rs

Frequently Asked Questions

How does Meetily handle when one audio stream lags behind the other?

The ring buffer implements a 400ms maximum capacity (line 35 in pipeline.rs) to prevent unbounded memory growth. When one stream lags, the add_samples method trims overflow samples and emits warnings while the extract_window function applies zero-padding to the shorter buffer. This allows mixing to continue uninterrupted without artificial repetition artifacts.

Why does Meetily use zero-padding instead of repeating the last sample?

Zero-padding inserts silence when a buffer lacks sufficient samples for a full window, whereas last-sample hold repeats the final captured value. According to the source code in extract_window (lines 92-140), silence was chosen to avoid audible repetition artifacts—such as stuttering or looping sounds—that occur when one stream temporarily pauses, particularly during speech gaps or system audio interruptions.

What prevents the mixed audio from clipping or distorting?

The ProfessionalAudioMixer::mix_window implementation (lines 54-88) applies soft-clipping rather than hard truncation. When the summed sample exceeds ±1.0, the algorithm proportionally scales the value back toward the limit. This preserves natural audio dynamics and prevents the "radio-breaks" distortion that would occur with hard clipping, while the system audio is pre-attenuated to 70% of full scale to reduce the likelihood of exceeding limits.

How does the 50ms window size affect transcription accuracy and latency?

The 50ms window (computed in new() at lines 27-30) represents a balance between responsiveness and synchronization quality. This duration is short enough to maintain low latency for real-time transcription through the VAD pipeline, yet long enough to ensure both asynchronous streams align consistently before mixing, preventing phase misalignment that could degrade transcription accuracy in the Whisper pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}