# Meetily Audio Pipeline Architecture: How It Synchronizes Dual Audio Streams

> Explore Meetily's audio pipeline architecture. Learn how it synchronizes dual audio streams with precise 50 ms windows for seamless mixing and transcription. Discover the lock-free buffering technique.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: architecture
- Published: 2026-07-13

---

**Meetily captures microphone and system audio as independent PCM streams, buffers them in a lock-free ring buffer, and synchronizes them by extracting fixed 50 ms windows with zero-padding to maintain perfect temporal alignment before mixing and transcription.**

The open-source meeting assistant Meetily (Zackriya-Solutions/meetily) implements a sophisticated audio pipeline architecture that records both user voice and desktop audio simultaneously. This Rust-based pipeline, primarily implemented in [`frontend/src-tauri/src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/pipeline.rs) and orchestrated by [`frontend/src-tauri/src/audio/recording_manager.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/recording_manager.rs), handles real-time resampling, professional-grade mixing, and voice activity detection while ensuring the two independent streams remain perfectly synchronized.

## Dual Stream Capture and Resampling

### Independent Device Capture

The pipeline captures two distinct **Pulse Code Modulation (PCM)** sources independently: the **microphone** (user voice) and the **system audio** (desktop output). Each stream originates from separate OS audio APIs—such as macOS Core Audio or Windows WASAPI—and enters the pipeline as raw audio chunks via device-specific implementations in [`frontend/src-tauri/src/audio/capture/microphone.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/capture/microphone.rs).

### Persistent Resampling with Rubato

To ensure uniformity, the architecture resamples all input to a target **48 kHz** sample rate. The `AudioCapture` struct maintains a reusable **Rubato resampler** with a dedicated `resampler_input_buffer`, allowing variable-size chunks to be correctly up-sampled or down-sampled without energy loss. This logic resides in `process_audio_data` (lines 1007–1052).

## The Synchronization Mechanism

### Lock-Free Ring Buffer Architecture

The core synchronization relies on the **`AudioMixerRingBuffer`** struct, which creates isolated buffers for microphone and system audio. Upon initialization (`AudioMixerRingBuffer::new` at lines 27–35), it establishes a fixed mixing window (approximately 50 ms) and a safety limit of eight times that window to prevent unbounded memory growth.

Incoming chunks route through `add_samples` (lines 49–84), which appends data to the appropriate `mic_buffer` or `system_buffer`. If either buffer exceeds the safety limit, the oldest samples drop to maintain real-time performance.

### Window-Based Extraction and Zero-Padding

Synchronization occurs in `extract_window` (lines 92–141). The pipeline checks readiness via `can_mix` (lines 87–90), which returns true when at least one buffer contains a full window of samples. This non-blocking approach ensures the pipeline never stalls waiting for the slower stream.

When extracting, the system retrieves fixed-size windows from both buffers. If one stream lags, the missing samples are **zero-padded** (filled with silence) rather than repeating the last sample. This technique prevents audible artifacts and maintains precise temporal alignment between the two sources.

### Soft-Scaling Mixing

The **`ProfessionalAudioMixer::mix_window`** method (lines 154–188) combines the aligned windows using soft scaling. When the sum of microphone and system samples exceeds ±1.0, the mixer proportionally scales both signals to prevent clipping without aggressive ducking. This yields a clean, undistorted final track written to the WAV file via `recording_sender_for_mixed` (lines 689–779).

## Audio Processing and Enhancement

### Microphone Signal Enhancement

Before entering the ring buffer, microphone audio undergoes enhancement in [`frontend/src-tauri/src/audio/audio_processing.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/audio_processing.rs). The signal chain includes:
- **High-pass filtering** to remove low-frequency rumble.
- **RNNoise noise suppression** for broadband noise reduction.
- **EBU R128 loudness normalization** to ensure consistent volume levels.

### Voice Activity Detection

The mixed audio stream feeds into the **`ContinuousVadProcessor`** defined in [`frontend/src-tauri/src/audio/vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/vad.rs). The `process_audio` method (lines 335–367) segments the audio into speech and non-speech regions, forwarding only segments exceeding **800 samples** (approximately 50 ms at 16 kHz) to the Whisper transcription engine. This dramatically reduces computational load while preserving transcription accuracy.

## Pipeline Orchestration and Thread Safety

### AudioPipelineManager Implementation

The **`AudioPipelineManager`** struct coordinates the entire flow. It initializes the capture interfaces and manages thread-safe communication using **`RecordingState`**, which stores an `Arc<AtomicBool>` and a `RwLock`-protected sender. This design allows capture threads to push chunks to the pipeline without blocking the Tauri UI thread.

### Graceful Shutdown with Flush Markers

To eliminate shutdown delays, the pipeline recognizes special **flush chunks** marked with `chunk_id >= u64::MAX - 10` (lines 781–889). These markers force the pipeline to empty all remaining buffers before exiting, ensuring no audio data is lost during the final write.

## Code Implementation

Below are practical examples showing how to instantiate the pipeline and push audio data:

```rust
// Create the pipeline manager from a Tauri command context
let mut pipeline_manager = AudioPipelineManager::new();

// Initialize with device metadata and senders
pipeline_manager.start(
    recording_state.clone(),
    transcription_sender.clone(),
    0,  // target_chunk_duration_ms
    48000,  // sample_rate
    Some(mixed_recording_sender.clone()),
    "Built‑in Microphone".to_string(),
    mic_device_kind,
    "BlackHole 2ch".to_string(),
    system_device_kind,
)?;

```

```rust
// Inside the audio callback for each device
let audio_capture = AudioCapture::new(
    device.clone(),
    recording_state.clone(),
    device.sample_rate(),
    device.channels(),
    DeviceType::Microphone,  // or DeviceType::System
    Some(pipeline_manager.audio_sender().clone()),
);
audio_capture.process_audio_data(&pcm_samples);

```

```rust
// Trigger a fast shutdown when stopping recording
pipeline_manager.force_flush_and_stop().await?;

```

## Summary

- **Dual independent capture**: Microphone and system audio are recorded separately as raw PCM to prevent cross-contamination.
- **48 kHz resampling**: The Rubato-based resampler in `AudioCapture` normalizes all input to a common sample rate.
- **Lock-free synchronization**: The `AudioMixerRingBuffer` aligns streams using fixed 50 ms windows with zero-padding for lagging sources.
- **Professional mixing**: Soft-scaling in `ProfessionalAudioMixer` prevents clipping while preserving dynamic range.
- **Efficient transcription**: The VAD processor filters non-speech audio, sending only relevant segments to Whisper.
- **Robust shutdown**: Flush markers ensure complete buffer drainage before recording termination.

## Frequently Asked Questions

### How does Meetily handle different sample rates between microphone and system audio?

According to the Meetily source code, each `AudioCapture` instance maintains a persistent **Rubato resampler** that converts arbitrary input rates to the pipeline's target 48 kHz. This occurs in `process_audio_data` (lines 1007–1052), where variable-size input chunks are resampled without energy loss before entering the synchronization buffer.

### What happens if one audio stream lags behind the other during recording?

The `extract_window` method (lines 92–141) handles latency by **zero-padding** the shorter buffer with silence. This ensures the mixer always receives temporally aligned windows of equal length, preventing the phasing or echo artifacts that would result from repeating samples or waiting for the slower stream.

### Why does Meetily use zero-padding instead of sample repetition for synchronization?

Zero-padding inserts literal silence (0.0 amplitude) for missing samples, whereas sample repetition would hold the last known value. According to the implementation in [`frontend/src-tauri/src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/pipeline.rs), zero-padding avoids audible clicks or droning artifacts when one source experiences jitter or temporary dropout, maintaining professional audio quality.

### How does the pipeline ensure the transcription engine receives only speech segments?

The **`ContinuousVadProcessor`** runs Voice Activity Detection on the mixed audio stream in `process_audio` (lines 335–367). It segments the audio and forwards only chunks containing speech—specifically those longer than 800 samples (approximately 50 ms at 16 kHz)—to the Whisper transcription sender, filtering out silence, background noise, and system sounds.