# How Meetily Handles Simultaneous Microphone and System Audio Capture and Mixing

> Discover how Meetily captures, synchronizes, and mixes microphone and system audio simultaneously using advanced ring-buffer and soft-clipping techniques for seamless recording and transcription.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: how-to-guide
- Published: 2026-07-21

---

**Meetily captures microphone and system audio as independent streams, synchronizes them through a ring-buffer mechanism, and mixes them using a professional-grade soft-clipping algorithm before sending the combined output to recording and transcription engines.**

The open-source meeting assistant Meetily, developed by Zackriya-Solutions, implements a sophisticated Rust-based audio pipeline to handle simultaneous microphone and system audio capture and mixing. This architecture ensures that both the user's voice and system sounds—such as meeting participants or media playback—are recorded in perfect synchronization without distortion. The entire workflow is orchestrated in [`frontend/src-tauri/src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/pipeline.rs), where raw audio streams undergo capture, buffering, and professional mixing before reaching Whisper for transcription or local storage.

## The Three-Stage Audio Pipeline Architecture

Meetily's audio processing divides into three distinct stages: device-specific capture with preprocessing, ring-buffer synchronization, and professional mixing with downstream routing.

### Stage 1: Device-Specific Capture and Preprocessing

Each audio source is managed by an `AudioCapture` instance that handles device-specific initialization and signal enhancement. Microphone input undergoes a comprehensive preprocessing chain including a high-pass filter, RNNoise noise suppression, and **EBU-R128 loudness normalization** to broadcast-level standards (-23 LUFS). System audio is captured as-is without enhancement but undergoes the same format standardization.

Both streams are converted to mono and resampled to the pipeline's native **48 kHz** sample rate when necessary. The `AudioCapture::process_audio_data` method applies the enhancement chain and forwards the processed chunks as `AudioChunk` structures to the shared pipeline state via `state.send_audio_chunk`.

### Stage 2: Ring-Buffer Synchronization

The `AudioPipeline` receives raw chunks from both devices through an unbounded Tokio channel. An `AudioMixerRingBuffer` stores microphone and system samples in independent `VecDeque` buffers until each accumulates sufficient data for a fixed-size mixing window of approximately **50 milliseconds**.

The ring buffer handles timing discrepancies gracefully. When one stream delivers data faster than the other, the buffer pads missing data with silence rather than repeating samples, eliminating audible artifacts. The buffer also enforces a **maximum size of 400 ms** (eight times the mixing window) to absorb bursty delivery patterns common in macOS Core Audio while preventing runaway memory growth. Key methods include `AudioMixerRingBuffer::add_samples` for ingestion, `can_mix` for readiness checks, and `extract_window` for retrieval.

### Stage 3: Professional Mixing and Downstream Routing

Once the ring buffer reports sufficient data via `can_mix`, the `ProfessionalAudioMixer` combines the windows. The mixer applies a **70% attenuation** to system audio to prevent it from overwhelming the microphone signal, then implements **soft-clipping** protection: if the sum of samples exceeds ±1.0, the result is proportionally scaled back to stay within the unit range, avoiding the harsh distortion of hard clipping.

The mixing logic resides in `ProfessionalAudioMixer::mix_window`. After mixing, the pipeline performs no additional gain adjustment—the microphone signal remains at broadcast-level loudness while the system audio stays attenuated. The resulting mixed chunk follows two paths:

- Forwarded to the VAD (Voice Activity Detection) processor in [`vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/vad.rs), which extracts speech-only segments for Whisper transcription.
- Sent to a recording sender that writes the final mixed output to a WAV file containing both microphone and system audio.

This flow is orchestrated in the `AudioPipeline::run` loop, which coordinates the ring buffer, mixer, VAD, and recording sender in sequential order.

## Handling Technical Challenges in Dual-Audio Capture

Meetily addresses several critical engineering challenges inherent in capturing two independent audio streams.

**Sample-Rate Mismatches**: When input devices operate at different sample rates (e.g., 44.1 kHz vs. 48 kHz), `AudioCapture` creates a persistent Rubato resampler that buffers variable-size chunks until accumulating 512 samples for processing. This approach preserves RMS energy across chunks and maintains phase coherence before the data reaches the ring buffer.

**Latency and Jitter**: The 400 ms maximum buffer size specifically targets jitter in system audio delivery, particularly on macOS where Core Audio can exhibit bursty callbacks. This capacity absorbs temporal variance without introducing synchronization drift into the final mix.

**Clipping Avoidance**: The soft-scaling algorithm in `mix_window` dynamically adjusts the mixed signal when the sum exceeds unity gain. Unlike hard clipping, which truncates waveforms and introduces harmonic distortion, this proportional scaling maintains audio fidelity during loud passages where microphone and system sounds coincide.

**Stream Desynchronization**: If one device temporarily stops delivering data—such as during a Bluetooth microphone dropout—`extract_window` pads the missing window with zeros. This prevents the "stuck audio" artifact that occurs when buffers repeat the last valid sample.

## Implementation Example: Starting the Pipeline

The following excerpt from the Tauri command layer demonstrates how Meetily initializes the dual-capture pipeline:

```rust
// In frontend/src-tauri/src/lib.rs – simplified initialization
#[tauri::command]
async fn start_recording(
    app: tauri::AppHandle,
    mic_device_name: Option<String>,
    system_device_name: Option<String>,
    meeting_name: Option<String>,
) -> Result<(), String> {
    let state = RecordingState::new();
    let (trans_tx, trans_rx) = mpsc::unbounded_channel();
    let recording_sender = Some(mpsc::unbounded_channel().0);

    let mut manager = AudioPipelineManager::new();
    manager.start(
        Arc::new(state),
        trans_tx,
        20,              // target_chunk_duration_ms
        48_000,          // sample_rate
        recording_sender,
        mic_device_name.unwrap_or_default(),
        detect_mic_kind(),
        system_device_name.unwrap_or_default(),
        detect_system_kind(),
    )?;
    Ok(())
}

```

## Extracting and Mixing Audio Windows

The core mixing operation handles sample alignment and soft-clipping protection:

```rust
// From ProfessionalAudioMixer::mix_window in pipeline.rs
fn mix_window(&mut self, mic_window: &[f32], sys_window: &[f32]) -> Vec<f32> {
    let max_len = mic_window.len().max(sys_window.len());
    let mut mixed = Vec::with_capacity(max_len);
    
    for i in 0..max_len {
        let mic = mic_window.get(i).copied().unwrap_or(0.0);
        let sys = sys_window.get(i).copied().unwrap_or(0.0);
        // System attenuated by 70% (keeping 30%), mic at full level
        let sum = mic + sys * 0.3;
        
        // Soft clipping - proportional scaling if exceeding unity
        let mixed_sample = if sum.abs() > 1.0 { 
            sum / sum.abs() 
        } else { 
            sum 
        };
        mixed.push(mixed_sample);
    }
    mixed
}

```

The extraction and mixing sequence operates within the main pipeline loop:

```rust
// Simplified flow from AudioPipeline::run in pipeline.rs
if let Some((mic_win, sys_win)) = self.ring_buffer.extract_window() {
    let mixed = self.mixer.mix_window(&mic_win, &sys_win);
    // Forward to VAD for transcription
    self.vad.process(&mixed);
    // Forward to recording manager for WAV output
    if let Some(sender) = &self.recording_sender {
        sender.send(mixed.clone());
    }
}

```

## Summary

- Meetily captures microphone and system audio through separate `AudioCapture` instances, applying broadcast-level normalization to microphone input while treating system audio as a reference signal.
- The `AudioMixerRingBuffer` synchronizes independent streams using 50 ms mixing windows and 400 ms maximum buffering, padding silence for missing data to prevent artifacts.
- `ProfessionalAudioMixer::mix_window` combines streams with 70% system attenuation and soft-clipping protection to maintain audio fidelity when signals sum beyond unity gain.
- The mixed output routes to both a local WAV recorder and a VAD processor, ensuring transcription receives speech-only segments while recordings preserve the full audio context.
- All functionality resides in [`frontend/src-tauri/src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/pipeline.rs), with supporting orchestration in [`recording_manager.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/recording_manager.rs) and voice detection in [`vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/vad.rs).

## Frequently Asked Questions

### Does Meetily record microphone and system audio as separate tracks?

No. According to the source code in [`pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/pipeline.rs), Meetily mixes the streams before storage. The `ProfessionalAudioMixer` combines both sources into a single interleaved stream using soft-clipping, and only this mixed output is written to the WAV file. Raw separate streams are never persisted to disk, though they exist as independent buffers during the 50 ms windowing phase.

### How does Meetily prevent audio distortion when mixing loud system audio with microphone input?

The mixing algorithm implements proportional soft-scaling in `mix_window`. When the sum of microphone and attenuated system samples exceeds ±1.0, the code divides by the absolute value to scale the result back to unity while preserving the waveform shape. This prevents the hard-clipping distortion that creates "radio break" artifacts common in simple additive mixing.

### What happens if the microphone and system audio devices use different sample rates?

The `AudioCapture` implementation detects sample-rate mismatches and instantiates a persistent Rubato resampler for the non-native stream. This resampler buffers variable-size input chunks until accumulating 512 samples for processing, ensuring both streams arrive at the `AudioMixerRingBuffer` as 48 kHz mono audio suitable for synchronous mixing.

### Why does Meetily use a 400 ms buffer for audio synchronization?

The 400 ms maximum buffer size (defined in `AudioMixerRingBuffer`) accommodates jitter in system audio delivery, particularly on macOS where Core Audio exhibits bursty callback patterns. This capacity absorbs timing variance without introducing latency into the final mix, as the pipeline still processes audio in 50 ms windows once both buffers contain sufficient data.