# How Meetily's Audio Pipeline Handles Microphone and System Audio Mixing

> Discover how Meetily's audio pipeline simultaneously captures mic and system audio, synchronizes streams with a ring buffer, and creates a single mix using soft-clipping. Learn more about Meetily's advanced audio handling.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: how-to-guide
- Published: 2026-07-20

---

**Meetily captures microphone input and system audio simultaneously through separate device streams, synchronizes them via a ring buffer, and blends them into a single professional-grade mix using soft-clipping algorithms before routing to Whisper transcription and WAV file storage.**

Meetily is an open-source meeting assistant built with Tauri and Rust that requires pristine audio capture for accurate AI transcription. The application's **audio pipeline** implements a sophisticated multi-stage subsystem that processes microphone and system audio as independent streams, applies broadcast-quality enhancement and mixing, and delivers synchronized output for both real-time speech recognition and archival recording. This implementation resides primarily in [`frontend/src-tauri/src/audio/pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/pipeline.rs) within the Zackriya-Solutions/meetily repository.

## Stage 1: Device-Specific Capture and Audio Enhancement

Each audio source operates through its own `AudioCapture` instance that handles device-specific ingestion and preprocessing. The **microphone stream** undergoes a professional enhancement chain before entering the mixing pipeline:

- **High-pass filtering** to remove low-frequency rumble
- **RNNoise noise suppression** for neural speech enhancement  
- **EBU-R128 loudness normalization** targeting broadcast-standard -23 LUFS

The **system audio stream** (capturing application sounds, notifications, and media) bypasses enhancement and enters the pipeline at raw volume. Both streams are converted to mono format and resampled to the pipeline's native **48 kHz** sample rate using Rubato resamplers when device-native rates differ. The processed data is packaged as `AudioChunk` structs and forwarded to the shared pipeline state via `AudioCapture::process_audio_data`.

```rust
// Simplified excerpt from AudioCapture implementation
impl AudioCapture {
    fn process_audio_data(&mut self, raw_samples: &[f32]) {
        // Apply enhancement chain for microphone only
        let processed = if self.device_kind == DeviceKind::Microphone {
            self.apply_highpass(raw_samples);
            self.apply_rnnoise(processed);
            self.normalize_loudness(processed)
        } else {
            raw_samples.to_vec()
        };
        
        // Convert to mono and resample if necessary
        let mono = stereo_to_mono(&processed);
        let resampled = self.resampler.process(mono);
        
        // Forward to pipeline
        self.state.send_audio_chunk(AudioChunk {
            samples: resampled,
            timestamp: Instant::now(),
            device_id: self.device_id,
        });
    }
}

```

## Stage 2: Ring-Buffer Synchronization for Dual-Stream Alignment

The `AudioPipeline` receives chunks from both devices through an unbounded Tokio channel and manages temporal alignment via an `AudioMixerRingBuffer`. This component stores microphone and system samples in two independent `VecDeque` buffers until sufficient data accumulates for a fixed-size mixing window (default **50 ms**).

The ring buffer implements critical synchronization logic:

- **Latency absorption**: Maintains a `max_buffer_size` of **400 ms** (8× the mixing window) to handle bursty delivery patterns common in macOS Core Audio drivers
- **Silence padding**: When one stream temporarily stalls, `extract_window` pads the missing window with zeros rather than repeating samples, eliminating click artifacts
- **Memory protection**: Caps total buffer size to prevent unbounded growth during extended recording sessions

Once both buffers contain sufficient samples, `can_mix` returns true and `extract_window` yields synchronized windows ready for combination.

```rust
// Ring buffer extraction logic
if let Some((mic_win, sys_win)) = self.ring_buffer.extract_window() {
    // Both windows guaranteed equal length and temporal alignment
    let mixed = self.mixer.mix_window(&mic_win, &sys_win);
    // Forward to downstream processors...
}

```

## Stage 3: Professional Mixing with Soft-Clipping Algorithm

The `ProfessionalAudioMixer` combines synchronized windows through `mix_window`, implementing broadcast-grade mixing techniques to prevent distortion. The mixer applies **proportional gain staging**: system audio receives a 70% volume reduction (attenuating to 30% of original level) while microphone audio maintains full normalized level from the EBU-R128 stage.

To prevent the digital clipping that occurs when combined samples exceed the ±1.0 floating-point range, Meetily implements **soft-clipping protection**:

```rust
impl ProfessionalAudioMixer {
    fn mix_window(&mut self, mic_window: &[f32], sys_window: &[f32]) -> Vec<f32> {
        let max_len = mic_window.len().max(sys_window.len());
        let mut mixed = Vec::with_capacity(max_len);
        
        for i in 0..max_len {
            let mic = mic_window.get(i).copied().unwrap_or(0.0);
            let sys = sys_window.get(i).copied().unwrap_or(0.0) * 0.3; // 70% reduction
            
            let sum = mic + sys;
            
            // Soft clipping: proportional scaling if exceeding unit range
            let mixed_sample = if sum.abs() > 1.0 {
                sum / sum.abs() // Scales back to ±1.0 proportionally
            } else {
                sum
            };
            
            mixed.push(mixed_sample);
        }
        mixed
    }
}

```

This approach preserves audio fidelity by smoothly compressing transients rather than introducing the harsh "radio-break" distortion of hard clipping.

## Handling Sample Rate Mismatches and Platform Latency

When input devices operate at different sample rates (e.g., 44.1 kHz consumer microphone vs. 48 kHz system audio), `AudioCapture` instantiates a persistent **Rubato resampler** that buffers variable-size input chunks until accumulating a **512-sample block** for processing. This block-wise approach preserves RMS energy across chunk boundaries while maintaining the pipeline's strict 48 kHz internal standard.

The ring buffer's 400 ms capacity specifically addresses platform-specific jitter characteristics, particularly the bursty callback patterns observed in macOS Core Audio implementations where system audio may arrive in irregular bursts rather than steady intervals.

## Downstream Routing: Transcription vs. Recording

After mixing, the pipeline performs **no additional gain staging**—the microphone signal already maintains broadcast-level loudness and the system audio has been attenuated. The mixed stream then splits for dual consumption:

1. **VAD Processing**: The mixed audio passes through voice-activity detection ([`vad.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/vad.rs)), which extracts speech-only segments and forwards them to Whisper for transcription
2. **Recording Archive**: The complete mixed stream (both mic and system audio) routes to a WAV file writer, ensuring saved meetings contain the full conversational context including background audio and participant system sounds

This separation prevents echo and double-recording while ensuring the transcription engine receives clean speech data without non-verbal system sounds.

```rust
// Orchestration in AudioPipeline::run
loop {
    if let Some((mic, sys)) = self.ring_buffer.extract_window() {
        let mixed = self.mixer.mix_window(&mic, &sys);
        
        // Parallel downstream routing
        self.vad.process(&mixed);           // Speech extraction for Whisper
        self.recording_sender.send(mixed);  // Full mix to WAV file
    }
}

```

## Summary

- **Dual-stream capture**: Separate `AudioCapture` instances handle microphone and system audio with device-specific preprocessing chains
- **Broadcast-quality enhancement**: Microphone audio receives RNNoise suppression and EBU-R128 normalization to -23 LUFS before mixing
- **Temporal alignment**: `AudioMixerRingBuffer` synchronizes streams using 50 ms windows with 400 ms jitter absorption and zero-padding for missing data
- **Professional mixing**: `ProfessionalAudioMixer` applies 70% system audio attenuation and soft-clipping to prevent distortion when summing streams
- **Flexible resampling**: Rubato-based resamplers handle arbitrary input sample rates by buffering to 512-sample blocks while preserving energy
- **Dual routing**: Mixed audio feeds both VAD/Whisper transcription (speech only) and WAV recording (full mix) without additional gain stages

## Frequently Asked Questions

### How does Meetily prevent hard clipping when mixing microphone and system audio?

The `ProfessionalAudioMixer::mix_window` implementation detects when the sum of microphone and scaled system samples exceeds the ±1.0 floating-point range. Instead of truncating values (hard clipping), it proportionally scales the entire sample back to the unit range using `sum / sum.abs()`, preserving the waveform's relative dynamics while preventing digital distortion.

### What synchronization mechanism ensures microphone and system audio align correctly?

Meetily uses an `AudioMixerRingBuffer` that stores each stream in independent `VecDeque` buffers. The `extract_window` method only returns paired windows when both buffers contain sufficient samples for the 50 ms mixing window. If one stream lags, the buffer pads with silence (zeros) rather than repeating samples, maintaining temporal alignment without introducing phase artifacts.

### How does Meetily handle different sample rates between input devices?

When `AudioCapture` detects a sample rate mismatch with the pipeline's native 48 kHz, it creates a persistent Rubato resampler for that device. This resampler accumulates variable-size input chunks until reaching a 512-sample processing block, converts the rate while preserving RMS energy, and forwards the resampled data to the mixing pipeline.

### Why does Meetily attenuate system audio before mixing?

The mixer reduces system audio by approximately 70% (to 30% of original level) to maintain proper vocal prominence in the final mix. Since microphone audio undergoes EBU-R128 loudness normalization to broadcast standards (-23 LUFS) while system audio enters raw, this attenuation prevents background application sounds and notifications from overpowering the primary speaker while still remaining audible in the recording.