How Meetily's Professional Audio Mixing with RMS-Based Ducking Works

Meetily synchronizes microphone and system audio streams using a lock-free ring buffer, calculates the RMS (Root-Mean-Square) level of microphone input to detect speech activity, and automatically attenuates system audio by 40% when speech is detected to ensure clear voice recordings while preventing clipping.

Meetily, developed by Zackriya-Solutions, captures both microphone and system audio simultaneously to create comprehensive meeting recordings. The challenge lies in mixing these streams so that voice remains intelligible over background music or notifications without manual intervention. The solution is a professional audio mixing system with RMS-based ducking implemented in Rust, found in the frontend/src-tauri/src/audio/ directory.

The Audio Pipeline Architecture

The mixing system consists of three primary components working together to process audio in real-time.

AudioMixerRingBuffer – Located in frontend/src-tauri/src/audio/pipeline.rs, this lock-free VecDeque buffers both microphone and system samples until sufficient data exists for a complete mixing window. It aligns the two streams into fixed 50 ms windows, pads missing data with silence, and drops old samples when the buffer exceeds a 400 ms safety limit to prevent memory overflow.

ProfessionalAudioMixer – Also in pipeline.rs, this core mixer implements the RMS-based ducking algorithm. It calculates the energy level of the microphone input and determines whether to attenuate the system audio stream.

FFmpegAudioMixer – Found in frontend/src-tauri/src/audio/ffmpeg_mixer.rs, this is a full-featured alternative that maintains separate per-source buffers and implements the same ducking logic with additional gap-handling for Bluetooth devices.

How RMS-Based Ducking Works

The ducking algorithm processes audio in discrete steps to ensure broadcast-quality results.

Window Extraction and Synchronization

When both buffers contain at least window_size_samples (representing 50 ms of audio), the AudioMixerRingBuffer::extract_window method returns aligned slices (mic_window and sys_window). If one stream arrives late, the buffer zero-pads the missing data to maintain synchronization and prevent artifacts.

RMS Calculation and Speech Detection

The mixer calculates the RMS level of the microphone window to detect speech presence. As implemented in pipeline.rs, the calculation follows this standard formula:

fn calculate_rms(samples: &[f32]) -> f32 {
    let sum_squares: f32 = samples.iter().map(|s| s * s).sum();
    (sum_squares / samples.len() as f32).sqrt()
}

The system compares the result against a hard-coded SPEECH_THRESHOLD of 0.01. When the RMS exceeds this threshold, the mixer identifies the window as containing speech.

Dynamic Gain Adjustment

Based on the speech detection result, the mixer applies a ducking factor to the system audio:

• Speech detected: system_gain = 0.60 (reducing system audio by approximately 40%)
• Silence detected: system_gain = 1.0 (full system audio volume)

This attenuation occurs in real-time, ensuring that microphone audio always maintains prominence during active speech.

Mixing and Clipping Prevention

For each sample pair, the mixer computes the combined output and applies clamping to prevent distortion:

let mixed = (mic_sample * self.mic_ducking) + (system_sample * system_gain);
result.push(mixed.clamp(-1.0, 1.0));

The clamp operation ensures values remain within the valid -1.0 to +1.0 range. Additionally, a soft-scaling step normalizes any over-range sums proportionally to maintain audio quality.

Implementation in pipeline.rs

The ProfessionalAudioMixer in frontend/src-tauri/src/audio/pipeline.rs provides a straightforward implementation suitable for standard recording scenarios. The core mixing logic appears in the mix method:

fn mix(&mut self, mic: &[f32], system: &[f32]) -> Vec<f32> {
    let mic_rms = calculate_rms(mic);
    let is_speech = mic_rms > 0.01;               // speech threshold
    let system_gain = if is_speech { self.system_ducking } else { 1.0 };

    mic.iter()
        .zip(system.iter())
        .map(|(&m, &s)| (m * self.mic_ducking + s * system_gain).clamp(-1.0, 1.0))
        .collect()
}

The AudioPipeline orchestrates this process through an async run loop that continues until the sender channel closes, guaranteeing that all buffered audio flushes before shutdown.

Implementation in ffmpeg_mixer.rs

For scenarios requiring advanced handling—such as Bluetooth audio devices with variable latency—the FFmpegAudioMixer in frontend/src-tauri/src/audio/ffmpeg_mixer.rs provides the same RMS-based ducking with additional buffering strategies.

You can use this mixer directly for custom tooling:

use meetily::frontend::src_tauri::src::audio::ffmpeg_mixer::FFmpegAudioMixer;
use meetily::frontend::src_tauri::src::audio::device_detection::InputDeviceKind;

let mut mixer = FFmpegAudioMixer::new(
    "Built‑in Microphone".into(),
    InputDeviceKind::Wired,
    "BlackHole".into(),
    InputDeviceKind::Wired,
    48_000,
);

// Push raw chunks as they arrive
mixer.push_mic(mic_chunk);
mixer.push_system(system_chunk);

// When both buffers are ready, obtain a mixed window
if let Some(mixed) = mixer.pop_mixed() {
    // `mixed` is a 50 ms slice already ducked according to RMS speech detection
    send_to_transcriber(mixed);
}

Integration with the Recording Pipeline

The AudioPipeline manages the complete flow from capture to output. It receives raw AudioChunk structures from the capture layer (implemented in frontend/src-tauri/src/audio/capture/microphone.rs and system.rs), feeds them to the ring-buffer, extracts aligned windows, and mixes them using the ducking algorithm.

The mixed output serves two downstream consumers simultaneously:

1. Transcription Pipeline – The mixed audio flows to the Voice Activity Detection (VAD) system in frontend/src-tauri/src/audio/vad.rs, which identifies speech segments and forwards them to the Whisper transcription engine.
2. Recording Manager – The final WAV file writer receives mixed audio via recording_sender_for_mixed, ensuring the saved recording contains the professionally ducked audio.

The pipeline initialization, typically triggered by the Tauri command start_recording, sets up these components:

let mut manager = AudioPipelineManager::default();
manager.start(
    state.clone(),
    transcription_sender,
    /* target_chunk_duration_ms */ 0,
    /* sample_rate */ 48000,
    Some(recording_sender),          // receive mixed audio for WAV files
    mic_device_name,
    mic_device_kind,
    system_device_name,
    system_device_kind,
)?;

Summary

• Synchronization: AudioMixerRingBuffer aligns microphone and system audio into 50 ms windows using a lock-free VecDeque, padding gaps with silence and enforcing a 400 ms safety limit.
• Speech Detection: The system calculates RMS values using calculate_rms() and compares them against a 0.01 threshold to identify active speech.
• Automatic Ducking: When speech is detected, system audio attenuates to 0.60 gain (40% reduction); otherwise, it plays at full volume.
• Clipping Prevention: Mixed samples are clamped to the -1.0 to +1.0 range and soft-scaled to prevent distortion.
• Dual Implementation: Both ProfessionalAudioMixer in pipeline.rs and FFmpegAudioMixer in ffmpeg_mixer.rs implement this algorithm, with the latter offering additional Bluetooth gap-handling.

Frequently Asked Questions

What is RMS-based ducking in Meetily?

RMS-based ducking is an audio processing technique where Meetily calculates the Root-Mean-Square energy of the microphone input to detect when someone is speaking. When the RMS level exceeds the threshold of 0.01, the system automatically reduces the volume of system audio (such as music or notifications) by 40% (to 0.60 gain) to ensure the microphone voice remains clear and prominent in the recording.

How does Meetily prevent audio desynchronization between microphone and system sources?

Meetily uses the AudioMixerRingBuffer structure in frontend/src-tauri/src/audio/pipeline.rs, which maintains separate buffers for each audio source. It holds samples in a lock-free VecDeque until both streams have enough data for a complete 50 ms window. If one source lags, the buffer zero-pads the missing data to maintain alignment, and it drops samples older than 400 ms to prevent memory overflow.

What is the difference between ProfessionalAudioMixer and FFmpegAudioMixer?

ProfessionalAudioMixer in pipeline.rs provides a straightforward implementation suitable for standard wired recording setups. FFmpegAudioMixer in ffmpeg_mixer.rs offers a more robust implementation with separate per-source buffers and additional gap-handling logic specifically designed to accommodate Bluetooth devices that may introduce variable latency or brief dropouts. Both implement identical RMS-based ducking algorithms.

How does the system prevent clipping when mixing two audio sources?

The mixer applies a clamp(-1.0, 1.0) operation to every mixed sample, ensuring the final output never exceeds the valid floating-point audio range. Additionally, ProfessionalAudioMixer applies soft-scaling to normalize any sums that exceed the valid range proportionally, preventing hard digital clipping that would cause distortion in the final WAV recording.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}