How Meetily Mixes Microphone and System Audio with RMS-Based Ducking

Meetily combines microphone and system audio streams by calculating the Root Mean Square (RMS) energy of the microphone input every 50 milliseconds, automatically attenuating (ducking) the system audio by 60% whenever speech is detected above a threshold of 0.01, ensuring voice clarity while maintaining background audio presence.

Meetily is an open-source meeting assistant from Zackriya‑Solutions that captures dual audio streams to record both participant voices and system sounds. The core mixing logic resides in frontend/src-tauri/src/audio/ffmpeg_mixer.rs, where an adaptive RMS analyzer continuously monitors microphone energy to drive real-time ducking decisions.

High-Level Pipeline Architecture

The audio pipeline processes two parallel SourceBuffer instances—mic_buffer and system_buffer—that receive raw 48 kHz PCM chunks from their respective capture modules. Every ~50 ms (mixing_window_samples), the mixer extracts equal-sized slices from each buffer and applies adaptive gain control before summing the streams.

The workflow follows six discrete stages:

1. Capture – Raw PCM chunks enter the microphone and system buffers.
2. Buffering – Adaptive timeouts account for device latency (wired vs. Bluetooth).
3. Windowing – Fixed 50 ms slices are popped from both buffers.
4. RMS Analysis – Energy calculation determines if the user is speaking.
5. Adaptive Ducking – System audio attenuates when speech is detected.
6. Clipping Protection – Final samples clamp to [-1.0, 1.0] to prevent distortion.

RMS Calculation for Speech Detection

The RMS (Root Mean Square) value provides a perceptual measure of audio energy. Meetily implements this calculation in a dedicated helper function that iterates over sample slices:

/// Calculate RMS (Root Mean Square) for audio samples
fn calculate_rms(samples: &[f32]) -> f32 {
    if samples.is_empty() {
        return 0.0;
    }
    let sum_squares: f32 = samples.iter().map(|s| s * s).sum();
    (sum_squares / samples.len() as f32).sqrt()
}

Source: frontend/src-tauri/src/audio/ffmpeg_mixer.rs, lines 48‑56

This value represents the effective power of the microphone signal, enabling the mixer to distinguish between silence and active speech.

Adaptive Ducking Logic

The mix method in the AudioMixer struct implements the ducking decision logic. It compares the microphone RMS against a hard-coded SPEECH_THRESHOLD of 0.01. When the threshold is exceeded, the system audio gain drops to the system_ducking factor (default 0.60); otherwise, it remains at full volume (1.0).

fn mix(&mut self, mic: &[f32], system: &[f32]) -> Vec<f32> {
    assert_eq!(mic.len(), system.len(), "Mic and system audio must have same length");

    let mic_rms = calculate_rms(mic);
    const SPEECH_THRESHOLD: f32 = 0.01;
    let is_speech = mic_rms > SPEECH_THRESHOLD;

    // Adjust system gain based on speech detection
    let system_gain = if is_speech {
        self.system_ducking          // duck system audio
    } else {
        1.0                          // full system audio
    };

    // Mix with ducking and clamp to avoid clipping
    mic.iter()
        .zip(system.iter())
        .map(|(m, s)| (m * self.mic_ducking + s * system_gain).clamp(-1.0, 1.0))
        .collect()
}

Source: frontend/src-tauri/src/audio/ffmpeg_mixer.rs, lines 76‑100

The mixing formula mic * mic_ducking + system * system_gain preserves voice priority while preventing hard cuts to background audio, creating a balanced output stream.

The FFmpegAudioMixer Implementation

The FFmpegAudioMixer struct provides the public API that orchestrates buffer management and mixing. Its constructor initializes the dual SourceBuffers and calculates the mixing window size based on the sample rate:

pub fn new(
    mic_device_name: String,
    mic_device_kind: InputDeviceKind,
    system_device_name: String,
    system_device_kind: InputDeviceKind,
    sample_rate: u32,
) -> Self {
    let mixing_window_samples = ((sample_rate as f64 * 0.050) as usize).max(1);
    Self {
        mic_buffer: SourceBuffer::new(mic_device_name, mic_device_kind, sample_rate),
        system_buffer: SourceBuffer::new(system_device_name, system_device_kind, sample_rate),
        mixer: AudioMixer::new(true),                // enable adaptive (RMS) ducking
        sample_rate,
        mixing_window_samples,
        windows_mixed: 0,
    }
}

Source: frontend/src-tauri/src/audio/ffmpeg_mixer.rs, lines 40‑75

When pop_mixed() is called, the mixer extracts the 50 ms window from each buffer, invokes AudioMixer::mix(), and returns the processed frame ready for transcription or recording.

Usage Example

use meetily::audio::ffmpeg_mixer::FFmpegAudioMixer;
use meetily::audio::devices::InputDeviceKind;

// Create the mixer with adaptive ducking enabled
let mut mixer = FFmpegAudioMixer::new(
    "Built‑in Mic".into(),
    InputDeviceKind::Wired,
    "BlackHole".into(),
    InputDeviceKind::Wired,
    48_000,
);

// Push incoming chunks from capture threads
mixer.push_mic(mic_chunk);
mixer.push_system(sys_chunk);

// Retrieve mixed frame when both buffers are ready
if let Some(mixed) = mixer.pop_mixed() {
    // mixed is Vec<f32> of 50ms ducked audio
    // → feed to Whisper, write to file, or stream to UI
}

Professional Mixer Alternative

Meetily also provides an advanced mixer in frontend/src-tauri/src/audio_v2/mixer.rs for professional use cases. This implementation replaces the simple threshold-based ducking with a dynamic processing chain featuring:

• RmsAnalyzer – Continuous RMS calculation with configurable window sizes.
• DuckingProcessor – Smooth gain reduction with attack and release envelopes.
• CrossfadeProcessor – Seamless transitions between ducked and full-volume states.

In Professional mode, each sample passes through ducking_processor.process() followed by crossfade_processor.process() before summing, eliminating abrupt volume changes that can occur with the threshold-based approach.

Source: frontend/src-tauri/src/audio_v2/mixer.rs, lines 9‑107

Summary

• Meetily captures microphone and system audio into separate SourceBuffer instances with adaptive timeouts.
• The RMS calculator evaluates microphone energy every 50 ms to detect speech.
• When the RMS exceeds 0.01, the mixer applies 0.60 gain to system audio, ducking it below the voice track.
• The final mix clamps to [-1.0, 1.0] to prevent digital clipping.
• An alternative Professional mixer in audio_v2/ offers dynamic ducking with smoothed transitions for production environments.

Frequently Asked Questions

What is the default speech threshold for ducking in Meetily?

The default SPEECH_THRESHOLD is hard-coded to 0.01 in ffmpeg_mixer.rs. When the microphone RMS value exceeds this threshold, the mixer identifies active speech and triggers the ducking algorithm to attenuate system audio.

How does Meetily prevent audio clipping when mixing two sources?

The mixer applies hardware-clamping protection using Rust's .clamp(-1.0, 1.0) method on every final sample. This ensures that the summed values of mic * mic_ducking + system * system_gain never exceed the valid float range, preventing digital distortion in the output stream.

Can the ducking attenuation level be customized?

Yes. The system_ducking parameter in the AudioMixer struct defaults to 0.60 (60% volume reduction), but this value can be adjusted during initialization. Lower values create more aggressive ducking, while values closer to 1.0 preserve more background audio during speech.

What is the difference between FFmpegAudioMixer and the audio_v2 mixer?

The FFmpegAudioMixer provides lightweight, threshold-based ducking suitable for general use with minimal CPU overhead. The audio_v2 mixer offers a professional-grade alternative with continuous RMS analysis, gain smoothing envelopes, and crossfade processing for broadcast-quality transitions between speech and background audio states.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}