How Meetily Mixes Microphone and System Audio with RMS-Based Ducking: 6‑Step Pipeline Explained

Meetily combines microphone and system audio by calculating the RMS energy of the voice stream every 50 ms and dynamically attenuating system audio when speech exceeds a 0.01 threshold, preventing background noise from overwhelming conversations while keeping both sources synchronized.

Meetily is an open-source meeting assistant that blends user voice with system audio (like shared video sound) into a single clean track for transcription and recording. According to the Meetily source code, the application implements an adaptive FFmpeg-style audio mixer in frontend/src-tauri/src/audio/ffmpeg_mixer.rs that uses root-mean-square (RMS) analysis to detect speech and duck background audio in real time.

High-Level Audio Pipeline

The mixing process orchestrates six distinct stages to maintain sync between sources while applying dynamic attenuation.

1. Capture – The mic_buffer and system_buffer (SourceBuffer instances) receive raw 48 kHz PCM chunks from the microphone and system capture modules.
2. Buffering – Each buffer employs adaptive timeouts accounting for device types (wired versus Bluetooth) to keep streams aligned.
3. Mixing window – Every ~50 ms (mixing_window_samples), the mixer extracts an equal-sized slice from each buffer.
4. RMS analysis – Before summing, the mixer evaluates the energy of the microphone slice to detect speech.
5. Adaptive gain – If speech is detected, system audio is attenuated by a configurable system_ducking factor (default 0.60).
6. Clipping protection – Final samples are clamped to the range [-1.0, 1.0] to prevent distortion.

RMS Calculation for Speech Detection

The RMS value provides a perceptual measure of audio energy. In frontend/src-tauri/src/audio/ffmpeg_mixer.rs (lines 48‑56), Meetily computes this with a helper function that iterates over sample slices:

/// Calculate RMS (Root Mean Square) for audio samples
fn calculate_rms(samples: &[f32]) -> f32 {
    if samples.is_empty() {
        return 0.0;
    }
    let sum_squares: f32 = samples.iter().map(|s| s * s).sum();
    (sum_squares / samples.len() as f32).sqrt()
}

This function returns the square root of the mean squared samples, yielding a float between 0.0 and 1.0 for normalized audio.

Adaptive Ducking Logic in AudioMixer::mix

The core ducking logic resides in the mix method (lines 76‑100 of ffmpeg_mixer.rs). The implementation compares microphone RMS against a hard-coded SPEECH_THRESHOLD of 0.01.

fn mix(&mut self, mic: &[f32], system: &[f32]) -> Vec<f32> {
    assert_eq!(mic.len(), system.len(), "Mic and system audio must have same length");

    let mic_rms = calculate_rms(mic);
    const SPEECH_THRESHOLD: f32 = 0.01;
    let is_speech = mic_rms > SPEECH_THRESHOLD;

    // Adjust system gain based on speech detection
    let system_gain = if is_speech {
        self.system_ducking          // duck system audio
    } else {
        1.0                          // full system audio
    };

    // Mix with ducking and clamp to avoid clipping
    mic.iter()
        .zip(system.iter())
        .map(|(m, s)| (m * self.mic_ducking + s * system_gain).clamp(-1.0, 1.0))
        .collect()
}

When the microphone RMS exceeds 0.01, the system_gain drops to the configured ducking level (typically 0.60), otherwise it remains at unity gain (1.0). The final mix applies separate attenuation factors for both channels (mic_ducking and system_ducking) before clamping to prevent digital clipping.

The FFmpegAudioMixer Implementation

The public API that orchestrates the pipeline is FFmpegAudioMixer. It instantiates dual SourceBuffer instances and processes audio in discrete 50 ms windows.

When constructing the mixer (lines 40‑75), Meetily calculates the window size based on the sample rate:

pub fn new(
    mic_device_name: String,
    mic_device_kind: InputDeviceKind,
    system_device_name: String,
    system_device_kind: InputDeviceKind,
    sample_rate: u32,
) -> Self {
    let mixing_window_samples = ((sample_rate as f64 * 0.050) as usize).max(1);
    Self {
        mic_buffer: SourceBuffer::new(mic_device_name, mic_device_kind, sample_rate),
        system_buffer: SourceBuffer::new(system_device_name, system_device_kind, sample_rate),
        mixer: AudioMixer::new(true),                // enable adaptive (RMS) ducking
        sample_rate,
        mixing_window_samples,
        windows_mixed: 0,
    }
}

Calling pop_mixed() triggers the extraction of synchronized slices from both buffers, delegates to AudioMixer::mix for RMS-based ducking, and returns the processed frame ready for encoding or transcription.

Alternative Professional Mixer (audio_v2)

Meetily also ships a more sophisticated mixer in frontend/src-tauri/src/audio_v2/mixer.rs for professional use cases. This implementation uses an RmsAnalyzer for continuous energy calculation, a DuckingProcessor for smooth gain transitions, and a CrossfadeProcessor for seamless blending.

In this variant, each sample passes through:

• RMS analysis via self.rms_analyzer.analyze(mic)
• Dynamic ducking via self.ducking_processor.process(mic_sample, system_sample)
• Cross-fade mixing via self.crossfade_processor.process(ducked_mic, system_sample)

This provides smoother transitions than the threshold-based binary ducking used in the FFmpeg mixer.

Summary

• Meetily captures microphone and system audio into separate SourceBuffer instances with adaptive timing.
• Every 50 ms, the mixer calculates RMS energy to detect speech against a 0.01 threshold.
• When speech is detected, system audio is attenuated by a system_ducking factor (default 0.60).
• The final mix clamps samples to [-1.0, 1.0] to prevent distortion.
• An alternative audio_v2 implementation provides professional-grade ducking with smoothing and cross-fade.

Frequently Asked Questions

How does Meetily prevent audio clipping when mixing two sources?

Meetily clamps every mixed sample to the range [-1.0, 1.0] using the .clamp(-1.0, 1.0) method inside the mix function. This hard limit prevents digital distortion that would occur if the summed microphone and system audio exceeded unity gain.

What sample rate does Meetily use for RMS-based ducking?

The mixer operates at 48 kHz by default, processing audio in 50 ms windows calculated as sample_rate * 0.050. You can verify this in the FFmpegAudioMixer::new constructor where mixing_window_samples is derived from the sample rate parameter.

Can I adjust the speech detection sensitivity in Meetily?

The SPEECH_THRESHOLD constant is currently hard-coded to 0.01 in the mix method of frontend/src-tauri/src/audio/ffmpeg_mixer.rs. To modify sensitivity, you would need to adjust this constant or modify the AudioMixer struct to accept a configurable threshold value at initialization.

How does the professional mixer differ from the FFmpeg mixer?

As implemented in frontend/src-tauri/src/audio_v2/mixer.rs, the professional mixer replaces the binary threshold logic with continuous RMS analysis and applies gain smoothing through DuckingProcessor, whereas the FFmpeg mixer uses immediate attenuation switches based on the 0.01 threshold.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}