How Meetily's FFmpeg Mixer Performs Professional Audio Ducking: RMS-Based Speech Detection Explained

Meetily implements professional audio ducking in its FFmpeg-style mixer by calculating RMS (Root Mean Square) values from microphone input and automatically attenuating system audio to 60% when speech exceeds a 0.01 threshold, ensuring clear voice recording while preserving background audio.

Meetily is an open-source meeting assistant that leverages Rust-based audio processing to deliver broadcast-quality recordings. The application uses an adaptive FFmpeg-style mixer that implements professional-grade audio ducking to balance microphone speech against system audio in real-time. This system, implemented in frontend/src-tauri/src/audio/ffmpeg_mixer.rs, ensures that spoken commentary remains intelligible without completely muting background audio during pauses.

Understanding Meetily's Adaptive Audio Architecture

Meetily's audio processing relies on a dual-buffer architecture that keeps microphone and system audio streams synchronized before applying intelligent gain control.

Per-Source Buffering with Device-Specific Timeouts

Each audio source (microphone and system) maintains its own SourceBuffer (lines 78-101 in ffmpeg_mixer.rs). These buffers handle device-specific timing differences between wired and Bluetooth inputs, automatically inserting silence when buffer under-runs occur. This synchronization ensures that both streams remain in lock-step before the mixing stage, preventing audio drift that could destabilize the ducking calculation.

RMS Calculation for Reliable Speech Detection

The mixer uses a helper function calculate_rms(samples: &[f32]) -> f32 (lines 48-56) to compute the root-mean-square of incoming microphone chunks. Unlike peak detection, RMS provides a reliable speech-activity metric that is insensitive to occasional audio spikes, making it ideal for detecting sustained speech versus transient noise.

The Ducking Algorithm Implementation

The core ducking logic lives inside AudioMixer::mix (lines 76-105), where the mixer continuously evaluates microphone activity and adjusts system audio gain accordingly.

Speech Threshold Detection

For each mixing iteration, the mixer calculates mic_rms (line 89) and compares it against a defined SPEECH_THRESHOLD of 0.01 (line 92). When mic_rms exceeds this threshold, the system identifies active speech and triggers the ducking mechanism.

Dynamic Gain Adjustment

When speech is detected, the system audio gain is set to self.system_ducking (default value 0.60, lines 70-71), effectively attenuating system audio to 40% of its original level. When the microphone falls silent, the system gain returns to 1.0, allowing background audio to play at full volume. This creates a smooth, professional ducking effect that prioritizes voice clarity without complete silence substitution.

Sample Mixing and Clipping Prevention

The final mixed output is computed using the formula (mic * mic_ducking) + (system * system_gain) (line 105). The result is immediately clamped to the [-1.0, 1.0] range (line 107) to prevent digital clipping, ensuring clean audio output even when both sources contain high-amplitude signals.

Enabling and Configuring the Ducking Behavior

The mixer supports both adaptive ducking and simple summing modes through a boolean flag in the constructor.

Initializing the Adaptive Mixer

To enable professional ducking, instantiate the mixer with the adaptive flag set to true:

// Create the adaptive mixer (adaptive ducking enabled)
let mut mixer = FFmpegAudioMixer::new(
    "Built-in Mic".to_string(),
    InputDeviceKind::Wired,
    "BlackHole 2ch".to_string(),
    InputDeviceKind::Bluetooth,
    48_000,
);

Processing Audio Streams

Push incoming audio chunks from both sources and retrieve mixed output:

// Push incoming audio chunks
mixer.push_mic(mic_chunk);
mixer.push_system(system_chunk);

// When both buffers are ready, retrieve a mixed 50ms window
if let Some(mixed) = mixer.pop_mixed() {
    // `mixed` already has RMS-based ducking applied
    // Send it to the recorder, playback, or Whisper engine
}

Setting the adaptive flag to false in AudioMixer::new(false) (line 70) falls back to a simple sum-mix without ducking (lines 110-115), useful for debugging or when ducking is not desired.

Integration with the Audio Pipeline

The ducking system integrates with several components across Meetily's codebase:

• frontend/src-tauri/src/audio/device_detection.rs: Defines InputDeviceKind and per-device timeout values used by the SourceBuffer instances
• frontend/src-tauri/src/audio/pipeline.rs: Orchestrates the audio pipeline and invokes FFmpegAudioMixer during recording sessions
• frontend/src-tauri/src/audio/recording_manager.rs: Manages the overall recording flow, feeding raw microphone and system chunks into the mixer

This architecture ensures that ducking decisions happen at the lowest level of audio processing, maintaining low latency and high fidelity throughout the recording chain.

Summary

• Meetily uses RMS-based ducking in ffmpeg_mixer.rs to detect speech activity through calculate_rms() and threshold comparison against SPEECH_THRESHOLD = 0.01
• System audio automatically attenuates to 60% when microphone RMS exceeds the threshold, preventing voice masking while preserving background presence
• Per-source SourceBuffer instances with device-specific timeouts keep streams synchronized before mixing, handling both wired and Bluetooth latencies
• The mixer clamps output to [-1.0, 1.0] to prevent digital clipping while maintaining professional audio quality
• Adaptive ducking can be toggled via AudioMixer::new(true) versus simple sum-mixing with false

Frequently Asked Questions

What is the default ducking level in Meetily's FFmpeg mixer?

The default system audio ducking level is set to 0.60 (60% of original volume), meaning system audio attenuates to 40% below its original level when speech is detected. This value is configurable in the AudioMixer struct initialization at lines 70-71 of ffmpeg_mixer.rs.

How does Meetily distinguish between speech and background noise?

Meetily uses RMS (Root Mean Square) calculation rather than peak detection to measure microphone energy. The calculate_rms() function processes sample chunks to produce an average energy level, which is then compared against a SPEECH_THRESHOLD of 0.01. This approach filters out transient spikes and identifies sustained speech patterns characteristic of human voice.

Can I disable audio ducking in Meetily?

Yes, you can disable ducking by initializing the mixer with AudioMixer::new(false) instead of true. This bypasses the RMS calculation and threshold logic, falling back to simple sum-mixing at lines 110-115 where both sources are added together without gain reduction.

Why does Meetily use separate buffers for microphone and system audio?

Separate SourceBuffer instances handle different device latencies (wired vs. Bluetooth) and compensate for clock drift between audio sources. Located at lines 78-101, these buffers insert silence during under-runs to maintain synchronization, ensuring that ducking decisions accurately align with the corresponding audio timestamps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}