How to Implement Custom Audio Preprocessing Before Transcription in Meetily: A Complete Guide

You can inject custom audio preprocessing in Meetily by implementing an AudioProcessor trait and inserting it into the AudioCapture::process_audio_data method after the built-in high-pass filter and RNNoise stages, or alternatively before the VAD step in AudioPipeline::run for processing mixed audio streams.

Meetily is an open-source meeting transcription tool built by Zackriya-Solutions that processes audio through a Rust-based Tauri pipeline before sending it to Whisper. The codebase provides two strategic hook points in frontend/src-tauri/src/audio/pipeline.rs where you can insert proprietary denoisers, spectral enhancers, or band-stop filters without disrupting the existing capture → mix → VAD flow.

Understanding the Audio Pipeline Architecture

Meetily’s audio flow follows a strict capture → pre-process → mix → VAD → Whisper pipeline implemented in Rust. Understanding these stages helps you choose the right insertion point for your custom logic.

• Capture: AudioCapture::process_audio_data (lines 85-115) reads raw PCM, converts to mono, and optionally resamples to 48 kHz.
• Built-in Preprocessing: Inside process_audio_data (lines 111-173), the pipeline applies an 80 Hz high-pass filter, optional RNNoise suppression, and EBU-R128 loudness normalization to -23 LUFS.
• Mixing: AudioMixerRingBuffer and ProfessionalAudioMixer (lines 16-90) align microphone and system audio in 50 ms windows.
• VAD: ContinuousVadProcessor (lines 78-90) segments speech frames before forwarding to Whisper at 16 kHz.

You have two natural hook points for custom audio preprocessing: after capture/before mixing (microphone only) or after mixing/before VAD (combined signal).

Hook Point 1: Microphone Stream Processing (Pre-Mix)

Inserting your processor here affects only the microphone stream after the built-in high-pass filter, RNNoise, and loudness normalization. This is ideal for proprietary denoisers or voice enhancement algorithms that should process only the local user's voice.

Define the Processor Trait

Create a new file at frontend/src-tauri/src/audio/custom_processor.rs that defines the contract for all custom processors:

// src/audio/custom_processor.rs
pub trait AudioProcessor: Send + Sync {
    /// Receives a mono PCM vector and returns a processed vector.
    fn process(&mut self, data: Vec<f32>) -> Vec<f32>;
}

Integrate with AudioCapture

Modify frontend/src-tauri/src/audio/pipeline.rs to add a field inside the AudioCapture struct:

// Inside AudioCapture struct (add after the normalizer field)
custom_processor: Arc<std::sync::Mutex<Option<Box<dyn AudioProcessor>>>>,

Wire the initialization in AudioCapture::new:

// Inside AudioCapture::new ...
let custom_processor: Option<Box<dyn AudioProcessor>> = {
    #[cfg(feature = "my_custom")]
    {
        use crate::audio::custom_processor::MyLimiter;
        Some(Box::new(MyLimiter::new()))
    }
    #[cfg(not(feature = "my_custom"))] 
    { None }
};

Self {
    // ... other fields ...
    custom_processor: Arc::new(std::sync::Mutex::new(custom_processor)),
    // ...
}

Apply the Processor in process_audio_data

Insert the processing call after the existing microphone pipeline (around line 158-173) and before the chunk is sent to RecordingState:

// After normalisation (around line 165 in the original file)
if matches!(self.device_type, DeviceType::Microphone) {
    // ... existing high-pass / RNNoise / normaliser code ...
}

// ----- NEW CUSTOM HOOK -----
if let Ok(mut guard) = self.custom_processor.lock() {
    if let Some(ref mut processor) = *guard {
        mono_data = processor.process(mono_data);
    }
}

Implement a Notch Filter Example

Here is a complete example of a custom processor that removes electrical hum at 60 Hz:

// src/audio/custom_processor.rs
pub struct NotchFilter {
    sample_rate: u32,
    freq_hz: f32,
    q_factor: f32,
    z1: f32,
    z2: f32,
}

impl NotchFilter {
    pub fn new(sample_rate: u32, freq_hz: f32, q_factor: f32) -> Self {
        Self {
            sample_rate,
            freq_hz,
            q_factor,
            z1: 0.0,
            z2: 0.0,
        }
    }
}

impl AudioProcessor for NotchFilter {
    fn process(&mut self, data: Vec<f32>) -> Vec<f32> {
        // Biquad implementation would go here
        // Update z1, z2 state and return filtered samples
        data
    }
}

Hook Point 2: Mixed Stream Processing (Pre-VAD)

If your algorithm needs to process the combined microphone and system audio (for example, removing a known interference tone from the entire meeting), inject your processor just before the VAD call in AudioPipeline::run (around lines 124-140):

// Inside the mixing loop, after let mixed_with_gain = mixed_clean;
if let Some(ref proc) = self.custom_post_processor {
    let processed = proc.process(mixed_with_gain.clone());
    // feed processed to VAD instead of mixed_with_gain
    self.vad_processor.process_audio(&processed)?;
}

Create custom_post_processor as an analogous field to the microphone hook, following the same Arc<Mutex<Option<Box<dyn AudioProcessor>>>> pattern.

Exposing Runtime Controls via Tauri Commands

Enable or disable your custom processor at runtime without rebuilding the application by exposing a Tauri command:

#[tauri::command]
fn enable_custom_processor(
    state: tauri::State<'_, AudioCapture>, 
    enable: bool
) -> Result<(), String> {
    let mut guard = state.custom_processor.lock().map_err(|e| e.to_string())?;
    if enable {
        *guard = Some(Box::new(NotchFilter::new(48000, 60.0, 30.0)));
    } else {
        *guard = None;
    }
    Ok(())
}

Register this command in frontend/src-tauri/src/lib.rs:

tauri::generate_handler![..., enable_custom_processor]

Key Source Files

File	Purpose	Location
pipeline.rs	Core capture, mixing, VAD, and pipeline orchestration	frontend/src-tauri/src/audio/pipeline.rs
audio_processing.rs	Helper functions including LoudnessNormalizer, NoiseSuppressionProcessor, and HighPassFilter	frontend/src-tauri/src/audio/audio_processing.rs
recording_state.rs	Shared state and channel plumbing	frontend/src-tauri/src/audio/recording_state.rs
lib.rs	Tauri command registration	frontend/src-tauri/src/lib.rs

Summary

• Meetily processes audio through a strict pipeline in frontend/src-tauri/src/audio/pipeline.rs with built-in high-pass filtering, RNNoise, and loudness normalization.
• Hook Point 1 (after line 165 in process_audio_data) allows processing the microphone stream after built-in steps but before mixing.
• Hook Point 2 (before VAD in AudioPipeline::run) allows processing the combined microphone and system audio.
• Implement the AudioProcessor trait to encapsulate your custom DSP logic and store it as an Arc<Mutex<Option<Box<dyn AudioProcessor>>>> in the appropriate struct.
• Expose Tauri commands to toggle processors at runtime, giving end-users control without recompiling the application.

Frequently Asked Questions

Can I replace the built-in RNNoise or high-pass filter instead of adding to them?

No, you should not modify the existing built-in pipeline directly if you want to maintain compatibility with upstream updates. Instead, disable RNNoise via configuration flags if available, then implement your replacement as a custom processor using the hooks described above. This keeps your changes isolated in separate files while preserving the original signal chain.

What sample rate should my custom processor expect?

Your custom processor should expect 48 kHz mono PCM at Hook Point 1 (microphone capture) and 48 kHz at Hook Point 2 (mixed audio), as the pipeline resamples input streams to 48 kHz before mixing and downsamples to 16 kHz only immediately before sending to Whisper. Always design your filters and algorithms for the 48 kHz sample rate to ensure consistency.

How do I debug my custom processor if transcription accuracy degrades?

Write processed audio chunks to disk using a debug_write_to_file flag in your processor implementation, then compare the waveform before and after your processing step. You can also temporarily bypass the VAD stage and route your processed audio directly to a test file to verify the output quality before it reaches the Whisper engine.

Can I apply different processing to microphone versus system audio?

Yes, but only at Hook Point 1. The AudioCapture struct distinguishes between DeviceType::Microphone and DeviceType::System through the device_type field. Check this variant inside your processor logic to apply different algorithms, or instantiate two separate AudioCapture instances with different custom processors if you need completely separate signal chains.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}