How to Implement System Audio Capture on a New Platform in Meetily (WASAPI, ScreenCaptureKit)

To add system audio capture for a new platform in Meetily, create a platform-specific module implementing the PlatformAudio trait with list_audio_devices() and open_system_capture(), register the implementation in the platform registry, and connect the capture stream to the AudioPipelineManager via async channels.

Meetily’s Rust-based audio subsystem uses a platform-agnostic abstraction layer to handle system audio capture across Windows, macOS, and Linux. This article provides a complete technical guide for extending the framework to support new backends like WASAPI loopback on Windows or ScreenCaptureKit on macOS, following the exact patterns established in the Zackriya-Solutions/meetily repository.

Architecture Overview

Meetily organizes audio functionality into three distinct layers that isolate platform specifics from business logic.

• Device Discovery: Enumerates available input and output devices, returning standardized AudioDeviceInfo structs that include display names, unique identifiers, and DeviceType classifications (Microphone or System).
• Capture Streams: Opens low-latency audio streams from selected devices. Each platform implementation spawns an async task that reads PCM buffers and forwards them via mpsc::UnboundedSender<AudioChunk> channels.
• Pipeline: The AudioPipelineManager consumes these streams, applies voice-activity detection (VAD), mixes microphone and system audio, and forwards speech-only chunks to the Whisper transcription engine.

To implement system audio capture on a new platform, you must extend only the first two layers while keeping the pipeline layer untouched.

Step-by-Step Implementation Guide

Create the Platform-Specific Module

Create a new Rust file at frontend/src-tauri/src/audio/devices/platform/<platform>.rs. This file must implement two core functions that satisfy the PlatformAudio trait:

• list_audio_devices() -> Result<Vec<AudioDeviceInfo>>: Enumerates all system audio endpoints capable of 48 kHz, 32-bit float, stereo capture.
• open_system_capture(device_name: &str) -> Result<SystemCapture>: Opens the specified device and returns a capture handle containing the sender end of an async channel.

In frontend/src-tauri/src/audio/devices/platform/mod.rs, add pub mod <platform>; to expose the new module to the rest of the crate.

Register the Implementation in the Platform Registry

Modify frontend/src-tauri/src/audio/devices/platform/mod.rs to include the platform selection logic. Use conditional compilation attributes to instantiate the correct implementation at compile time:

pub fn get_platform_impl() -> Box<dyn PlatformAudio> {
    #[cfg(target_os = "windows")]
    { Box::new(windows::WindowsPlatform) }

    #[cfg(target_os = "macos")]
    { Box::new(macos::MacosPlatform) }

    #[cfg(target_os = "linux")]
    { Box::new(linux::LinuxPlatform) }

    #[cfg(target_os = "your_new_os")]
    { Box::new(yourplatform::YourPlatform) }
}

This registry ensures that higher-level code remains platform-agnostic while the compiler selects the appropriate backend for the target OS.

Implement the Capture Driver (WASAPI or ScreenCaptureKit)

For Windows WASAPI, implement loopback capture by opening the default audio endpoint in loopback mode, configuring the stream for 48 kHz stereo 32-bit float, and spawning a Tokio task that continuously reads buffers:

// frontend/src-tauri/src/audio/devices/platform/windows.rs (template)
pub async fn open_system_capture(device_name: &str) -> Result<SystemCapture> {
    let format = WaveFormat::new(48000, 2, 32); // 48kHz, stereo, f32
    let client = CaptureClient::new(device_name, format, true)?; // true = loopback
    
    let (tx, rx) = mpsc::unbounded_channel();
    
    tokio::spawn(async move {
        while let Ok(buffer) = client.get_next_buffer().await {
            let chunk = AudioChunk::new(buffer.samples, chrono::Utc::now());
            if tx.send(chunk).is_err() { break; }
        }
    });
    
    Ok(SystemCapture { sender: tx, receiver: rx })
}

For macOS ScreenCaptureKit, create a SCStream configured for audio-only capture, set the sample rate to 48 kHz, and implement the SCStreamOutput delegate to forward CMSampleBuffer data converted to f32 PCM into the same channel structure.

Expose the Public API Wrapper

In frontend/src-tauri/src/audio/devices/mod.rs, create a thin wrapper that delegates to the platform registry:

pub async fn start_system_capture(device_name: &str) -> Result<SystemCapture> {
    let platform = get_platform_impl();
    platform.open_system_capture(device_name).await
}

This abstraction prevents the rest of the application from directly depending on platform-specific types.

Connect to the Audio Pipeline

In frontend/src-tauri/src/audio/recording_commands.rs, obtain the capture stream and pass its sender to the pipeline manager:

pub async fn start_recording(
    mic_name: Option<String>,
    system_name: Option<String>,
) -> Result<(), String> {
    let mic = devices::open_microphone(mic_name).await?;
    let sys = devices::start_system_capture(system_name.unwrap_or_default()).await?;
    
    let pipeline = AudioPipelineManager::new(mic.sender, sys.sender);
    pipeline.start().await?;
    Ok(())
}

The AudioPipelineManager defined in frontend/src-tauri/src/audio/pipeline.rs expects UnboundedSender<AudioChunk> channels for both microphone and system inputs. No changes are required inside the pipeline logic itself.

Update the Tauri Command Interface

If the UI must allow users to select the new platform’s devices, extend the command signature in frontend/src-tauri/src/lib.rs. The start_recording command already accepts device names; ensure it forwards the system device identifier to recording_commands.rs:

#[tauri::command]
async fn start_recording(
    mic_device: Option<String>,
    system_device: Option<String>,
    meeting_name: Option<String>,
) -> Result<(), String> {
    recording_commands::start_recording(mic_device, system_device, meeting_name).await
}

Test on the Target Platform

Run the application in debug mode to verify the implementation:

• Windows: Execute clean_run_windows.bat and confirm WASAPI loopback devices appear in the device picker.
• macOS: Execute ./clean_run.sh debug and verify ScreenCaptureKit permissions trigger correctly; check that the waveform visualizes both microphone and system streams.
• All platforms: Confirm that the transcription console receives VAD-filtered audio chunks, indicating the pipeline successfully consumed the new capture stream.

Add Platform-Specific Dependencies

Add conditional dependencies to frontend/src-tauri/Cargo.toml using target-specific sections:

[target.'cfg(windows)'.dependencies]
wasapi = "0.12"

[target.'cfg(target_os = "macos")'.dependencies]
core-audio = "0.11"
screen-capture-kit = "0.2"

For new platforms, follow the same pattern to avoid bloating builds on other operating systems.

Key Technical Requirements

All platform implementations must adhere to these strict contract requirements to ensure compatibility with the existing pipeline:

• Sample Rate: 48 kHz (non-negotiable; the VAD model expects this rate).
• Format: 32-bit floating-point PCM (f32).
• Channels: Stereo (2 channels) for system audio; mono for microphone.
• Channel Type: Use mpsc::UnboundedSender<AudioChunk> to transmit data; the pipeline uses the receiver end.
• Timestamps: Each AudioChunk must include a UTC timestamp from chrono::Utc::now() for synchronization.

Summary

• Create a new platform module in frontend/src-tauri/src/audio/devices/platform/ implementing list_audio_devices() and open_system_capture().
• Register the module in platform/mod.rs using conditional compilation (#[cfg(target_os)]).
• Implement the capture logic using native APIs (WASAPI, ScreenCaptureKit, etc.), spawning an async task that sends AudioChunk structs via an unbounded channel.
• Wire the capture sender to AudioPipelineManager::new() in recording_commands.rs without modifying the pipeline internals.
• Expose the functionality through the Tauri command layer in lib.rs to enable UI selection.
• Test using the provided shell scripts (clean_run.sh or clean_run_windows.bat) and verify VAD-filtered output reaches the Whisper engine.

Frequently Asked Questions

What audio format does Meetily require for system capture streams?

Meetily requires 48 kHz sample rate, 32-bit floating-point PCM, stereo channels. The AudioPipelineManager in frontend/src-tauri/src/audio/pipeline.rs assumes this format for VAD processing and synchronization. If your platform API returns different formats (e.g., 44.1 kHz or 16-bit integer), you must resample and convert before sending AudioChunk data into the channel.

How do I handle platform permissions for system audio capture?

For macOS ScreenCaptureKit, the system prompts users for screen recording permission (which includes audio) when you initialize the SCStream. You must add NSScreenCaptureUsageDescription to the Info.plist. For Windows WASAPI, loopback capture requires no special user permissions, but your application manifest should declare uiAccess="false" to run without elevation. Always handle permission errors in open_system_capture() by returning a descriptive Result::Err that the UI can display.

Can I implement system audio capture without modifying the pipeline logic?

Yes. The architecture explicitly separates capture drivers from the pipeline. By implementing the PlatformAudio trait and connecting your capture stream’s sender to AudioPipelineManager::new(), you add support without touching pipeline.rs, vad.rs, or the Whisper integration logic. This design ensures that platform-specific code remains isolated to the devices/platform/ directory.

Where should I add platform-specific dependencies in Meetily?

Add dependencies to frontend/src-tauri/Cargo.toml using target-specific sections rather than the global [dependencies] table. For example, use [target.'cfg(windows)'.dependencies] for WASAPI crates and [target.'cfg(target_os = "macos")'.dependencies] for ScreenCaptureKit bindings. This keeps compile times fast and binary sizes small for other platforms.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}