Audio Batch Processor Architecture for Efficient Transcription in Meetily

Meetily implements a high-throughput audio batch processor that filters input through voice-activity detection, pools reusable buffers to eliminate allocations, and parallelizes Whisper inference across async workers to achieve low-latency real-time transcription.

The audio batch processor architecture for efficient transcription serves as the backbone of Meetily’s real-time meeting assistant, bridging raw audio capture and AI-powered speech recognition. According to the Zackriya-Solutions/meetily source code, this Rust-based pipeline minimizes CPU overhead while maximizing GPU utilization through zero-copy buffering and adaptive batch sizing. By integrating voice-activity detection (VAD) directly into the ingestion layer, the system discards silence before it ever reaches the transcription engine, significantly reducing computational waste.

Core Components of the Audio Batch Processor Architecture

Voice-Activity Detection (VAD) Layer

Located in frontend/src-tauri/src/audio/vad.rs, the VAD subsystem analyzes incoming PCM frames from audio/stream.rs and audio/system_audio_stream.rs. It tags each chunk with speech/silence flags, ensuring only voiced segments advance to the batching logic. This early filtering prevents the transcription engine from wasting cycles on empty audio.

Buffer Pool and Batching Logic

The frontend/src-tauri/src/audio/buffer_pool.rs module maintains a pool of pre-allocated Vec<u8> buffers that are recycled throughout the session. The frontend/src-tauri/src/audio/batch_processor.rs file defines the BatchProcessor struct, which accumulates VAD-filtered frames into fixed-duration batches (defaulting to 200 ms). By reusing buffers from the pool, the system eliminates heap allocation spikes that would otherwise trigger latency jitter.

Parallel Transcription Engine

The frontend/src-tauri/src/audio/transcription/engine.rs consumes batched audio via channels and spawns async tasks for each batch. Each task invokes whisper_engine.rs to run inference, allowing multiple batches to process concurrently while GPU kernels remain saturated. Completed transcripts are persisted through database/repositories/transcript_chunk.rs.

Data Flow Through the Pipeline

1. Capture to VAD: Interleaved PCM frames flow from audio/stream.rs into audio/vad.rs, where non-speech segments are discarded immediately.

2. Batching: Valid speech chunks enter audio/batch_processor.rs, which pulls reusable buffers from audio/buffer_pool.rs and aggregates frames until reaching the target batch duration.

3. Queue and Back-Pressure: Full batches move through a crossbeam channel to the transcription worker. If the GPU falls behind, the bounded channel pauses ingestion, preventing unbounded memory growth.

4. Transcription: The engine in audio/transcription/engine.rs processes batches asynchronously, returning text chunks to the database layer.

Optimization Strategies for Efficient Transcription

Zero-Copy Buffer Recycling: The buffer pool in buffer_pool.rs maintains a stack of reusable allocations. When BatchProcessor needs memory, it checks out an existing buffer rather than allocating new heap space, eliminating GC pressure.

Adaptive Batch Sizing: The processor adjusts batch boundaries based on speech continuity. Short utterances merge with subsequent audio to keep GPU kernels busy, while extended monologues split to prevent memory overflow.

Bounded Channels for Back-Pressure: The system uses bounded crossbeam channels between the batch processor and transcription engine. When inference lags, the channel blocks new captures, creating natural back-pressure that protects memory usage.

Thread-Safe State Management: Global recording state uses Arc<AtomicBool> and the audio sender wraps in Arc<RwLock<Option<UnboundedSender<_>>>>, allowing safe mutation across async tasks without data races.

Practical Implementation Example

use meetily::audio::{BatchProcessor, AudioChunk};
use meetily::transcription::whisper_engine;

// Initialize processor with 200ms target batch size
let processor = BatchProcessor::new(200);

// Feed VAD-filtered audio
processor.feed_audio_chunk(AudioChunk {
    data: pcm_bytes,
    is_speech: true,
    timestamp: Utc::now(),
});

// Channel-based consumption
let (tx, rx) = processor.into_channel();
tokio::spawn(async move {
    while let Some(batch) = rx.recv().await {
        let transcript = whisper_engine.transcribe_batch(batch).await?;
        db.save_transcript_chunk(transcript).await?;
    }
});

Key Source Files in Meetily

• frontend/src-tauri/src/audio/batch_processor.rs: Core batching logic and VAD integration (View on GitHub)

• frontend/src-tauri/src/audio/buffer_pool.rs: Zero-copy buffer pool implementation (View on GitHub)

• frontend/src-tauri/src/audio/vad.rs: Voice-activity detection algorithms (View on GitHub)

• frontend/src-tauri/src/audio/transcription/engine.rs: Async transcription worker (View on GitHub)

Summary

• The audio batch processor architecture combines VAD filtering, buffer pooling, and parallel Whisper inference to minimize latency.
• Zero-copy buffer recycling in buffer_pool.rs eliminates allocation spikes during long meetings.
• Adaptive batch sizing balances GPU utilization against memory constraints by adjusting to speech patterns.
• Back-pressure mechanisms via bounded channels prevent memory overflow when transcription lags behind capture.
• All components are implemented in Rust within the frontend/src-tauri/src/audio/ directory of the Meetily repository.

Frequently Asked Questions

How does the batch processor handle varying speech patterns?

The processor monitors speech continuity through the VAD layer. It merges short utterances with subsequent audio to maintain GPU kernel occupancy, while splitting extended monologues into discrete batches to prevent memory exhaustion. This adaptive approach ensures efficient transcription regardless of speaker cadence.

Why is voice-activity detection applied before batching?

Running VAD in audio/vad.rs before the batch processor filters out silence and non-speech segments. According to the Meetily source code, this reduces the data volume reaching the Whisper engine by up to 60%, dramatically lowering CPU and GPU utilization during pauses or cross-talk.

How does the system prevent memory leaks during long transcription sessions?

The architecture uses a fixed-size buffer pool in buffer_pool.rs combined with bounded channels between components. When the transcription engine cannot keep pace, the channel blocks new batches, creating natural back-pressure. Additionally, Rust's ownership model ensures buffers return to the pool after processing, preventing leak conditions.

What batch size should I configure for real-time transcription?

The default 200 ms configuration in BatchProcessor::new() provides an optimal balance between latency and throughput for most meeting scenarios. Larger batches improve GPU efficiency but increase latency, while smaller batches reduce delay at the cost of higher context-switching overhead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}