Meetily Parallel Transcription Processing Architecture: Audio Chunk Batching Explained

Meetily processes meeting recordings using a client-side Rust architecture that mixes microphone and system audio through a voice activity detector, then distributes speech segments across a configurable worker pool with automatic resource monitoring and smart batching for telemetry.

Meetily is an open-source meeting transcription tool built by Zackriya Solutions that runs entirely on the client side. Its parallel transcription processing architecture leverages a Rust-based core to handle real-time audio mixing, voice activity detection, and distributed Whisper inference without sending data to external servers. Understanding how this system batches audio chunks and manages parallel workers reveals how the application maintains low latency while preserving system resources.

Architecture Overview

The transcription pipeline consists of three tightly-coupled Rust components that operate across separate Tauri modules:

• AudioPipeline ([frontend/src-tauri/src/audio/pipeline.rs](frontend/src-tauri/src/audio/pipeline.rs)) – Mixes raw microphone and system audio using AudioMixerRingBuffer and ProfessionalAudioMixer, segments speech via ContinuousVadProcessor, and dispatches AudioChunk structs to the transcription channel.

• BatchProcessor ([frontend/src-tauri/src/audio/batch_processor.rs](frontend/src-tauri/src/audio/batch_processor.rs)) – A generic smart-batching utility that collects high-frequency AudioMetric events and aggregates them into AudioMetricsSummary objects to reduce UI overhead.

• ParallelProcessor ([frontend/src-tauri/src/whisper_engine/parallel_processor.rs](frontend/src-tauri/src/whisper_engine/parallel_processor.rs)) – The core parallel transcription engine that owns a worker pool, chunk queue, and SystemMonitor for CPU/memory throttling. Each worker maintains an isolated WhisperEngine instance.

Parallel Transcription Processing Architecture

Audio Pipeline and VAD Integration

The AudioPipeline::run method continuously receives raw audio streams from the capture layer. It first mixes microphone and system audio into a single stream, then applies voice activity detection through ContinuousVadProcessor. Valid speech segments are wrapped in AudioChunk structs with 16 kHz sample rate and timestamp metadata in seconds.

Once created, chunks flow through a tokio::mpsc::UnboundedSender channel to the ParallelProcessor, ensuring lock-free hand-off between the high-frequency audio path and the transcription workers.

The Chunk Queue and Worker Pool

The ParallelProcessor maintains a ChunkQueue with four distinct collections:

• pending – Freshly enqueued chunks awaiting processing
• processing – Chunks currently assigned to active workers
• completed – Successful TranscriptionResult objects
• failed – Permanent failures after exhausting max_retries

A separate retry_queue stores chunks that need reprocessing. The system spawns up to four workers (safety-capped) via ParallelProcessor::create_worker, where each worker initializes its own WhisperEngine and loads the requested model once.

The worker loop executes process_chunk_safely, which wraps engine.transcribe_audio with a 120-second timeout. Success emits a ChunkCompleted event; failure generates ChunkFailed and triggers re-queuing logic up to the configured retry limit.

Resource Monitoring and Safety Caps

A background task (start_resource_monitoring) periodically checks system constraints via SystemMonitor. If CPU or memory usage violates thresholds, the pipeline automatically enters ProcessingPaused state and resumes only when resources recover. This prevents the Whisper workers from overwhelming the host machine during long meetings.

Consumers query current status through ParallelProcessor::get_processing_status, which returns totals for all queue states plus pause/stop flags.

Audio Chunk Batching Strategies

Metrics Batching with BatchProcessor

To prevent telemetry from stalling the audio pipeline, Meetily implements a BatchProcessor<AudioMetric, AudioMetricsSummary> pattern. The AudioPipeline creates an AudioMetricsBatcher that accumulates lightweight metrics (timestamp, chunk ID, sample count, duration, average level) and flushes them every 50 chunks or 5 seconds—whichever comes first.

This "smart-batching" approach dramatically reduces logging overhead while providing the UI with regular AudioMetricsSummary updates for visualization.

Chunk-Level Processing and Retry Logic

While the ParallelProcessor processes one chunk per worker at a time, the ChunkQueue effectively batch-manages work through its retry mechanism. Failed chunks automatically re-enter the queue with priority over new items, creating a "soft-batching" effect that balances near-real-time transcription with robustness against transient errors.

The queue handles backpressure naturally: new chunks accumulate in pending while workers process existing items, ensuring no audio data is dropped during CPU-intensive transcription phases.

Implementation Flow and Code Examples

Initializing the Parallel Processor

From the frontend TypeScript layer, initialize the worker pool with resource constraints:

// Configure the parallel processor with 2 workers and 1GB memory budget
await invoke('initialize_parallel_processor', {
  max_workers: 2,
  memory_budget_mb: 1024,
});

// Prepare audio chunks from raw float array (48kHz input, 30s segments)
const chunks = await invoke('prepare_audio_chunks', {
  audio_data: rawFloatArray,
  sample_rate: 48000,
  chunk_duration_ms: 30000,
});

// Start processing with the 'base' Whisper model
await invoke('start_parallel_processing', {
  audio_chunks: chunks,
  model_name: 'base',
});

Consuming Metrics and Status

React hooks consume batched metrics and processing status:

useEffect(() => {
  // Subscribe to batched audio metrics from BatchProcessor
  const unsub = listen<AudioMetricsSummary[]>('audio-metrics', ({ payload }) => {
    setMetrics(payload); // Vec<AudioMetricsSummary> from Rust
  });
  
  return () => unsub();
}, []);

// Query current queue status
const status = await invoke('get_parallel_processing_status');
// Returns: { total_chunks, completed_chunks, failed_chunks, is_paused, ... }

Graceful Shutdown

Force a flush of pending chunks before stopping:

await invoke('force_flush_and_stop');
// Sends special flush chunks through the pipeline, waits for completion, then stops workers

Summary

• Meetily's parallel transcription processing architecture runs entirely client-side in Rust, mixing microphone and system audio before VAD segmentation.
• The ParallelProcessor manages up to four isolated Whisper workers with a four-state ChunkQueue (pending, processing, completed, failed) and automatic retry logic.
• BatchProcessor aggregates high-frequency audio metrics into summaries every 50 chunks or 5 seconds, preventing UI bottlenecking.
• SystemMonitor provides automatic CPU and memory throttling, pausing transcription when resources are constrained and resuming when available.
• All components communicate via tokio::mpsc channels, ensuring the audio capture thread never blocks on transcription or logging operations.

Frequently Asked Questions

How does Meetily handle CPU and memory limits during transcription?

The ParallelProcessor integrates a SystemMonitor that runs in a background task (start_resource_monitoring). It periodically checks system resources against configurable thresholds. When CPU or memory usage exceeds limits, the processor emits a ProcessingPaused event and stops feeding new chunks to workers. Processing automatically resumes when metrics return to safe levels, preventing system overload during long meetings.

What happens if a transcription worker fails to process a chunk?

Failed chunks trigger the retry mechanism in ParallelProcessor::process_chunk_safely. The chunk moves to a retry_queue and the worker attempts transcription again up to max_retries times. After exhausting retries, the chunk moves to the failed collection and emits a ChunkFailed event. This ensures temporary inference errors don't result in permanent data loss.

Why does each worker create its own WhisperEngine instance?

According to the source code in [parallel_processor.rs](frontend/src-tauri/src/whisper_engine/parallel_processor.rs), each worker calls create_worker to instantiate an isolated WhisperEngine. This design prevents model reload overhead for each chunk and isolates inference failures to individual workers. It also allows GPU/CPU resources to be partitioned safely among parallel threads without contention.

How does the metrics batching affect real-time performance?

The BatchProcessor implementation decouples metric collection from the audio pipeline. By buffering AudioMetric structs and flushing only every 50 chunks or 5 seconds (whichever comes first), the system reduces logging overhead by 90% or more compared to per-chunk emissions. This occurs on a separate channel from the transcription queue, ensuring that telemetry processing never stalls the VAD or chunk dispatch loops.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}