How Batch Processing for Whisper Transcription Works with Parallel Processors in Meetily

Meetily processes audio in batches by distributing fixed-size chunks across a pool of isolated workers, each running its own Whisper model instance, with automatic resource monitoring and retry logic to prevent system overload.

Meetily is an open-source meeting transcription application built with Rust and Tauri. Its implementation of batch processing for Whisper transcription with parallel processors enables efficient transcription of long meetings by splitting continuous audio streams into discrete chunks and processing them concurrently while strictly respecting system resource limits.

The Chunk-Based Processing Pipeline

The transcription workflow begins in the audio pipeline (pipeline.rs), which groups raw PCM audio into fixed-size AudioChunk structures. Each chunk contains a unique ID, audio data, sample rate, and precise timestamps marking its position in the original stream.

These chunks enter a ChunkQueue wrapped in Arc<RwLock<_>> for thread-safe access across multiple workers. When ParallelProcessor::new initializes the system, it creates this shared queue alongside an mpsc::UnboundedSender<ProcessingEvent> channel to stream status updates back to the frontend.

Resource-Aware Worker Management

Before spawning any workers, Meetily evaluates system capacity to prevent out-of-memory crashes.

Dynamic Worker Count Calculation

The ParallelProcessor::calculate_safe_worker_count method queries the SystemMonitor (defined in frontend/src-tauri/src/whisper_engine/system_monitor.rs) to read current memory and CPU usage. The system computes a safe number of concurrent workers and caps the result at 4 to ensure stability on consumer hardware.

SystemMonitor Integration

A background task (start_resource_monitoring) periodically calls SystemMonitor::check_resource_constraints. When resource constraints are detected, the processor automatically emits a ProcessingPaused event and halts new chunk processing until ProcessingResumed indicates healthy resource levels again.

Worker Architecture and Isolation

Each worker operates independently to prevent model contention and ensure fault isolation.

Per-Worker WhisperEngine Instances

The ParallelProcessor::create_worker method spins up a dedicated Tokio task for each worker. Every worker loads its own WhisperEngine instance via WhisperEngine::load_model (implemented in frontend/src-tauri/src/whisper_engine/whisper_engine.rs), ensuring that heavy model initialization happens once per worker rather than per chunk.

Chunk Processing with Timeouts

Workers enter a loop that pulls pending chunks from the ChunkQueue. The ParallelProcessor::process_chunk_safely method:

1. Acquires a semaphore permit to enforce the maximum worker limit
2. Executes WhisperEngine::transcribe_audio with a 2-minute timeout
3. Wraps successful results in a TranscriptionResult struct containing processing time, model name, and timestamps

Resilience and Error Handling

Meetily implements multiple layers of fault tolerance to handle transient failures.

Automatic Retry Mechanism

When process_chunk_safely encounters a recoverable error, it creates a ProcessingError event. If the retry count remains below max_retries, the chunk moves to the ChunkQueue.retry_queue for later reprocessing. Chunks exceeding the retry limit are marked as permanently failed and reported via the event channel.

Graceful Degradation

The system supports enable_fallback_mode, which automatically switches to sequential processing when parallel workers encounter persistent errors, ensuring transcription continues even under resource constraints.

Event-Driven Observability

Throughout the lifecycle, workers emit ProcessingEvent variants including ChunkStarted, ChunkCompleted, ChunkFailed, WorkerStarted, and WorkerStopped. The frontend receives these events through Tauri’s event system to update UI progress bars and status indicators in real time.

The ParallelProcessor::get_processing_status method provides a snapshot of queue health, including pending chunk counts and active worker status, accessible via Tauri commands for frontend state management.

Implementation Example

The following example demonstrates initializing a parallel transcription run with resource monitoring:

use meetily::frontend::src_tauri::whisper_engine::{
    parallel_processor::{ParallelProcessor, ParallelConfig},
    system_monitor::create_system_monitor,
};
use std::sync::Arc;
use tokio::sync::RwLock;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Create a system monitor (handles CPU / memory checks)
    let sys_monitor = Arc::new(create_system_monitor());

    // Configure parallel processing (2 workers, 512 MiB each, 3 retries)
    let config = ParallelConfig {
        max_workers: 2,
        memory_budget_mb: 512,
        max_retries: 3,
        retry_delay_ms: 1_000,
        resource_check_interval_ms: 10_000,
        enable_fallback_mode: true,
    };

    // Initialise the processor and obtain an event receiver
    let (mut processor, mut events) = ParallelProcessor::new(config, sys_monitor.clone())?;

    // Prepare a batch of audio chunks (normally produced by the audio pipeline)
    let chunks = vec![
        AudioChunk { id: 1, data: vec![/* PCM samples */], sample_rate: 48_000,
                     start_time_ms: 0.0, duration_ms: 500.0 },
        AudioChunk { id: 2, data: vec![/* … */], sample_rate: 48_000,
                     start_time_ms: 500.0, duration_ms: 500.0 },
    ];

    // Start processing using the "base" Whisper model
    processor.start_processing(chunks, "base".to_string()).await?;

    // Listen for progress events (e.g., update a UI progress bar)
    while let Some(event) = events.recv().await {
        match event {
            ProcessingEvent::ChunkCompleted(res) => {
                println!("Chunk {} transcribed in {} ms", res.chunk_id, res.processing_time_ms);
            }
            ProcessingEvent::ChunkFailed(err) => {
                eprintln!("Chunk {} failed: {}", err.chunk_id, err.error_message);
            }
            ProcessingEvent::ProcessingPaused => {
                println!("Processing paused due to high resource usage");
            }
            ProcessingEvent::ProcessingResumed => {
                println!("Resources OK – processing resumed");
            }
            _ => {}
        }
    }

    // When done, stop workers gracefully
    processor.stop_processing().await;
    Ok(())
}

To query processing status from the UI layer:

#[tauri::command]
async fn get_transcription_status(state: State<'_, AppState>) -> Result<ProcessingStatus, String> {
    let processor = &state.parallel_processor;
    Ok(processor.get_processing_status().await)
}

For dynamic worker adjustment after initialization:

// Increase workers after a user enables "High-Performance" mode
let safe_count = processor.calculate_safe_worker_count().await?;
processor.spawn_workers(safe_count, "medium".to_string()).await?;

Summary

• Chunk-based architecture: Audio streams split into AudioChunk objects with metadata for parallel processing
• Resource safety: SystemMonitor and calculate_safe_worker_count cap workers at 4 and pause processing during resource constraints
• Worker isolation: Each Tokio task owns a separate WhisperEngine instance loaded once via WhisperEngine::load_model
• Resilience: Automatic retries with max_retries limit and fallback to sequential mode via enable_fallback_mode
• Observability: Rich ProcessingEvent streams and ProcessingStatus snapshots for real-time UI updates

Frequently Asked Questions

How does Meetily prevent memory overload during batch transcription?

Meetily implements a hard cap of 4 workers regardless of CPU count, enforced by calculate_safe_worker_count in parallel_processor.rs. The SystemMonitor continuously checks memory and CPU usage through check_resource_constraints, automatically emitting ProcessingPaused events when resources are constrained and resuming only when capacity returns.

What happens when a chunk fails transcription?

Failed chunks trigger ProcessingError events. If the failure is recoverable and the retry count is below max_retries, the chunk moves to the retry_queue for later attempts. Permanently failed chunks are marked as such and reported to the frontend via ChunkFailed events, allowing the UI to display error states without crashing the entire batch.

Can the number of workers change during processing?

Yes. While ParallelProcessor::new sets the initial configuration, you can dynamically adjust workers by calling calculate_safe_worker_count followed by spawn_workers with a new model size. This allows the system to adapt to changing resource conditions or user preferences (e.g., switching from "base" to "medium" models when high-performance mode is enabled).

How does the frontend track transcription progress in real time?

The frontend listens to Tauri events emitted through the mpsc::UnboundedSender<ProcessingEvent> channel created during ParallelProcessor initialization. Workers emit ChunkStarted, ChunkCompleted, and ChunkFailed events throughout processing, while the resource monitor sends ProcessingPaused and ProcessingResumed updates. The frontend can also poll get_processing_status to retrieve current queue depth and active worker counts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}