# How Batch Processing for Whisper Transcription Works with Parallel Processors in Meetily

> Discover how Meetily leverages batch processing and parallel processors for efficient Whisper transcription. See how audio is distributed across workers for faster results.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: internals
- Published: 2026-07-17

---

**Meetily processes audio in batches by distributing fixed-size chunks across a pool of isolated workers, each running its own Whisper model instance, with automatic resource monitoring and retry logic to prevent system overload.**

Meetily is an open-source meeting transcription application built with Rust and Tauri. Its implementation of **batch processing for Whisper transcription with parallel processors** enables efficient transcription of long meetings by splitting continuous audio streams into discrete chunks and processing them concurrently while strictly respecting system resource limits.

## The Chunk-Based Processing Pipeline

The transcription workflow begins in the audio pipeline ([`pipeline.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/pipeline.rs)), which groups raw PCM audio into fixed-size **AudioChunk** structures. Each chunk contains a unique ID, audio data, sample rate, and precise timestamps marking its position in the original stream.

These chunks enter a **ChunkQueue** wrapped in `Arc<RwLock<_>>` for thread-safe access across multiple workers. When `ParallelProcessor::new` initializes the system, it creates this shared queue alongside an `mpsc::UnboundedSender<ProcessingEvent>` channel to stream status updates back to the frontend.

## Resource-Aware Worker Management

Before spawning any workers, Meetily evaluates system capacity to prevent out-of-memory crashes.

### Dynamic Worker Count Calculation

The `ParallelProcessor::calculate_safe_worker_count` method queries the **SystemMonitor** (defined in [`frontend/src-tauri/src/whisper_engine/system_monitor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/system_monitor.rs)) to read current memory and CPU usage. The system computes a safe number of concurrent workers and caps the result at **4** to ensure stability on consumer hardware.

### SystemMonitor Integration

A background task (`start_resource_monitoring`) periodically calls `SystemMonitor::check_resource_constraints`. When resource constraints are detected, the processor automatically emits a `ProcessingPaused` event and halts new chunk processing until `ProcessingResumed` indicates healthy resource levels again.

## Worker Architecture and Isolation

Each worker operates independently to prevent model contention and ensure fault isolation.

### Per-Worker WhisperEngine Instances

The `ParallelProcessor::create_worker` method spins up a dedicated Tokio task for each worker. Every worker loads its own **WhisperEngine** instance via `WhisperEngine::load_model` (implemented in [`frontend/src-tauri/src/whisper_engine/whisper_engine.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/whisper_engine.rs)), ensuring that heavy model initialization happens once per worker rather than per chunk.

### Chunk Processing with Timeouts

Workers enter a loop that pulls pending chunks from the `ChunkQueue`. The `ParallelProcessor::process_chunk_safely` method:

1. Acquires a semaphore permit to enforce the maximum worker limit
2. Executes `WhisperEngine::transcribe_audio` with a **2-minute timeout**
3. Wraps successful results in a `TranscriptionResult` struct containing processing time, model name, and timestamps

## Resilience and Error Handling

Meetily implements multiple layers of fault tolerance to handle transient failures.

### Automatic Retry Mechanism

When `process_chunk_safely` encounters a recoverable error, it creates a `ProcessingError` event. If the retry count remains below `max_retries`, the chunk moves to the `ChunkQueue.retry_queue` for later reprocessing. Chunks exceeding the retry limit are marked as permanently failed and reported via the event channel.

### Graceful Degradation

The system supports `enable_fallback_mode`, which automatically switches to sequential processing when parallel workers encounter persistent errors, ensuring transcription continues even under resource constraints.

## Event-Driven Observability

Throughout the lifecycle, workers emit **ProcessingEvent** variants including `ChunkStarted`, `ChunkCompleted`, `ChunkFailed`, `WorkerStarted`, and `WorkerStopped`. The frontend receives these events through Tauri’s event system to update UI progress bars and status indicators in real time.

The `ParallelProcessor::get_processing_status` method provides a snapshot of queue health, including pending chunk counts and active worker status, accessible via Tauri commands for frontend state management.

## Implementation Example

The following example demonstrates initializing a parallel transcription run with resource monitoring:

```rust
use meetily::frontend::src_tauri::whisper_engine::{
    parallel_processor::{ParallelProcessor, ParallelConfig},
    system_monitor::create_system_monitor,
};
use std::sync::Arc;
use tokio::sync::RwLock;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Create a system monitor (handles CPU / memory checks)
    let sys_monitor = Arc::new(create_system_monitor());

    // Configure parallel processing (2 workers, 512 MiB each, 3 retries)
    let config = ParallelConfig {
        max_workers: 2,
        memory_budget_mb: 512,
        max_retries: 3,
        retry_delay_ms: 1_000,
        resource_check_interval_ms: 10_000,
        enable_fallback_mode: true,
    };

    // Initialise the processor and obtain an event receiver
    let (mut processor, mut events) = ParallelProcessor::new(config, sys_monitor.clone())?;

    // Prepare a batch of audio chunks (normally produced by the audio pipeline)
    let chunks = vec![
        AudioChunk { id: 1, data: vec![/* PCM samples */], sample_rate: 48_000,
                     start_time_ms: 0.0, duration_ms: 500.0 },
        AudioChunk { id: 2, data: vec![/* … */], sample_rate: 48_000,
                     start_time_ms: 500.0, duration_ms: 500.0 },
    ];

    // Start processing using the "base" Whisper model
    processor.start_processing(chunks, "base".to_string()).await?;

    // Listen for progress events (e.g., update a UI progress bar)
    while let Some(event) = events.recv().await {
        match event {
            ProcessingEvent::ChunkCompleted(res) => {
                println!("Chunk {} transcribed in {} ms", res.chunk_id, res.processing_time_ms);
            }
            ProcessingEvent::ChunkFailed(err) => {
                eprintln!("Chunk {} failed: {}", err.chunk_id, err.error_message);
            }
            ProcessingEvent::ProcessingPaused => {
                println!("Processing paused due to high resource usage");
            }
            ProcessingEvent::ProcessingResumed => {
                println!("Resources OK – processing resumed");
            }
            _ => {}
        }
    }

    // When done, stop workers gracefully
    processor.stop_processing().await;
    Ok(())
}

```

To query processing status from the UI layer:

```rust
#[tauri::command]
async fn get_transcription_status(state: State<'_, AppState>) -> Result<ProcessingStatus, String> {
    let processor = &state.parallel_processor;
    Ok(processor.get_processing_status().await)
}

```

For dynamic worker adjustment after initialization:

```rust
// Increase workers after a user enables "High-Performance" mode
let safe_count = processor.calculate_safe_worker_count().await?;
processor.spawn_workers(safe_count, "medium".to_string()).await?;

```

## Summary

- **Chunk-based architecture**: Audio streams split into `AudioChunk` objects with metadata for parallel processing
- **Resource safety**: `SystemMonitor` and `calculate_safe_worker_count` cap workers at 4 and pause processing during resource constraints
- **Worker isolation**: Each Tokio task owns a separate `WhisperEngine` instance loaded once via `WhisperEngine::load_model`
- **Resilience**: Automatic retries with `max_retries` limit and fallback to sequential mode via `enable_fallback_mode`
- **Observability**: Rich `ProcessingEvent` streams and `ProcessingStatus` snapshots for real-time UI updates

## Frequently Asked Questions

### How does Meetily prevent memory overload during batch transcription?

Meetily implements a **hard cap of 4 workers** regardless of CPU count, enforced by `calculate_safe_worker_count` in [`parallel_processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/parallel_processor.rs). The `SystemMonitor` continuously checks memory and CPU usage through `check_resource_constraints`, automatically emitting `ProcessingPaused` events when resources are constrained and resuming only when capacity returns.

### What happens when a chunk fails transcription?

Failed chunks trigger `ProcessingError` events. If the failure is recoverable and the retry count is below `max_retries`, the chunk moves to the `retry_queue` for later attempts. Permanently failed chunks are marked as such and reported to the frontend via `ChunkFailed` events, allowing the UI to display error states without crashing the entire batch.

### Can the number of workers change during processing?

Yes. While `ParallelProcessor::new` sets the initial configuration, you can dynamically adjust workers by calling `calculate_safe_worker_count` followed by `spawn_workers` with a new model size. This allows the system to adapt to changing resource conditions or user preferences (e.g., switching from "base" to "medium" models when high-performance mode is enabled).

### How does the frontend track transcription progress in real time?

The frontend listens to Tauri events emitted through the `mpsc::UnboundedSender<ProcessingEvent>` channel created during `ParallelProcessor` initialization. Workers emit `ChunkStarted`, `ChunkCompleted`, and `ChunkFailed` events throughout processing, while the resource monitor sends `ProcessingPaused` and `ProcessingResumed` updates. The frontend can also poll `get_processing_status` to retrieve current queue depth and active worker counts.