# How the Parallel Processor Enables Batch Transcription Workloads in Meetily

> Discover how Meetily's parallel processor handles batch transcription. Learn how its resource-aware worker pool and queue system process large audio batches efficiently and safely.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: internals
- Published: 2026-07-25

---

**The ParallelProcessor employs a resource-aware worker pool with isolated WhisperEngine instances per worker and a robust queue system to safely transcribe large batches of audio chunks in parallel while dynamically adjusting to system constraints.**

The Meetily open-source meeting assistant relies on efficient batch transcription to process recorded audio. Its parallel processor architecture, implemented in Rust within the Tauri backend, combines adaptive resource management with isolated model inference to handle high-throughput workloads without overwhelming the host system.

## Core Architecture Components

The parallel processor centers around five key structures defined in [`frontend/src-tauri/src/whisper_engine/parallel_processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/parallel_processor.rs) that work together to manage concurrent transcription tasks.

### ParallelConfig and Safety Limits

The `ParallelConfig` struct establishes conservative defaults to prevent resource exhaustion. It restricts workers to a maximum of **2** by default and never exceeds **4**, while also defining memory budgets and retry policies:

- **`max_workers`**: Capped at 4 for safety
- **`memory_budget_mb`**: Per-worker memory allocation limit
- **`max_retries`**: Automatic retry attempts for transient failures

*Source: `ParallelConfig` struct, lines 52-70*

### ChunkQueue State Management

The `ChunkQueue` tracks the lifecycle of every audio chunk through distinct states. It maintains separate collections for pending, processing, completed, and failed chunks, plus a dedicated retry queue for transient errors. When initialized via `ChunkQueue::new()`, it prepares these tracking collections for batch ingestion.

*Source: `ChunkQueue` struct and `new()`, lines 93-100 and 59-68*

### Worker Isolation

Each `Worker` struct (lines 86-90) runs as an independent async task with its own `WhisperEngine` instance. The `create_worker()` function (lines 199-226) spawns these workers, ensuring complete isolation between transcription threads.

## Resource-Aware Worker Scaling

Before processing begins, the system evaluates available CPU and memory to determine a safe concurrency level that protects the host from overload.

### Dynamic Worker Creation

The `start_processing()` method (lines 44-63) queries the `SystemMonitor` to calculate a safe worker count via `calculate_safe_worker_count()` (lines 33-42), clamping the result to the configured maximum:

```rust
let safe_worker_count = self.calculate_safe_worker_count().await?;
self.spawn_workers(safe_worker_count, model_name).await?;

```

This approach ensures the processor never spawns more workers than the system can safely support, even if the user requests a higher number in `ParallelConfig`.

### System Health Monitoring

The `start_resource_monitoring()` task (lines 72-89) runs continuously in the background, checking system constraints and automatically pausing processing when resources are constrained. When the monitor detects high load, it sets a shared pause flag (lines 84-105):

```rust
if !status.can_proceed { 
    *is_paused.write().await = true; 
} else if status.can_proceed && *is_paused.read().await { 
    *is_paused.write().await = false; 
}

```

This mechanism automatically pauses the batch pipeline (`ProcessingPaused`) when constraints are hit and resumes (`ProcessingResumed`) when resources become available again.

## Isolated Model Inference

Unlike architectures that share a single model instance, Meetily's parallel processor gives each worker exclusive access to its own WhisperEngine to eliminate contention.

### Per-Worker WhisperEngine Instances

The `create_worker()` function initializes a dedicated engine for each worker thread (lines 22-28):

```rust
let engine = WhisperEngine::new()?;
engine.load_model(&model_name).await?;
*engine_guard = Some(engine);

```

This design prevents model contention and allows workers to process chunks independently without blocking each other, though it requires sufficient memory to load multiple model instances simultaneously.

## Queue-Driven Batch Processing

The processor accepts arbitrary batch sizes by loading all chunks into a pending queue before workers begin consumption, enabling true batch transcription workflows.

### Batch Ingestion

When `start_processing()` receives a vector of audio chunks, it atomically assigns them to the pending queue (lines 64-71):

```rust
self.chunk_queue.write().await.pending = chunks;

```

### Semaphore-Based Concurrency Control

A `tokio::Semaphore` ensures the system never exceeds the calculated safe worker count. Each worker must acquire a permit before processing (lines 16-18):

```rust
let _permit = semaphore.acquire().await?;

```

Workers pull chunks preferring items from the retry queue, move them to the `processing` list, and execute `process_chunk_safely()`. Completed results populate the `completed` collection, while failures enter the `failed` list or return to the retry queue up to `max_retries` times.

## Event-Driven Observability

The processor emits strongly-typed events through an unbounded channel, allowing the UI layer to track transcription progress. The `ProcessingEvent` enum (lines 38-48) includes variants such as `ChunkStarted`, `ChunkCompleted`, `WorkerStarted`, and `ResourceConstraint`.

Listening for progress:

```rust
while let Some(event) = events.recv().await {
    match event {
        ProcessingEvent::ChunkCompleted(result) => {
            println!("Chunk {}: {}", result.chunk_id, result.text);
        },
        ProcessingEvent::ResourceConstraint(msg) => {
            eprintln!("Warning: {}", msg);
        },
        _ => {}
    }
}

```

This event stream makes the batch workflow observable without blocking the transcription pipeline.

## Practical Implementation

Integrating the parallel processor into a Tauri command involves configuring safety limits, instantiating the processor, and managing lifecycle events according to the patterns found in [`frontend/src-tauri/src/lib.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/lib.rs).

### Starting a Batch Transcription Job

```rust
use frontend::src_tauri::src::whisper_engine::{
    parallel_processor::{ParallelProcessor, ParallelConfig},
    system_monitor::SystemMonitor,
};
use tokio::sync::RwLock;
use std::sync::Arc;

// Configure safety limits
let config = ParallelConfig {
    max_workers: 3,
    memory_budget_mb: 1024,
    ..Default::default()
};

let system_monitor = Arc::new(SystemMonitor::new()?);
let (mut processor, mut events) = ParallelProcessor::new(config, system_monitor)?;

// Load batch and start
let audio_chunks = vec![/* AudioChunk structs */];
processor.start_processing(audio_chunks, "large-v3".to_string()).await?;

```

*Source: `ParallelProcessor::new` (lines 101-130) and `start_processing` (lines 44-52)*

### Lifecycle Controls

The processor supports graceful pause, resume, and shutdown operations:

```rust
processor.pause_processing().await;   // Halt all workers temporarily
processor.resume_processing().await;  // Continue processing
processor.stop_processing().await;    // Gracefully shut down the pool

```

*Source: `pause_processing`, `resume_processing`, `stop_processing` (lines 15-27)*

## Summary

- **Resource-aware scaling**: The `calculate_safe_worker_count()` function and `SystemMonitor` ensure worker pools never exceed safe CPU/memory thresholds, automatically capping at 4 workers.
- **Complete isolation**: Each worker maintains its own `WhisperEngine` instance in `create_worker()`, eliminating model contention and thread-safety issues during parallel inference.
- **Robust queue management**: The `ChunkQueue` handles pending, processing, completed, and failed states with automatic retry logic for transient failures up to `max_retries`.
- **Safety mechanisms**: A `tokio::Semaphore` enforces concurrency limits, while `start_resource_monitoring()` pauses processing during resource constraints and resumes when capacity returns.
- **Observable pipeline**: Event-driven architecture via `ProcessingEvent` allows real-time monitoring of batch transcription progress through an unbounded channel.

## Frequently Asked Questions

### How does the parallel processor prevent system overload?

The processor checks system resources before spawning workers using `calculate_safe_worker_count()` in [`system_monitor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/system_monitor.rs). It enforces hard limits through `ParallelConfig` (max 4 workers) and uses a semaphore to restrict concurrent execution. A background monitoring task automatically pauses transcription when CPU or memory constraints are detected, resuming only when resources become available.

### Why does each worker use a separate WhisperEngine instance?

According to the source code in `create_worker()` (lines 22-28 and 199-226), each worker instantiates its own `WhisperEngine` and loads the model once at startup. This isolation prevents thread contention on the model state and allows true parallel execution without synchronization bottlenecks, though it requires additional memory per worker.

### What happens when a chunk fails transcription?

Failed chunks enter a retry queue managed by the `ChunkQueue` structure. Workers preferentially pull from the retry queue before processing new pending chunks. If a chunk fails repeatedly beyond the `max_retries` limit defined in `ParallelConfig`, it moves to the `failed` collection and emits a `ProcessingEvent` for logging or manual intervention.

### How do I monitor the progress of a batch transcription job?

The `ParallelProcessor::new()` constructor returns an unbounded event receiver alongside the processor instance. Spawn an async task to listen for `ProcessingEvent` variants such as `ChunkCompleted`, `WorkerStarted`, or `ResourceConstraint` (defined at lines 38-48). This pattern enables real-time UI updates and progress tracking without blocking the transcription pipeline.