How the Parallel Processor Enables Batch Transcription Workloads in Meetily

The ParallelProcessor employs a resource-aware worker pool with isolated WhisperEngine instances per worker and a robust queue system to safely transcribe large batches of audio chunks in parallel while dynamically adjusting to system constraints.

The Meetily open-source meeting assistant relies on efficient batch transcription to process recorded audio. Its parallel processor architecture, implemented in Rust within the Tauri backend, combines adaptive resource management with isolated model inference to handle high-throughput workloads without overwhelming the host system.

Core Architecture Components

The parallel processor centers around five key structures defined in frontend/src-tauri/src/whisper_engine/parallel_processor.rs that work together to manage concurrent transcription tasks.

ParallelConfig and Safety Limits

The ParallelConfig struct establishes conservative defaults to prevent resource exhaustion. It restricts workers to a maximum of 2 by default and never exceeds 4, while also defining memory budgets and retry policies:

• max_workers: Capped at 4 for safety
• memory_budget_mb: Per-worker memory allocation limit
• max_retries: Automatic retry attempts for transient failures

Source: ParallelConfig struct, lines 52-70

ChunkQueue State Management

The ChunkQueue tracks the lifecycle of every audio chunk through distinct states. It maintains separate collections for pending, processing, completed, and failed chunks, plus a dedicated retry queue for transient errors. When initialized via ChunkQueue::new(), it prepares these tracking collections for batch ingestion.

Source: ChunkQueue struct and new(), lines 93-100 and 59-68

Worker Isolation

Each Worker struct (lines 86-90) runs as an independent async task with its own WhisperEngine instance. The create_worker() function (lines 199-226) spawns these workers, ensuring complete isolation between transcription threads.

Resource-Aware Worker Scaling

Before processing begins, the system evaluates available CPU and memory to determine a safe concurrency level that protects the host from overload.

Dynamic Worker Creation

The start_processing() method (lines 44-63) queries the SystemMonitor to calculate a safe worker count via calculate_safe_worker_count() (lines 33-42), clamping the result to the configured maximum:

let safe_worker_count = self.calculate_safe_worker_count().await?;
self.spawn_workers(safe_worker_count, model_name).await?;

This approach ensures the processor never spawns more workers than the system can safely support, even if the user requests a higher number in ParallelConfig.

System Health Monitoring

The start_resource_monitoring() task (lines 72-89) runs continuously in the background, checking system constraints and automatically pausing processing when resources are constrained. When the monitor detects high load, it sets a shared pause flag (lines 84-105):

if !status.can_proceed { 
    *is_paused.write().await = true; 
} else if status.can_proceed && *is_paused.read().await { 
    *is_paused.write().await = false; 
}

This mechanism automatically pauses the batch pipeline (ProcessingPaused) when constraints are hit and resumes (ProcessingResumed) when resources become available again.

Isolated Model Inference

Unlike architectures that share a single model instance, Meetily's parallel processor gives each worker exclusive access to its own WhisperEngine to eliminate contention.

Per-Worker WhisperEngine Instances

The create_worker() function initializes a dedicated engine for each worker thread (lines 22-28):

let engine = WhisperEngine::new()?;
engine.load_model(&model_name).await?;
*engine_guard = Some(engine);

This design prevents model contention and allows workers to process chunks independently without blocking each other, though it requires sufficient memory to load multiple model instances simultaneously.

Queue-Driven Batch Processing

The processor accepts arbitrary batch sizes by loading all chunks into a pending queue before workers begin consumption, enabling true batch transcription workflows.

Batch Ingestion

When start_processing() receives a vector of audio chunks, it atomically assigns them to the pending queue (lines 64-71):

self.chunk_queue.write().await.pending = chunks;

Semaphore-Based Concurrency Control

A tokio::Semaphore ensures the system never exceeds the calculated safe worker count. Each worker must acquire a permit before processing (lines 16-18):

let _permit = semaphore.acquire().await?;

Workers pull chunks preferring items from the retry queue, move them to the processing list, and execute process_chunk_safely(). Completed results populate the completed collection, while failures enter the failed list or return to the retry queue up to max_retries times.

Event-Driven Observability

The processor emits strongly-typed events through an unbounded channel, allowing the UI layer to track transcription progress. The ProcessingEvent enum (lines 38-48) includes variants such as ChunkStarted, ChunkCompleted, WorkerStarted, and ResourceConstraint.

Listening for progress:

while let Some(event) = events.recv().await {
    match event {
        ProcessingEvent::ChunkCompleted(result) => {
            println!("Chunk {}: {}", result.chunk_id, result.text);
        },
        ProcessingEvent::ResourceConstraint(msg) => {
            eprintln!("Warning: {}", msg);
        },
        _ => {}
    }
}

This event stream makes the batch workflow observable without blocking the transcription pipeline.

Practical Implementation

Integrating the parallel processor into a Tauri command involves configuring safety limits, instantiating the processor, and managing lifecycle events according to the patterns found in frontend/src-tauri/src/lib.rs.

Starting a Batch Transcription Job

use frontend::src_tauri::src::whisper_engine::{
    parallel_processor::{ParallelProcessor, ParallelConfig},
    system_monitor::SystemMonitor,
};
use tokio::sync::RwLock;
use std::sync::Arc;

// Configure safety limits
let config = ParallelConfig {
    max_workers: 3,
    memory_budget_mb: 1024,
    ..Default::default()
};

let system_monitor = Arc::new(SystemMonitor::new()?);
let (mut processor, mut events) = ParallelProcessor::new(config, system_monitor)?;

// Load batch and start
let audio_chunks = vec![/* AudioChunk structs */];
processor.start_processing(audio_chunks, "large-v3".to_string()).await?;

Source: ParallelProcessor::new (lines 101-130) and start_processing (lines 44-52)

Lifecycle Controls

The processor supports graceful pause, resume, and shutdown operations:

processor.pause_processing().await;   // Halt all workers temporarily
processor.resume_processing().await;  // Continue processing
processor.stop_processing().await;    // Gracefully shut down the pool

Source: pause_processing, resume_processing, stop_processing (lines 15-27)

Summary

• Resource-aware scaling: The calculate_safe_worker_count() function and SystemMonitor ensure worker pools never exceed safe CPU/memory thresholds, automatically capping at 4 workers.
• Complete isolation: Each worker maintains its own WhisperEngine instance in create_worker(), eliminating model contention and thread-safety issues during parallel inference.
• Robust queue management: The ChunkQueue handles pending, processing, completed, and failed states with automatic retry logic for transient failures up to max_retries.
• Safety mechanisms: A tokio::Semaphore enforces concurrency limits, while start_resource_monitoring() pauses processing during resource constraints and resumes when capacity returns.
• Observable pipeline: Event-driven architecture via ProcessingEvent allows real-time monitoring of batch transcription progress through an unbounded channel.

Frequently Asked Questions

How does the parallel processor prevent system overload?

The processor checks system resources before spawning workers using calculate_safe_worker_count() in system_monitor.rs. It enforces hard limits through ParallelConfig (max 4 workers) and uses a semaphore to restrict concurrent execution. A background monitoring task automatically pauses transcription when CPU or memory constraints are detected, resuming only when resources become available.

Why does each worker use a separate WhisperEngine instance?

According to the source code in create_worker() (lines 22-28 and 199-226), each worker instantiates its own WhisperEngine and loads the model once at startup. This isolation prevents thread contention on the model state and allows true parallel execution without synchronization bottlenecks, though it requires additional memory per worker.

What happens when a chunk fails transcription?

Failed chunks enter a retry queue managed by the ChunkQueue structure. Workers preferentially pull from the retry queue before processing new pending chunks. If a chunk fails repeatedly beyond the max_retries limit defined in ParallelConfig, it moves to the failed collection and emits a ProcessingEvent for logging or manual intervention.

How do I monitor the progress of a batch transcription job?

The ParallelProcessor::new() constructor returns an unbounded event receiver alongside the processor instance. Spawn an async task to listen for ProcessingEvent variants such as ChunkCompleted, WorkerStarted, or ResourceConstraint (defined at lines 38-48). This pattern enables real-time UI updates and progress tracking without blocking the transcription pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}