How Meetily's Transcription Worker Pool Manages Concurrent Processing Tasks

Meetily's transcription worker pool processes audio chunks concurrently through a resource-aware ParallelProcessor that enforces a hard limit of four workers, dynamically adjusts capacity based on system resources, and automatically pauses processing when CPU or memory thresholds are exceeded.

The Meetily transcription worker pool is the core concurrency engine in the Meetily open-source meeting assistant. Built in Rust as part of the Tauri-based desktop application at frontend/src-tauri/src/whisper_engine/, this architecture prevents resource exhaustion while maximizing transcription throughput across CPU cores.

Resource-Aware Worker Initialization

The pool initializes with strict resource boundaries to prevent runaway CPU or memory usage.

Hard-Capped Semaphore Creation

When ParallelProcessor::new is invoked, it creates a Semaphore that caps concurrent workers at four, regardless of user configuration. The implementation deliberately hardens this limit in frontend/src-tauri/src/whisper_engine/parallel_processor.rs at lines 8-15:

let safe_max_workers = std::cmp::min(config.max_workers, 4);
let semaphore = Arc::new(Semaphore::new(safe_max_workers));

This guarantees that safe_max_workers never exceeds four, protecting the host system from overload even if the configuration requests more threads.

Dynamic System-Based Sizing

Before processing begins, the pool calculates an optimal worker count by querying the SystemMonitor. The ParallelProcessor::calculate_safe_worker_count method (lines 33-38 in parallel_processor.rs) delegates to system_monitor.calculate_safe_worker_count(), which evaluates available memory and CPU cores against a per-worker memory budget. The SystemMonitor::calculate_safe_worker_count implementation in frontend/src-tauri/src/whisper_engine/system_monitor.rs (lines 69-85) ensures at least one worker always runs while capping the total at four.

Queue-Driven Task Distribution

The pool decouples task submission from execution using a thread-safe queue structure.

The ChunkQueue Structure

All incoming audio chunks are stored in a ChunkQueue that tracks four states: pending, processing, completed, and failed. Defined in parallel_processor.rs at lines 93-98, this structure enables the pool to detect completion and handle retries without losing work.

Worker Event Loop

Each worker runs an asynchronous loop defined in create_worker (lines 44-76 of parallel_processor.rs) that:

1. Acquires a semaphore permit from the Arc<Semaphore> created during initialization
2. Pulls the next chunk from the ChunkQueue (prioritizing retries)
3. Processes the chunk through an isolated WhisperEngine instance
4. Emits results via an mpsc::UnboundedSender<ProcessingEvent>

This architecture ensures workers block naturally when the semaphore is exhausted, preventing memory pressure from unbounded task spawning.

Concurrency Safety Mechanisms

The pool employs multiple strategies to ensure thread safety without contention.

Isolated State Management

Shared mutable state—including the chunk_queue, is_paused, and is_stopped flags—is protected behind Arc<RwLock<...>> primitives. Each worker maintains its own WhisperEngine instance wrapped in Arc<RwLock<Option<WhisperEngine>>> (lines 86-90 of parallel_processor.rs), eliminating model-loading contention during inference.

Semaphore-Based Backpressure

The entry point for each worker acquires a semaphore permit before processing (lines 15-18 of parallel_processor.rs), creating a natural backpressure mechanism that throttles work when the system is saturated.

Runtime Resource Monitoring

The pool actively monitors host health to prevent transcription tasks from destabilizing the system.

Automatic Pause and Resume

A background task (start_resource_monitoring) periodically polls SystemMonitor::check_resource_constraints. If memory exceeds 70% or CPU exceeds 80%, the processor emits a ProcessingEvent::ResourceConstraint and pauses all workers by setting the is_paused flag. Processing resumes automatically when resources return to safe levels. This logic is implemented in the resource-monitoring loop at lines 73-101 of parallel_processor.rs.

Graceful Shutdown

Clean termination is ensured through cooperative cancellation. The stop_processing method (lines 27-40 of parallel_processor.rs) sets the atomic is_stopped flag, causing each worker to exit its processing loop, and then awaits all worker handles to completion—preventing zombie threads and ensuring all ProcessingEvent channels close properly.

Implementation Example

Below is a complete example demonstrating how to configure and run the transcription worker pool:

// 1️⃣ Create a system monitor (uses default limits)
let system_monitor = Arc::new(create_system_monitor());

// 2️⃣ Configure parallel processing (max 2 workers, 512 MiB each)
let config = ParallelConfig {
    max_workers: 2,
    memory_budget_mb: 512,
    ..Default::default()
};

// 3️⃣ Build the processor
let (mut processor, mut events) = ParallelProcessor::new(config, system_monitor.clone())?;

// 4️⃣ Prepare audio chunks (already split elsewhere)
let chunks: Vec<AudioChunk> = …;

// 5️⃣ Start processing with a chosen Whisper model
processor.start_processing(chunks, "base".to_string()).await?;

// 6️⃣ Listen for events (e.g., completed transcriptions)
tokio::spawn(async move {
    while let Some(event) = events.recv().await {
        match event {
            ProcessingEvent::ChunkCompleted(res) => {
                println!("Chunk {} → {}", res.chunk_id, res.text);
            }
            ProcessingEvent::ResourceConstraint(msg) => {
                eprintln!("⚠️  Resource issue: {}", msg);
            }
            _ => {}
        }
    }
});

Summary

• Hard limits: The ParallelProcessor enforces a maximum of four concurrent workers via a Semaphore created in parallel_processor.rs, regardless of configuration.
• Dynamic sizing: Worker counts are calculated from real-time CPU and memory availability via SystemMonitor::calculate_safe_worker_count in system_monitor.rs.
• Safe concurrency: Arc<RwLock<...>> protects shared state, while isolated WhisperEngine instances per worker prevent model contention.
• Resource protection: Automatic pause/resume triggers when memory exceeds 70% or CPU exceeds 80%, as monitored by the background resource task.
• Graceful shutdown: The stop_processing method ensures all workers exit cleanly and await completion before terminating.

Frequently Asked Questions

How does Meetily prevent the transcription worker pool from consuming all system resources?

Meetily implements a hard cap of four workers in ParallelProcessor::new (lines 8-15 of parallel_processor.rs) using std::cmp::min(config.max_workers, 4) combined with a Semaphore. Additionally, a background monitor pauses processing when memory exceeds 70% or CPU exceeds 80%, resuming only when resources return to safe levels.

What happens when system resources are low during transcription?

When SystemMonitor::check_resource_constraints detects resource pressure, the processor emits a ProcessingEvent::ResourceConstraint and sets the is_paused flag. Workers check this flag in their event loop and idle until the monitor clears the pause condition, preventing transcription from destabilizing the host machine.

How does the ChunkQueue handle failed transcription attempts?

The ChunkQueue tracks failed chunks separately from pending, processing, and completed states. Workers prioritize retrying failed chunks over new ones when pulling work from the queue, ensuring transient failures (such as temporary memory pressure) do not result in lost audio segments.

Can I configure the worker pool to use more than four threads?

No. Even if you set max_workers higher in the ParallelConfig, the ParallelProcessor constructor explicitly clamps the value to four via safe_max_workers = std::cmp::min(config.max_workers, 4). This safety mechanism prevents configuration errors from overwhelming the system with transcription threads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}