Parallel Whisper Processor Architecture in Meetily: High-Throughput Batch Transcription Design

The parallel Whisper processor is a resource-aware subsystem in Meetily's Tauri backend that orchestrates multiple isolated WhisperEngine workers to process audio chunks concurrently while preventing system overload through dynamic resource monitoring and hard safety limits.

Meetily implements a sophisticated batch transcription pipeline designed to handle large-scale audio processing without compromising system stability. The parallel Whisper processor architecture balances GPU utilization and CPU parallelism against memory safety constraints, making it suitable for desktop environments where resource contention must be carefully managed.

Core System Components

The architecture consists of specialized structures that manage worker lifecycles, queue state, and resource constraints.

ParallelProcessor Orchestrator

The ParallelProcessor struct serves as the central coordinator, defined in frontend/src-tauri/src/whisper_engine/parallel_processor.rs. It maintains a configurable worker pool, an event channel for UI communication, a ChunkQueue for task management, and integration with the SystemMonitor for resource-aware throttling. The constructor enforces a hard cap of 4 concurrent workers regardless of user configuration to prevent overwhelming the host system.

Isolated Worker Instances

Each Worker runs inside its own Tokio task and owns a dedicated WhisperEngine instance. Workers repeatedly pull chunks from the shared queue and execute transcription jobs. Isolating engine instances prevents contention over internal C-library state (whisper.cpp/whisper-rs), enabling true parallel GPU utilization when compiled with OpenMP, CUDA, or Vulkan support.

ChunkQueue State Management

The ChunkQueue struct tracks pending, in-flight, completed, and failed chunks using an Arc<RwLock<>> pattern for thread-safe access. It also maintains a separate retry queue for recoverable errors, allowing transient failures (such as brief GPU hiccups) to be reprocessed without losing the entire batch.

ProcessingEvent Communication

A lightweight ProcessingEvent enum flows through an unbounded mpsc channel to inform the frontend of state changes. Events include ChunkStarted, ChunkCompleted, WorkerStopped, and ResourceConstraint, enabling real-time progress updates and resource warnings in the UI.

ParallelConfig Safety Limits

The ParallelConfig struct exposes user-tunable parameters including max_workers, memory_budget_mb, max_retries, retry_delay_ms, and resource_check_interval_ms. While users can request specific worker counts, the processor constructor overrides values exceeding 4 to maintain safety boundaries.

SystemMonitor Integration

Located in frontend/src-tauri/src/whisper_engine/system_monitor.rs, the SystemMonitor periodically queries OS metrics for CPU load, memory pressure, and GPU usage. The processor invokes check_resource_constraints before spawning workers and continuously monitors conditions via start_resource_monitoring to auto-pause processing when resources are constrained.

Step-by-Step Processing Workflow

Understanding the execution flow reveals how the architecture maintains safety while maximizing throughput.

1. Initialization – The frontend invokes initialize_parallel_processor via Tauri command, passing a ParallelConfig. The constructor validates the configuration, caps workers at 4, and initializes the semaphore and event channels (lines 8–18).

2. Resource Validation – Before spawning workers, start_processing calls SystemMonitor::check_resource_constraints. If the system reports insufficient memory or high CPU load, the processor returns an error immediately without allocating resources (lines 53–60).

3. Queue Population – The caller provides a vector of AudioChunk structs containing raw Vec<f32> audio data, sample rate, timing metadata, and duration. The processor populates the ChunkQueue and resets state flags (is_paused, is_stopped) to ensure clean execution state (lines 64–72).

4. Worker Calculation and Spawning – The processor computes a safe worker count via calculate_safe_worker_count based on both the monitor's assessment and the configured maximum. It then spawns workers via spawn_workers and create_worker (lines 88–95, 99–124).

5. Per-Worker Execution – Each worker performs:

   • Model Loading: Creates a fresh WhisperEngine, loads the requested model (engine.load_model(&model_name)), and stores it in an Arc<RwLock<Option<WhisperEngine>>> (lines 22–30).
   • Main Processing Loop: Checks stop flags, respects pause states, prioritizes the retry queue over pending queue, and calls process_chunk_safely with a 2-minute timeout using WhisperEngine::transcribe_audio (lines 31–104).
   • Event Emission: Updates queue status and emits ChunkCompleted or ChunkFailed events upon finish (result handling lines 70–84).

6. Error Handling and Retry Logic – When process_chunk_safely returns an error, the worker constructs a ProcessingError. If retry count is below max_retries, the chunk enters the retry queue; otherwise it moves to the failed state (lines 128–166).

7. Background Resource Monitoring – An independent task (start_resource_monitoring) polls the SystemMonitor at resource_check_interval_ms intervals. Upon detecting constraints, it emits a ResourceConstraint event and auto-pauses processing; it auto-resumes when metrics recover (lines 72–106).

8. Lifecycle Management – The UI can trigger pause_parallel_processing, resume_parallel_processing, or stop_parallel_processing at any time. Stopping forces awaiting all worker handles and clearing the pool (lines 115–132).

Safety Mechanisms and Performance Optimization

The architecture implements multiple safeguards to prevent common failure modes in batch transcription.

Memory Budgeting – ParallelConfig.memory_budget_mb reserves a per-worker RAM allocation, preventing out-of-memory errors when loading large Whisper models (e.g., large-v3) simultaneously.

Timeout Protection – Each chunk execution is bounded by a 2-minute timeout, ensuring that hung transcriptions ( corrupt audio, infinite loops in the C++ backend ) cannot stall the entire batch indefinitely.

Dynamic Resource Throttling – Unlike static worker pools, the processor adjusts behavior based on real-time CPU, memory, and GPU metrics. When thresholds are exceeded, processing pauses automatically; when resources free up, transcription resumes without user intervention.

Engine Isolation – By giving each worker its own WhisperEngine instance ( implemented in frontend/src-tauri/src/whisper_engine/whisper_engine.rs ), the system avoids global lock contention and allows the underlying whisper.cpp to utilize parallel compute capabilities fully.

Implementation Code Examples

Initializing the Processor

import { invoke } from '@tauri-apps/api/tauri';
import { listen } from '@tauri-apps/api/event';

// Create a processor with a custom configuration
await invoke('initialize_parallel_processor', {
  config: {
    max_workers: 3,
    memory_budget_mb: 1024,
    max_retries: 2,
    retry_delay_ms: 1500,
    enable_fallback_mode: true,
  },
});

// Listen for processing events
await listen('processing-event', (e) => {
  const ev = e.payload as any;
  switch (ev.type) {
    case 'ChunkCompleted':
      console.log(`Chunk ${ev.chunk_id} → ${ev.text}`);
      break;
    case 'ResourceConstraint':
      alert(`Resources low: ${ev.message}`);
      break;
  }
});

Starting Batch Transcription

import { readFile } from '@tauri-apps/api/fs';

// Load audio chunks metadata
const chunks = await readFile('audio_chunks.json', { encoding: 'utf8' });

// Start processing with a specific model from WHISPER_MODEL_CATALOG
await invoke('start_parallel_processing', {
  chunks: JSON.parse(chunks),
  model_name: 'small'
});

Controlling Execution Flow

// Pause processing to free resources for other tasks
await invoke('pause_parallel_processing');

// Resume when ready
await invoke('resume_parallel_processing');

// Stop and cleanup all workers
await invoke('stop_parallel_processing');

Querying Processing Status

const status = await invoke('get_parallel_processing_status');
console.log('Pending:', status.pending_chunks);
console.log('Processing:', status.processing_chunks);
console.log('Completed:', status.completed_chunks);
console.log('Failed:', status.failed_chunks);

Key Source Files Reference

File	Purpose
frontend/src-tauri/src/whisper_engine/parallel_processor.rs	Core orchestration logic implementing ParallelProcessor, Worker, ChunkQueue, and ProcessingEvent. Contains start_processing, spawn_workers, and resource monitoring loops.
frontend/src-tauri/src/whisper_engine/whisper_engine.rs	Wrapper around whisper-rs providing model loading (load_model) and transcription (transcribe_audio).
frontend/src-tauri/src/whisper_engine/parallel_commands.rs	Tauri command implementations: initialize_parallel_processor, start_parallel_processing, pause_parallel_processing, etc.
frontend/src-tauri/src/whisper_engine/system_monitor.rs	SystemMonitor implementation for CPU, memory, and GPU metrics collection.
frontend/src-tauri/src/lib.rs	Tauri application setup registering parallel processor commands from whisper_engine::parallel_commands.
frontend/src-tauri/src/audio/transcription/worker.rs	Higher-level transcription worker that may delegate batch jobs to the ParallelProcessor.

Summary

• The parallel Whisper processor architecture in Meetily uses a worker pool pattern with isolated WhisperEngine instances per worker to maximize GPU utilization while avoiding state contention.
• Safety limits include a hard cap of 4 workers, configurable memory budgets per worker, and 2-minute timeouts on transcription operations.
• Resource awareness is achieved through the SystemMonitor integration, which auto-pauses processing when CPU, memory, or GPU constraints are detected and resumes when conditions improve.
• Retry logic with configurable max_retries and retry_delay_ms provides resilience against transient failures without manual intervention.
• Event-driven communication over Tauri's mpsc channels keeps the frontend synchronized with batch progress, error states, and resource warnings in real-time.

Frequently Asked Questions

How does the parallel Whisper processor prevent system overload?

The processor implements multiple defense layers: the constructor caps workers at 4 maximum regardless of configuration, the SystemMonitor validates resource availability before spawning workers and auto-pauses processing when CPU, memory, or GPU thresholds are exceeded, and ParallelConfig enforces per-worker memory budgets. These mechanisms collectively prevent out-of-memory errors and CPU starvation in desktop environments.

What happens if an audio chunk fails to transcribe?

Failed chunks trigger the error handling logic in process_chunk_safely. The system constructs a ProcessingError and checks the retry count against max_retries. If below the limit, the chunk moves to the retry queue with an incremented counter; otherwise, it is marked as permanently failed and reported via the ProcessingEvent channel. This ensures transient GPU hiccups or corrupt audio segments do not terminate the entire batch.

Can I configure the number of parallel workers?

Yes, through ParallelConfig.max_workers, but the system enforces a hard safety limit of 4 workers in the ParallelProcessor constructor. Even if you request 8 workers, the processor will override this to 4 to prevent overwhelming desktop systems. The actual number spawned may be further reduced if SystemMonitor reports resource constraints during calculate_safe_worker_count.

How does the frontend communicate with the parallel processor?

Communication occurs through Tauri commands and events. The frontend invokes commands such as initialize_parallel_processor and start_parallel_processing defined in parallel_commands.rs, while the Rust backend emits ProcessingEvent enums over an unbounded mpsc channel. The frontend listens via listen('processing-event', ...) to receive real-time updates on chunk completion, resource constraints, and worker status changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}