Memory Usage Patterns When Loading Different Whisper Model Sizes in Meetily

Meetily loads Whisper models through memory-mapped files and dynamically allocated decoder buffers, with RAM consumption scaling from approximately 350 MiB for the small model to 2.5 GiB for large-v3 on CPU-only systems.

Meetily’s real-time transcription engine leverages OpenAI’s Whisper models through a Rust-based Tauri backend. Understanding the memory usage patterns when loading different Whisper model sizes in Meetily is critical for deploying the application across diverse hardware, from resource-constrained laptops to GPU-accelerated workstations.

How Meetily Loads Whisper Models

The WhisperEngine::load_model implementation in frontend/src-tauri/src/whisper_engine/whisper_engine.rs orchestrates model initialization. The process follows four distinct stages that determine the final memory footprint.

Memory-Mapping Model Binaries

When a model is requested, Meetily uses the memmap2 crate to create a read-only memory map of the .bin file on disk. This establishes the baseline RAM footprint equal to the uncompressed model size. The mapping is performed in WhisperEngine::load_model before any inference buffers are allocated.

Decoder and Context Buffer Allocation

After mapping the weights, Whisper’s inference runtime allocates internal tensors for the encoder, decoder, and attention mechanisms. These buffer sizes grow linearly with the model’s dimension (d_model), meaning larger variants require proportionally more working memory beyond the raw weight file.

GPU vs. CPU Backend Selection

Memory residency depends on the compilation features enabled:

• GPU-enabled builds (--features cuda or --features vulkan): Weights are transferred to GPU VRAM, leaving only a small staging buffer in system RAM
• CPU-only builds: All weights remain resident in system RAM, consuming roughly 3–4× the disk file size

Audio Buffer Overhead

Each transcription request creates temporary buffers for 30-second chunks of 48 kHz audio. These allocations remain modest (< 10 MiB) and do not significantly impact the overall memory profile compared to the model weights.

RAM and VRAM Footprint by Model Size

The following table shows approximate memory consumption for each supported model variant when running on CPU-only systems versus GPU-accelerated configurations:

Model	Approx. Disk Size	Approx. RAM Usage (CPU)	Approx. VRAM (GPU)
base	~142 MiB	~500 MiB	~300 MiB
small	~75 MiB	~350 MiB	~200 MiB
medium	~1.4 GiB	~1.2 GiB	~800 MiB
large-v3	~2.9 GiB	~2.5 GiB	~1.6 GiB

Values derived from model file sizes and Whisper’s runtime buffer overhead. Actual consumption varies by OS, Rust allocator behavior, and memmap2 copy-on-write settings.

Practical Implications for Deployment

Small and Base Models

These variants suit most laptops and low-end desktops. Even without GPU acceleration, they remain under 1 GiB of RAM, leaving sufficient headroom for the Tauri frontend, audio pipelines, and operating system processes.

Medium Model

The medium variant requires at least 2 GiB of free RAM. On memory-constrained systems, OS swapping introduces noticeable transcription latency. Enabling CUDA or Vulkan acceleration offloads weights to VRAM, dramatically reducing system RAM pressure.

Large-v3 Model

Optimized for high-accuracy transcription, large-v3 demands at least 4 GiB of system RAM or a GPU with 2 GiB+ VRAM. Loading this model on low-memory environments triggers an out-of-memory error returned by WhisperEngine::load_model, which the frontend surfaces to users via the Audio Metrics Dashboard implemented in frontend/src-tauri/src/audio/pipeline.rs.

Handling Memory Errors Programmatically

Meetily exposes model loading through Tauri commands and provides graceful fallback mechanisms. The following patterns demonstrate how to handle memory constraints in both the Rust backend and TypeScript frontend.

// Backend: Loading a model with error handling
#[tauri::command]
async fn load_whisper_model(app: AppHandle, model_name: String) -> Result<(), String> {
    match app.state::<WhisperEngine>().load_model(&model_name).await {
        Ok(_) => Ok(()),
        Err(e) => Err(format!("Failed to load {}: {}", model_name, e)),
    }
}

// Inside WhisperEngine::load_model (simplified)
pub async fn load_model(&self, name: &str) -> anyhow::Result<()> {
    let path = self.model_path(name)?;
    // Memory-map the model file – directly ties RAM usage to file size
    let mmap = unsafe { Mmap::map(&File::open(&path)?)? };
    // Allocate decoder context (scales with model dimension)
    let ctx = whisper::Context::new(&mmap)?;
    self.current_ctx.replace(Some(ctx));
    Ok(())
}

// Frontend: Adaptive model selection based on hardware
async function selectModel() {
  const hasGpu = await invoke<boolean>('detect_gpu_capability');
  const model = hasGpu ? 'large-v3' : 'small'; // Fallback for CPU-only
  
  try {
    await invoke('load_whisper_model', { modelName: model });
    console.log(`Loaded Whisper ${model}`);
  } catch (err) {
    console.warn('Memory constraint detected:', err);
    // Gracefully downgrade to base model
    await invoke('load_whisper_model', { modelName: 'base' });
  }
}

The SystemMonitor implementation in frontend/src-tauri/src/whisper_engine/system_monitor.rs tracks these allocations in real-time, exposing metrics to the UI when approaching memory limits.

Summary

• Memory mapping via memmap2 establishes the baseline footprint equal to the model file size (75 MiB to 2.9 GiB depending on variant)
• CPU-only builds consume roughly 3–4× the disk size in RAM due to encoder/decoder buffers, while GPU builds offload weights to VRAM
• Small and base models operate comfortably under 1 GiB, suitable for most consumer hardware
• Large-v3 requires 4 GiB+ RAM or dedicated GPU memory, with automatic fallback handling via WhisperEngine error propagation
• Runtime monitoring through system_monitor.rs and audio/pipeline.rs provides visibility into transcription memory pressure

Frequently Asked Questions

How does Meetily handle out-of-memory errors when loading large Whisper models?

When WhisperEngine::load_model fails due to insufficient memory, it returns a Result::Err propagated through the Tauri command layer to the frontend. The application displays a warning in the Audio Metrics Dashboard and prompts users to select a smaller model or enable GPU acceleration via CUDA or Vulkan features.

Can I run the large-v3 model on a laptop without a dedicated GPU?

Running large-v3 on integrated graphics or CPU-only systems requires at least 4 GiB of free system RAM. According to the memory usage patterns in Meetily’s source code, the model consumes approximately 2.5 GiB of RAM on CPU, leaving minimal headroom for the operating system and UI components. Systems with less memory will experience allocation failures or severe swapping degradation.

Why does the CPU RAM usage exceed the model file size by 3–4×?

Whisper’s inference engine allocates additional tensors for encoder states, decoder caches, and attention mechanisms beyond the weight file. The WhisperEngine implementation in whisper_engine.rs loads the model via memory mapping (equal to file size), but the whisper::Context initialization creates working buffers scaled by the model’s dimensionality, resulting in the multiplier effect observed in CPU deployments.

Where does Meetily monitor real-time memory usage during transcription?

The application tracks memory metrics in two locations: frontend/src-tauri/src/whisper_engine/system_monitor.rs exposes VRAM and system RAM utilization to the UI, while frontend/src-tauri/src/audio/pipeline.rs monitors chunk-level audio buffer allocations. These components feed the Audio Metrics Dashboard with live data during active transcription sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}