# How the Whisper Engine in Meetily Loads and Manages Transcription Models with GPU Acceleration (Metal/CUDA/Vulkan)

> Explore how Meetily's Whisper engine loads and manages transcription models using GPU acceleration. Learn about Metal, CUDA, and Vulkan integration for enhanced performance.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: internals
- Published: 2026-07-06

---

**Meetily’s Whisper engine uses a dual-layer detection strategy—compile-time Cargo features select the GPU backend (Metal, CUDA, Vulkan, or HIPBLAS), while runtime hardware probing determines device capabilities and performance tiers, automatically configuring `whisper_rs` with GPU acceleration parameters when loading transcription models.**

The Meetily transcription pipeline, built on the `whisper_rs` crate, dynamically adapts to heterogeneous hardware by combining static feature flags with runtime GPU detection. Whether running on macOS with Metal, Linux with CUDA, or cross-platform Vulkan, the engine automatically selects optimal acceleration parameters, including flash attention support for compatible high-performance GPUs.

## Compile-Time Backend Selection via Cargo Features

The engine first determines which GPU backend was compiled into the binary through the `WhisperCompiledBackend` enum in [`frontend/src-tauri/src/whisper_engine/acceleration.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/acceleration.rs). The `current()` method inspects Cargo feature flags to identify the available acceleration target:

```rust
// frontend/src-tauri/src/whisper_engine/acceleration.rs
impl WhisperCompiledBackend {
    pub fn current() -> Self {
        if cfg!(feature = "cuda")   { Self::Cuda   }
        else if cfg!(feature = "vulkan") { Self::Vulkan }
        else if cfg!(feature = "hipblas") { Self::HipBlas }
        else if cfg!(target_os = "macos") || cfg!(feature = "metal") {
            Self::Metal
        } else { Self::Cpu }
    }
}

```

This compile-time check prioritizes CUDA and Vulkan features explicitly, falls back to Metal on macOS or when the `metal` feature is enabled, and defaults to CPU if no GPU features are active. The selected backend remains fixed for the application lifecycle.

## Runtime GPU Capability Detection

After establishing the compiled backend, the engine probes the actual hardware capabilities through `HardwareProfile::detect()` in [`frontend/src-tauri/src/audio/hardware_detector.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/audio/hardware_detector.rs). This function returns a `GpuType` enum (`Metal`, `Cuda`, `Vulkan`, or `None`) alongside a `PerformanceTier` classification (`Low`, `Medium`, `High`, or `Ultra`).

The `WhisperEngine::detect_gpu_acceleration()` method logs the detection results:

```rust
fn detect_gpu_acceleration() -> bool {
    match WhisperCompiledBackend::current() {
        WhisperCompiledBackend::Metal => { 
            log::info!("macOS detected – enabling Metal"); 
            true 
        },
        WhisperCompiledBackend::Cuda  => { 
            log::info!("CUDA feature enabled – enabling GPU"); 
            true 
        },
        WhisperCompiledBackend::Vulkan => { 
            log::info!("Vulkan feature enabled – enabling GPU"); 
            true 
        },
        _ => { 
            log::info!("No GPU acceleration – using CPU"); 
            false 
        },
    }
}

```

This runtime validation ensures the engine only attempts GPU initialization when both the compiled feature and physical hardware are present.

## Configuring Acceleration Parameters

The `whisper_context_acceleration_for()` function bridges compile-time flags and runtime detection to produce a `WhisperContextAcceleration` struct. Located in [`frontend/src-tauri/src/whisper_engine/acceleration.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/acceleration.rs), this logic determines whether to enable GPU inference and flash attention:

```rust
// frontend/src-tauri/src/whisper_engine/acceleration.rs
pub fn whisper_context_acceleration_for(
    compiled_backend: WhisperCompiledBackend,
    runtime_detected_gpu: GpuType,
    performance_tier: PerformanceTier,
) -> WhisperContextAcceleration {
    let use_gpu = !matches!(compiled_backend, WhisperCompiledBackend::Cpu);
    let fast_tier = matches!(performance_tier, PerformanceTier::High | PerformanceTier::Ultra);
    let flash_attn = matches!(compiled_backend, WhisperCompiledBackend::Metal | WhisperCompiledBackend::Cuda) 
        && fast_tier;
    
    WhisperContextAcceleration { 
        compiled_backend, 
        runtime_detected_gpu, 
        use_gpu, 
        flash_attn, 
        gpu_device: 0 
    }
}

```

**Flash attention**—a memory-efficient attention kernel—activates only when using Metal or CUDA backends on High or Ultra tier hardware. The function defaults to device index `0` for multi-GPU systems.

## Loading Models with GPU Context

When `WhisperEngine::load_model()` in [`frontend/src-tauri/src/whisper_engine/whisper_engine.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/whisper_engine/whisper_engine.rs) initializes a transcription model, it orchestrates the hardware detection and parameter construction:

```rust
let hardware_profile = crate::audio::HardwareProfile::detect();
let adaptive_config = hardware_profile.get_whisper_config();
let acceleration = whisper_context_acceleration_for(
    WhisperCompiledBackend::current(),
    hardware_profile.gpu_type,
    hardware_profile.performance_tier,
);

let context_param = WhisperContextParameters {
    use_gpu: acceleration.use_gpu,
    gpu_device: acceleration.gpu_device,
    flash_attn: acceleration.flash_attn,
    ..Default::default()
};

```

The engine then instantiates the whisper context with these parameters:

```rust
log::info!(
    "Whisper acceleration decision: compiled_backend={} runtime_detected_gpu={:?} use_gpu={} flash_attn={} gpu_device={}",
    acceleration.compiled_backend.as_str(),
    acceleration.runtime_detected_gpu,
    acceleration.use_gpu,
    acceleration.flash_attn,
    acceleration.gpu_device,
);

// Load the model with GPU parameters
let context = WhisperContext::new_with_params(&model_path, context_param)?;

```

If `use_gpu` evaluates to `false`—either because the compiled backend is `Cpu` or runtime detection returns `GpuType::None`—the model runs entirely on CPU threads without GPU memory allocation.

## Cross-Platform Build Configuration

Enable specific GPU backends by modifying the `features` array in your [`Cargo.toml`](https://github.com/Zackriya-Solutions/meetily/blob/main/Cargo.toml). The engine supports conditional compilation for different operating systems:

```toml

# Cargo.toml (partial)

[features]
default = ["metal"]        # macOS default

cuda = []                  # Enable NVIDIA CUDA support

vulkan = []                # Enable Vulkan support for AMD/Intel GPUs

hipblas = []               # Enable AMD ROCm/HIP support

```

Compile with feature flags to target specific hardware:

```bash

# macOS with Metal (default)

cargo build

# Linux with NVIDIA CUDA

cargo build --features cuda

# Cross-platform Vulkan

cargo build --features vulkan

```

The model files (`.ggml` or `.bin` binaries listed in `WHISPER_MODEL_CATALOG` from [`frontend/src-tauri/src/config.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/config.rs)) remain identical across all backends; only the acceleration layer changes based on compilation flags.

## Summary

- **Compile-time detection** uses `cfg!` macros in [`acceleration.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/acceleration.rs) to identify enabled GPU features (Metal, CUDA, Vulkan, HIPBLAS).
- **Runtime hardware profiling** through `HardwareProfile::detect()` provides actual GPU availability and performance tiers.
- **Acceleration resolution** via `whisper_context_acceleration_for()` enables GPU inference and conditional flash attention for Metal/CUDA on High/Ultra tier devices.
- **Model loading** constructs `WhisperContextParameters` with `use_gpu`, `gpu_device`, and `flash_attn` flags before calling `WhisperContext::new_with_params()`.
- **Automatic fallback** to CPU occurs when compiled features are absent or hardware detection fails.

## Frequently Asked Questions

### How does Meetily choose between Metal, CUDA, and Vulkan?

Meetily selects the backend at **compile time** through Cargo feature flags checked by `WhisperCompiledBackend::current()`. If multiple features are enabled, the code prioritizes CUDA, then Vulkan, then HIPBLAS, then Metal. At **runtime**, the engine verifies that the compiled backend matches available hardware through `HardwareProfile::detect()`, ensuring the engine never attempts to initialize Metal on a Linux system or CUDA on a Mac.

### What triggers flash attention activation in the Whisper engine?

Flash attention activates only when two conditions are met: the **compiled backend** must be `Metal` or `Cuda` (enforced via pattern matching), and the **runtime performance tier** must be `High` or `Ultra` as determined by the hardware detector. This optimization, configured in `whisper_context_acceleration_for()`, reduces memory bandwidth during attention calculations but requires modern GPU architectures.

### Can I force CPU-only mode even if a GPU is detected?

Yes, by compiling without GPU features or ensuring the `WhisperCompiledBackend::current()` returns `Cpu`. The engine treats `WhisperCompiledBackend::Cpu` as a hard override—when this variant is active, `use_gpu` is set to `false` regardless of what `HardwareProfile::detect()` reports. Alternatively, you can modify the build configuration to exclude all GPU features (`cargo build` with no GPU features enabled).

### Where does Meetily store downloaded transcription models?

Models are stored in the configured `models_dir` path (typically a subdirectory of the application data directory) as defined in [`frontend/src-tauri/src/config.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/config.rs). The `WHISPER_MODEL_CATALOG` static map defines the supported model names (e.g., "base", "small", "large-v3") and their corresponding filenames. The same binary files work across all GPU backends; no separate model downloads are required for Metal versus CUDA inference.