Meetily GPU Acceleration Backends: How to Enable Metal, CUDA, Vulkan, and HIP BLAS for Whisper

Meetily supports four GPU acceleration backends—Metal for Apple Silicon, CUDA for NVIDIA GPUs, Vulkan for AMD/Intel, and HIP BLAS for AMD ROCm—which are selected at compile time via Cargo feature flags and automatically validated at runtime through hardware detection routines.

Meetily is an open-source meeting transcription application from Zackriya-Solutions that leverages GPU acceleration to dramatically speed up OpenAI Whisper processing. The application implements a sophisticated backend selection system that determines the optimal acceleration method based on your hardware and build configuration. Understanding how Meetily detects and enables these GPU backends ensures you achieve maximum transcription throughput on macOS, Linux, or Windows systems.

Supported GPU Acceleration Backends in Meetily

Meetily can accelerate Whisper transcription using several GPU backends, each targeting specific hardware configurations. The available backends are determined by which Cargo features you enable during compilation.

Metal (Apple Silicon)

Metal is the default backend on macOS systems with Apple Silicon (aarch64 architecture). This backend utilizes Apple's Metal Performance Shaders for optimized neural network inference on M1, M2, and M3 chips. According to the source code in frontend/src-tauri/src/audio/hardware_detector.rs, Metal support is detected via a simple architecture check when the target operating system is macOS.

CUDA (NVIDIA GPUs)

CUDA acceleration is available for systems with NVIDIA GPUs that have proper CUDA drivers installed. The runtime detection mechanism in hardware_detector.rs checks for the presence of CUDA_PATH or CUDA_HOME environment variables, or validates the existence of /usr/local/cuda on Unix systems.

Vulkan (Cross-Platform)

Vulkan provides GPU acceleration for AMD GPUs, Intel integrated graphics, and any Vulkan-compatible hardware across Linux and Windows. Detection logic inspects the VULKAN_SDK environment variable or searches common library paths; on Windows, it specifically checks for the presence of vulkan-1.dll.

HIP BLAS (AMD ROCm)

HIP BLAS enables acceleration on AMD GPUs using the ROCm stack. This backend is selected at compile time when the hipblas feature flag is enabled, making it suitable for Linux systems with AMD Radeon or Instinct cards.

CPU Fallback

When no GPU feature flags are enabled, or when hardware detection fails to find compatible drivers, Meetily falls back to CPU-only processing via the standard Whisper implementation.

Compile-Time Backend Selection

The WhisperCompiledBackend::current() function in frontend/src-tauri/src/whisper_engine/acceleration.rs determines which backend the binary was built with by evaluating Cargo feature flags in the following priority order:

if cfg!(feature = "cuda") { Self::Cuda }
else if cfg!(feature = "vulkan") { Self::Vulkan }
else if cfg!(feature = "hipblas") { Self::HipBlas }
else if cfg!(target_os = "macos") || cfg!(feature = "metal") { Self::Metal }
else { Self::Cpu }

To build Meetily with a specific backend, enable the corresponding feature in your Cargo.toml or build command:

• CUDA: --features cuda
• Vulkan: --features vulkan
• HIP BLAS: --features hipblas
• Metal: Enabled by default on macOS or explicitly via --features metal

Runtime GPU Detection

When the application starts, HardwareProfile::detect() in frontend/src-tauri/src/audio/hardware_detector.rs executes detect_gpu() to probe the host system and confirm that the compiled backend can actually run.

Detection Logic by Backend

• Metal: Confirmed when ARCH == "aarch64" on macOS
• CUDA: Validates CUDA_PATH/CUDA_HOME environment variables or /usr/local/cuda directory existence
• Vulkan: Checks VULKAN_SDK environment variable or locates vulkan-1.dll on Windows

If detection fails, the system sets has_gpu_acceleration = false and gpu_type = GpuType::None, forcing the application to use CPU inference even if a GPU backend was compiled.

Enabling GPU Acceleration and Flash Attention

The function whisper_context_acceleration_for() in frontend/src-tauri/src/whisper_engine/acceleration.rs combines the compiled backend, runtime-detected GPU type, and a performance tier to determine the final acceleration configuration.

GPU Usage is enabled automatically unless WhisperCompiledBackend::Cpu is selected. Flash Attention—which reduces memory usage and increases speed—is enabled specifically for Metal and CUDA backends, but only when the hardware performance tier is High or Ultra:

let use_gpu = !matches!(compiled_backend, WhisperCompiledBackend::Cpu);
let fast_tier = matches!(performance_tier, PerformanceTier::High | PerformanceTier::Ultra);
let flash_attn = matches!(compiled_backend, WhisperCompiledBackend::Metal | WhisperCompiledBackend::Cuda) && fast_tier;

Note that Vulkan and HIP BLAS never enable flash attention, even when a compatible GPU is present.

Practical Implementation Examples

Checking the Detected Hardware Profile

To log the detected hardware capabilities from within the Rust codebase:

use crate::audio::HardwareProfile;

fn log_hardware() {
    let profile = HardwareProfile::detect();
    log::info!(
        "CPU cores: {}, GPU: {:?} (acceleration: {}), Memory: {} GB, Tier: {:?}",
        profile.cpu_cores,
        profile.gpu_type,
        profile.has_gpu_acceleration,
        profile.memory_gb,
        profile.performance_tier,
    );
}

Initializing Whisper with Appropriate Acceleration

To select the correct Whisper context based on hardware detection:

use crate::whisper_engine::{whisper_context_acceleration_for, WhisperCompiledBackend};
use crate::audio::{HardwareProfile, PerformanceTier};

fn create_whisper_context() {
    let hw = HardwareProfile::detect();
    let compiled = WhisperCompiledBackend::current();

    let accel = whisper_context_acceleration_for(
        compiled,
        hw.gpu_type,
        hw.performance_tier,
    );

    log::info!("Whisper acceleration: {}", accel.status_label());
    // Pass `accel` to the Whisper engine initializer
}

Summary

• Meetily supports four GPU acceleration backends: Metal (Apple Silicon), CUDA (NVIDIA), Vulkan (AMD/Intel), and HIP BLAS (AMD ROCm)
• Compile-time selection occurs via Cargo feature flags in WhisperCompiledBackend::current(), determining which GPU code paths are included in the binary
• Runtime detection in hardware_detector.rs validates the presence of required drivers and libraries through environment variable checks and file system inspection
• Flash Attention is automatically enabled for Metal and CUDA backends on High or Ultra performance tiers, but never for Vulkan or HIP BLAS
• CPU fallback occurs automatically when GPU detection fails or when no GPU features are compiled

Frequently Asked Questions

How do I check which GPU backend my Meetily build is using?

Meetily logs the detected hardware profile at startup, including the GPU type and acceleration status. You can also programmatically check by calling WhisperCompiledBackend::current() to see which feature flag was compiled, and HardwareProfile::detect() to verify runtime availability. If the compiled backend does not match your hardware (e.g., CUDA compiled but no NVIDIA drivers), the system will silently fall back to CPU processing.

Why is GPU acceleration not working even though I have an NVIDIA GPU?

GPU acceleration requires both compile-time support and runtime detection. Ensure you built Meetily with the --features cuda flag, and verify that the CUDA_PATH or CUDA_HOME environment variables point to your CUDA installation, or that /usr/local/cuda exists on Linux systems. The has_cuda_support() function in hardware_detector.rs must return true for the GPU to be utilized.

Does Meetily support multiple GPU backends in a single binary?

No, Meetily selects a single compiled backend at build time based on Cargo feature flag priority. The WhisperCompiledBackend::current() function evaluates flags in a strict order (CUDA, Vulkan, HIP BLAS, Metal), enabling only one GPU backend per binary. To use a different backend, you must recompile with the appropriate feature flags.

When should I use Vulkan instead of CUDA for Whisper transcription?

Use Vulkan when running on AMD or Intel GPUs that lack native CUDA support, or when targeting Linux systems where proprietary NVIDIA drivers are unavailable. While Vulkan provides GPU acceleration, note that it does not support Flash Attention optimization, so CUDA remains preferable on NVIDIA hardware for maximum performance. Vulkan serves as the best cross-platform option for non-NVIDIA, non-Apple systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}