How Meetily Implements GPU Acceleration Using Metal, CUDA, and Vulkan

Meetily combines compile-time feature flags with runtime hardware detection to automatically enable GPU acceleration via Metal on macOS and CUDA or Vulkan on Windows and Linux, while automatically configuring Flash-Attention for high-tier GPUs.

Meetily's local transcription pipeline leverages the whisper_rs crate to deliver cross-platform GPU acceleration using Metal, CUDA, and Vulkan. The architecture separates backend selection into compile-time decisions controlled by Cargo features and runtime detection that inspects the host hardware to determine which GPU API is actually available.

Compile-Time Backend Selection via Cargo Features

The frontend/src-tauri/Cargo.toml defines optional features that compile in support for specific GPU APIs. These features map directly to corresponding flags in the underlying whisper-rs dependency:

# Cargo.toml (excerpt) – feature flags

[features]
metal = ["whisper-rs/metal"]
cuda  = ["whisper-rs/cuda"]
vulkan = ["whisper-rs/vulkan"]
hipblas = ["whisper-rs/hipblas"]
coreml = ["whisper-rs/coreml"]

When building the project (e.g., cargo build --features cuda), the specified backend is compiled into the binary. Platform-specific sections in Cargo.toml automatically enable sensible defaults: macOS targets default to metal (and coreml for Apple Silicon), while Windows and Linux builds default to openblas for CPU inference unless the user explicitly enables cuda, vulkan, or hipblas.

Runtime GPU Detection in hardware_detector.rs

At startup, Meetily interrogates the host system to determine which GPU type is actually present. The hardware_detector.rs module defines a GpuType enum that classifies the available hardware as Metal, Cuda, Vulkan, OpenCL, or None. The detection logic inspects the operating system, environment variables (CUDA_PATH, CUDA_HOME, VULKAN_SDK), and filesystem locations (/usr/lib/libvulkan.so, vulkan-1.dll, etc.) to construct a HardwareProfile:

use crate::audio::hardware_detector::HardwareProfile;

/// Returns a cached profile describing the host's CPU/GPU capabilities.
let profile = HardwareProfile::detect();
println!(
    "GPU: {:?}, cores: {}, tier: {:?}",
    profile.gpu_type, profile.cpu_cores, profile.performance_tier
);

This profile reports has_gpu_acceleration (boolean), gpu_type, and a performance_tier (ranging from Low to Ultra) derived from CPU core count and available memory.

Mapping Backends to Runtime Configuration

The frontend/src-tauri/src/whisper_engine/acceleration.rs module bridges the gap between compile-time features and runtime hardware. It defines the WhisperCompiledBackend enum (variants: Metal, Cuda, Vulkan, HipBlas, Cpu) and the helper function whisper_context_acceleration_for. This function combines the compiled backend, the runtime-detected GpuType, and the PerformanceTier to produce a WhisperContextAcceleration struct:

let use_gpu = !matches!(compiled_backend, WhisperCompiledBackend::Cpu);
let fast_tier = matches!(performance_tier, PerformanceTier::High | PerformanceTier::Ultra);
let flash_attn = match compiled_backend {
    WhisperCompiledBackend::Metal | WhisperCompiledBackend::Cuda => fast_tier,
    _ => false,
};

The resulting configuration indicates whether to use the GPU (use_gpu), whether to enable Flash-Attention (flash_attn), and which device index to target (gpu_device, currently hardcoded to 0 for the first available GPU).

Engine Initialization and Logging

When a WhisperEngine instance is created in frontend/src-tauri/src/whisper_engine/whisper_engine.rs, it calls detect_gpu_acceleration() to log the active backend and acceleration status. These logs are conditioned on compile-time feature flags, ensuring a macOS build automatically reports "Metal" while a Windows build without GPU features reports "CPU only":

use meetily::frontend::src_tauri::whisper_engine::WhisperEngine;

let engine = WhisperEngine::new()?; // logs which backend (Metal, CUDA, etc.) is active

Complete Implementation Flow

The full GPU acceleration lifecycle follows four distinct phases:

1. Build Phase – Cargo resolves feature flags (metal, cuda, vulkan) and compiles the corresponding whisper-rs backend into the binary.

2. Startup Phase – WhisperEngine::new() invokes detect_gpu_acceleration(), which queries the system hardware.

3. Detection Phase – hardware_detector::detect_gpu() returns the actual GpuType by checking for Metal support, CUDA libraries, or Vulkan SDK installation.

4. Configuration Phase – whisper_context_acceleration_for() synthesizes the compile-time backend, runtime GPU, and performance tier into a WhisperContextAcceleration struct that initializes the whisper-rs context with optimal settings.

Summary

• Compile-time features in Cargo.toml select which GPU backend (Metal, CUDA, Vulkan) is compiled into the binary.
• Runtime detection in hardware_detector.rs inspects the host to determine actual GPU availability via environment variables and library paths.
• Configuration mapping in acceleration.rs combines the compiled backend with detected hardware to enable GPU inference and Flash-Attention for high-tier devices.
• Engine logging in whisper_engine.rs confirms which acceleration path is active during initialization.
• The architecture gracefully falls back to CPU inference (OpenBLAS) when no compatible GPU is detected or no GPU feature is enabled.

Frequently Asked Questions

How does Meetily choose between Metal, CUDA, and Vulkan?

Meetily uses platform-specific defaults combined with compile-time feature flags. According to the source code in frontend/src-tauri/Cargo.toml, macOS builds automatically enable the metal feature, while Windows and Linux builds default to CPU-only (openblas) unless the user explicitly compiles with --features cuda or --features vulkan. At runtime, hardware_detector.rs verifies that the required libraries (e.g., CUDA_PATH or VULKAN_SDK) are actually present before attempting to use them.

What triggers Flash-Attention enablement in Meetily?

Flash-Attention is automatically enabled when two conditions are met: the compiled backend must be Metal or Cuda, and the runtime PerformanceTier must be High or Ultra. As implemented in whisper_engine/acceleration.rs, this optimization is disabled for Vulkan and CPU backends, and only activates when the hardware detection logic determines the GPU has sufficient compute capability.

Can Meetily run on machines without a dedicated GPU?

Yes. If hardware_detector.rs finds no GPU libraries or if the binary was compiled without GPU features, Meetily defaults to CPU inference using OpenBLAS. The WhisperCompiledBackend::Cpu variant ensures the transcription engine initializes successfully without GPU dependencies, though processing speed will be significantly slower than Metal, CUDA, or Vulkan accelerated paths.

How can developers force a specific GPU backend when building Meetily?

Developers can force a specific backend by passing feature flags to Cargo during the build process. For Metal acceleration on macOS, use cargo build --features metal. For NVIDIA GPUs, compile with cargo build --features cuda. For cross-platform Vulkan support, use cargo build --features vulkan. These flags ensure only the specified backend is compiled into the binary, reducing binary size and eliminating runtime ambiguity about which GPU API to use.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}