How to Build llama.cpp with CMake for Different Backends: A Complete Guide

Build llama.cpp with CMake for different hardware backends by passing -DGGML_<BACKEND>=ON configuration flags, which enables accelerator support in the root CMakeLists.txt before compiling with cmake --build.

The llama.cpp repository (ggml-org/llama.cpp) is a lightweight C++ inference library that supports diverse hardware through compile-time backend selection. By toggling GGML_* options during the CMake configuration phase, you can generate optimized binaries for NVIDIA GPUs, Apple Silicon, AMD accelerators, or standard CPUs without modifying source code.

Core CMake Build Flow

The project uses a standard out-of-source build pattern controlled by the root CMakeLists.txt. This file defines global options and delegates to backend-specific scripts in ggml/src/ggml-<backend>/CMakeLists.txt based on your flags.

The build process follows three steps:

1. Create a build directory to isolate generated files from source.
2. Configure with CMake using -D flags to enable your target backend (-DGGML_CUDA=ON, -DGGML_VULKAN=ON, etc.).
3. Compile using cmake --build with your preferred generator.

By default, only the CPU backend is enabled. Each backend's CMake script automatically detects dependencies, selects compute architectures, and defines the corresponding target library (e.g., ggml-cuda).

Backend-Specific CMake Options

You enable hardware acceleration by setting specific boolean options at configure time. Multiple backends can be enabled simultaneously (e.g., -DGGML_CUDA=ON -DGGML_VULKAN=ON), allowing runtime selection.

Backend	CMake Option	Required Additional Flags	Source Location
CPU	None (always built)	-DCMAKE_BUILD_TYPE=Release	Default in root CMakeLists.txt
CUDA (NVIDIA)	-DGGML_CUDA=ON	-DGGML_NATIVE=OFF (for portable binaries), -DCMAKE_CUDA_ARCHITECTURES="86;89"	ggml/src/ggml-cuda/CMakeLists.txt
Metal (Apple GPU)	-DGGML_METAL=ON	None (auto-enabled on macOS)	Metal backend CMake
Vulkan (Cross-vendor)	-DGGML_VULKAN=ON	-DGGML_METAL=OFF (if using MoltenVK on macOS)	Vulkan backend CMake
SYCL (Intel GPU)	-DGGML_SYCL=ON	See docs/backend/SYCL.md for toolkit requirements	SYCL backend CMake
HIP (AMD GPU)	-DGGML_HIP=ON	-DGPU_TARGETS=gfx1030 (architecture specific)	HIP backend CMake
MUSA (Moore Threads)	-DGGML_MUSA=ON	-DMUSA_ARCHITECTURES="21"	ggml/src/ggml-musa/CMakeLists.txt
BLAS (Accelerated CPU)	-DGGML_BLAS=ON	-DGGML_BLAS_VENDOR=OpenBLAS (or Intel, etc.)	BLAS detection CMake
CANN (Huawei Ascend)	-DGGML_CANN=ON	Requires CANN toolkit installation	docs/backend/CANN.md
ZenDNN (AMD CPU)	-DGGML_ZENDNN=ON	Downloads and compiles ZenDNN on first build	docs/backend/ZenDNN.md
WebGPU (Experimental)	-DGGML_WEBGPU=ON	Requires Dawn and Emscripten for Wasm	WebGPU backend CMake

CPU-only builds produce the most portable binaries but lack GPU acceleration. BLAS improves batch-processing throughput on CPUs without requiring a GPU backend.

Step-by-Step Build Examples

CPU-Only Static Build

For a lightweight binary without GPU dependencies:

cmake -B build -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j$(nproc)

NVIDIA CUDA Build

Native build (optimized for your local GPU only):

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

Portable build (runs on any CUDA GPU):

cmake -B build -DGGML_CUDA=ON -DGGML_NATIVE=OFF
cmake --build build --config Release

Explicit architecture targeting (for specific compute capabilities like Ampere and Ada):

cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86;89"
cmake --build build --config Release

Apple Metal Build

Metal support is enabled by default on macOS, but you can force it explicitly:

cmake -B build -DGGML_METAL=ON
cmake --build build --config Release

Vulkan Cross-Platform Build

For Intel, AMD, or NVIDIA GPUs via the Vulkan API:

cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release

On macOS using MoltenVK, disable Metal first:

cmake -B build -DGGML_METAL=OFF -DGGML_VULKAN=ON

Multi-Backend Configuration

Enable both CUDA and Vulkan to choose at runtime:

cmake -B build -DGGML_CUDA=ON -DGGML_VULKAN=ON
cmake --build build --config Release

AMD ROCm/HIP Build

Target specific AMD GPU architectures:

cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1030
cmake --build build --config Release

SYCL for Intel GPUs

cmake -B build -DGGML_SYCL=ON
cmake --build build --config Release

Moore Threads MUSA Build

Specify the compute capability version:

cmake -B build -DGGML_MUSA=ON -DMUSA_ARCHITECTURES="21"
cmake --build build --config Release

OpenBLAS Acceleration

Improve CPU performance for large batch sizes:

cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cmake --build build --config Release

Runtime Backend Selection

After building with multiple backends, the llama-cli and llama-server binaries support runtime device selection:

# List all compiled backends and available devices

build/bin/llama-cli --list-devices

# Use GPU acceleration (automatic selection)

build/bin/llama-cli -m model.gguf -ngl 99

# Force specific backend (e.g., Vulkan on a multi-backend build)

build/bin/llama-cli --device vulkan -m model.gguf -p "Hello"

# Force CPU-only inference despite GPU backends being compiled

build/bin/llama-cli -m model.gguf --n-gpu-layers 0

Set the HIP_VISIBLE_DEVICES environment variable at runtime to control AMD GPU visibility when using the HIP backend.

Summary

• Configure backends using -DGGML_<BACKEND>=ON flags passed to the root CMakeLists.txt before building.
• The ggml/src/ggml-cuda/CMakeLists.txt and similar files handle backend-specific logic like architecture detection.
• Enable multiple backends simultaneously to create portable binaries that work across different hardware.
• Use -DGGML_NATIVE=OFF when building CUDA binaries for distribution to other machines.
• Control device selection at runtime using --device, --list-devices, or -ngl flags without recompiling.

Frequently Asked Questions

Can I enable multiple GPU backends in a single build?

Yes. The CMake system supports enabling multiple backends simultaneously, such as -DGGML_CUDA=ON -DGGML_VULKAN=ON -DGGML_METAL=ON. The resulting binary will include all specified backends, and you can select between them at runtime using the --device flag or let the system choose automatically based on availability.

How do I target a specific NVIDIA GPU architecture with CUDA?

Pass the -DCMAKE_CUDA_ARCHITECTURES flag with a semicolon-separated list of compute capabilities. For example, -DCMAKE_CUDA_ARCHITECTURES="75;86;89" targets Turing (75), Ampere (86), and Ada Lovelace (89) architectures. If you omit this, the build system defaults to native detection unless you specify -DGGML_NATIVE=OFF, which builds for a generic set of common architectures.

What is the difference between GGML_NATIVE and CMAKE_CUDA_ARCHITECTURES?

-DGGML_NATIVE=ON (the default) tells the compiler to optimize specifically for the CPU or GPU detected on the build machine, resulting in binaries that may not run on older hardware. -DCMAKE_CUDA_ARCHITECTURES explicitly lists which GPU generations to support. For maximum compatibility, set -DGGML_NATIVE=OFF and specify architectures explicitly; for maximum performance on a known target, keep -DGGML_NATIVE=ON.

How do I disable GPU acceleration after building with CUDA or Metal support?

Use the --n-gpu-layers 0 or --device none command-line argument when running llama-cli or llama-server. This forces inference on the CPU even if GPU backends are compiled into the binary, which is useful for testing or when GPU memory is constrained.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}