# How to Use ODS with AMD GPUs: Automated ROCm Setup and Configuration Guide

> Easily use ODS with AMD GPUs. Our guide automates ROCm setup and configuration, simplifying GPU device node management for Docker containers. Get started now!

- Repository: [Osmantic/ODS](https://github.com/Osmantic/ODS)
- Tags: how-to-guide
- Published: 2026-09-02

---

**ODS automatically detects AMD hardware during installation by scanning `/sys/class/drm` for vendor ID `0x1002`, then configures the ROCm runtime and mounts GPU device nodes into Docker containers.**

The Osmantic/ODS repository streamlines GPU-accelerated inference by automatically configuring AMD ROCm support without manual Docker tuning. When you use ODS with AMD GPUs, the installer queries sysfs interfaces to classify your hardware as APU, discrete, or mixed, then applies the appropriate container overlays and environment variables.

## Prerequisites for AMD GPU Support

ODS requires a ROCm-compatible kernel and specific device nodes to enable GPU acceleration.

### Kernel and Driver Requirements

You need Linux kernel 6.0 or newer with the `amdgpu` and `amdkfd` modules loaded. Verify and load these modules:

```bash
sudo modprobe amdgpu
sudo modprobe amdkfd

```

Install the ROCm development stack:

```bash
sudo apt update && sudo apt install -y rocm-dkms rocm-dev

```

### Device Node Verification

Before running the installer, confirm the kernel created the required device nodes:

```bash
ls -l /dev/kfd /dev/dri/renderD*

```

These nodes must exist on the host filesystem. If you are running inside an LXD or LXC container, the installer will trigger `show_amd_gpu_device_guidance()` and fall back to CPU mode until you expose these devices to the container.

## How ODS Detects AMD Hardware

The detection logic resides in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh). The `detect_gpu()` function iterates through GPU vendors and identifies AMD hardware by reading vendor IDs from `/sys/class/drm`.

When the script encounters vendor ID `0x1002`, it performs the following classification using [`ods/installers/lib/amd-topo.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/amd-topo.sh):

- **APU**: Unified memory architecture with shared system RAM (measured via `mem_info_gtt_total`)
- **Discrete**: Dedicated VRAM detected via `mem_info_vram_total`
- **Mixed**: Hybrid configurations containing both integrated and discrete AMD GPUs

The script sets `GPU_BACKEND="amd"` and `GPU_MEMORY_TYPE` based on these readings, enabling ROCm-specific configuration paths.

## Installation and Configuration Process

To use ODS with AMD GPUs, execute the standard installer after satisfying the kernel prerequisites:

```bash
curl -fsSL https://raw.githubusercontent.com/Osmantic/ODS/main/install.sh | bash

```

The installer invokes [`ods/installers/phases/02-detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/phases/02-detection.sh), which writes the detected configuration to your `.env` file:

```bash
GPU_BACKEND=amd

```

The installer then merges the AMD-specific Docker Compose overlay. The resulting orchestration command becomes:

```bash
docker compose -f ods/docker-compose.base.yml \
               -f ods/docker-compose.amd.yml up -d

```

The [`ods/docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/ods/docker-compose.amd.yml) overlay substitutes standard container images for ROCm-enabled variants and mounts `/dev/kfd` and `/dev/dri/renderD*` into service containers like `llama-server`.

## Verifying AMD Backend Configuration

After installation, confirm successful GPU detection using the diagnostic utility:

```bash
ods doctor | grep GPU_BACKEND

```

You should see `GPU_BACKEND=amd` in the output, confirming that the stack uses ROCm rather than CUDA or CPU fallbacks.

Verify GPU accessibility from within running containers:

```bash
docker exec -it llama-server bash -c "lspci | grep -i amd"
docker exec -it llama-server bash -c "cat /sys/class/drm/card0/device/mem_info_vram_total"

```

## Troubleshooting AMD GPU Detection

If hardware detection fails, the installer falls back to CPU mode and logs warnings via `amd_gpu_missing_runtime_devices()`.

### Missing Device Nodes

When `/dev/kfd` or `/dev/dri/renderD*` are inaccessible, the detection script cannot proceed. This commonly occurs in unprivileged containers or when kernel modules are not loaded. Execute `modprobe amdgpu` and `modprobe amdkfd` on the host, then restart the installer.

### Hardware Classification Limitations

While ODS fully validates **AMD Strix Halo APUs** with unified memory, discrete RDNA GPUs may be classified as "mixed" mode. These cards function but receive limited validation compared to APU configurations. Consult [`ods/docs/HARDWARE-GUIDE.md`](https://github.com/Osmantic/ODS/blob/main/ods/docs/HARDWARE-GUIDE.md) for the current support matrix.

### Forcing AMD Backend Mode

For testing or recovery scenarios, manually specify the backend before running detection:

```bash
export GPU_BACKEND=amd
ods doctor

```

This forces [`ods/installers/phases/02-detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/phases/02-detection.sh) to apply ROCm configuration regardless of automated scan results.

## Summary

- ODS detects AMD GPUs automatically in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) by scanning `/sys/class/drm` for vendor ID `0x1002` and reading `mem_info_vram_total` and `mem_info_gtt_total`
- The installer sets `GPU_BACKEND=amd` and applies [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml) to enable ROCm container images and device mounts
- Prerequisites include kernel modules `amdgpu` and `amdkfd`, plus visible device nodes at `/dev/kfd` and `/dev/dri/renderD*`
- Run `ods doctor` to verify backend configuration and inspect detection logs
- Containerized environments may require host-level device passthrough or will fall back to CPU mode with guidance from `show_amd_gpu_device_guidance()`

## Frequently Asked Questions

### Does ODS support discrete AMD GPUs or only APUs?

ODS provides full validation for AMD Strix Halo APUs with unified memory. Discrete RDNA GPUs operate but may be classified as "mixed" mode by [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh), resulting in limited validation compared to APU configurations. Check [`ods/docs/HARDWARE-GUIDE.md`](https://github.com/Osmantic/ODS/blob/main/ods/docs/HARDWARE-GUIDE.md) for specific model support.

### Why does ODS fall back to CPU mode on my AMD system?

This occurs when the installer cannot access `/dev/kfd` or `/dev/dri/renderD*`. The `amd_gpu_missing_runtime_devices()` function logs warnings when running inside unprivileged LXC/LXD containers or when the `amdgpu` module is unloaded. Load the kernel modules on the host and ensure device nodes are visible before running the installer.

### How do I switch from CUDA to AMD ROCm after installing ODS?

Run `export GPU_BACKEND=amd` followed by `ods doctor` to force redetection. The installer phase in [`ods/installers/phases/02-detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/phases/02-detection.sh) will regenerate your `.env` file and apply the [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml) overlay during the next stack deployment.

### Are Docker flags required to expose AMD GPUs to containers?

No manual flags are necessary. When `GPU_BACKEND=amd` is set, the [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml) overlay automatically configures device mounts for `/dev/kfd` and render nodes, and selects ROCm-enabled container images without additional runtime parameters.