# How ODS Handles Different GPU Backends: NVIDIA, AMD, and Apple Silicon Detection

> ODS automatically detects NVIDIA, AMD, and Apple Silicon GPUs during installation. Learn how ODS configures your system for optimal GPU performance.

- Repository: [Osmantic/ODS](https://github.com/Osmantic/ODS)
- Tags: internals
- Published: 2026-09-01

---

**ODS detects your GPU during the installation phase by scanning PCI vendor IDs and system paths, then sets environment variables that determine which container orchestration files, driver configurations, and resource limits to apply.**

The Osmantic/ODS (Open-Source Distributed Stack) repository automates GPU backend detection through a centralized shell library that identifies hardware from NVIDIA, AMD, and Apple Silicon before the stack deploys any containers. Understanding how ODS handles different GPU backends ensures you can troubleshoot detection failures and optimize resource allocation for your specific hardware.

## The Detection Pipeline

ODS populates a standardized set of environment variables during the `detect_gpu` phase in [[`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh)](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh). These variables drive backend-specific orchestration decisions:

- `GPU_BACKEND`: One of `nvidia`, `amd`, `intel`, `apple`, `jetson`, or `cpu` (fallback)
- `GPU_NAME`: Human-readable model name (e.g., *“RTX 4090”*)
- `GPU_VRAM`: Total video memory in MiB (or unified RAM for Apple/Jetson)
- `GPU_COUNT`: Number of GPUs detected
- `GPU_MEMORY_TYPE`: `discrete`, `unified`, `mixed`, or `none`
- `GPU_DEVICE_ID`: PCI device ID or unique identifier for Apple/Jetson
- `HAS_NPU`: Set to `true` for AMD systems with a Ryzen AI NPU

If no GPU is detected, the script falls back to CPU-only mode (`GPU_BACKEND=cpu`) and issues a warning (**line 119-126**).

## NVIDIA GPU Detection

The NVIDIA detection branch handles everything from legacy GTX cards to Blackwell architecture and Grace-Hopper unified memory systems.

### Hardware Identification

The script scans `/sys/class/drm/*/device/vendor` for the NVIDIA PCI vendor ID `0x10de` (**line 335-344**). On WSL2 environments where PCI paths may not exist, it falls back to checking for the presence of the `nvidia-smi` binary (**line 445-448**).

### VRAM and Multi-GPU Naming

Once identified, ODS executes `nvidia-smi --query-gpu=name,memory.total` (**line 511-518**) to populate `GPU_NAME` and `GPU_VRAM`. For systems reporting `0` or `[N/A]` VRAM—common in Grace-Hopper unified memory architectures—the script treats system RAM as video memory (**line 561-573**) and sets `GPU_MEMORY_TYPE=unified`.

For multi-GPU configurations, the script constructs concise display names like *“RTX 4090 × 2”* or *“RTX 4090 + RTX 4080”* (**line 581-595**).

### Blackwell and Secure Boot

Later installation phases invoke `fix_nvidia_secure_boot` (**line 166**) to verify that Blackwell GPUs use open kernel modules. If Secure Boot is enabled, ODS triggers a key-enrollment flow to ensure the NVIDIA driver loads properly.

## AMD GPU Detection

The AMD branch supports both discrete RDNA GPUs and integrated APUs with unified memory.

### Device Discovery and Classification

The detection logic scans `/sys/class/drm/*/device/vendor` for the AMD PCI vendor ID `0x1002` (**line 430-438**), collecting all matching card directories into `amd_card_dirs`. It then reads `mem_info_vram_total` and `mem_info_gtt_total` for each device (**line 446-452**) to distinguish between discrete GPUs (dedicated VRAM), APUs (unified memory), or mixed configurations (**line 562-670**).

### NPU Detection for Hybrid Inference

ODS checks for `/sys/class/misc/amdnpu` or AMD NPU entries in `lspci` (**line 706-709**). When found, it sets `HAS_NPU=true`, enabling Lemonade-hybrid mode for distributing workloads between GPU and NPU.

## Apple Silicon Detection

For macOS hosts, ODS delegates initial detection to [[`ods/scripts/detect-hardware.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/detect-hardware.sh)](https://github.com/Osmantic/ODS/blob/main/ods/scripts/detect-hardware.sh). The script runs only when the OS detection reports `macos` (**line 50-52**), then extracts the CPU brand via `sysctl -n machdep.cpu.brand_string` (**line 53**) and unified memory size via `hw.memsize` (**line 55-56**).

The function returns identifiers like *“Apple M1 Pro”* with *“16GB unified”*, causing the main detection script to set `GPU_BACKEND=apple` and `GPU_MEMORY_TYPE=unified`, treating system RAM as VRAM for container resource calculations.

## From Detection to Deployment

After identification, the `GPU_BACKEND` value drives several downstream configuration steps:

### Backend Contracts

The `load_backend_contract` function (**line 72-88**) maps the detected backend to a JSON configuration in `config/backends/`, specifying the LLM engine, driver versions, and container runtime requirements for NVIDIA, AMD, or Apple Silicon.

### Compose Overlay Selection

The [[`ods/scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/resolve-compose-stack.sh)](https://github.com/Osmantic/ODS/blob/main/ods/scripts/resolve-compose-stack.sh) script maps each backend to a specific Docker Compose overlay file (**line 151-158**):
- [`docker-compose.nvidia.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.nvidia.yml) for CUDA workloads
- [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml) for ROCm stacks
- [`docker-compose.apple.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.apple.yml) for Apple Silicon optimized containers

### Resource Tier Assignment

The [[`ods/installers/lib/tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/tier-map.sh)](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/tier-map.sh) module uses `GPU_VRAM` and `GPU_MEMORY_TYPE` to assign performance tiers, capping CPU usage appropriately when running in CPU-only fallback mode.

You can query the current detection status at any time:

```bash

# Query detected GPU status

$ ods gpu status
GPU Backend: nvidia
GPU Name: RTX 4090
VRAM: 24576 MB
Memory Type: discrete

```

For testing or forcing a specific backend:

```bash

# Force Apple Silicon backend for testing

$ GPU_BACKEND=apple ods gpu status
GPU Backend: apple
GPU Name: Apple M2 Pro
VRAM: 16000 MB
Memory Type: unified

```

## Summary

- ODS detects GPU hardware by scanning PCI vendor IDs (`0x10de` for NVIDIA, `0x1002` for AMD) and macOS system calls for Apple Silicon.
- The `detect_gpu` function in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) exports standardized environment variables that determine backend selection.
- NVIDIA detection includes special handling for Blackwell Secure Boot and unified memory (Grace-Hopper) configurations.
- AMD detection parses `mem_info_vram_total` to distinguish discrete GPUs from APUs and detects Ryzen AI NPUs via `/sys/class/misc/amdnpu`.
- Apple Silicon relies on `sysctl` calls to identify M1/M2/M3 chips and unified memory sizes.
- The detected `GPU_BACKEND` value selects Docker Compose overlays, backend contracts, and resource limits automatically.

## Frequently Asked Questions

### How does ODS detect GPUs in WSL2 environments where PCI paths are unavailable?

When `/sys/class/drm/*/device/vendor` does not exist, ODS falls back to checking for the `nvidia-smi` binary presence (**line 445-448** in [`detection.sh`](https://github.com/Osmantic/ODS/blob/main/detection.sh)). If found, it assumes an NVIDIA GPU and proceeds to query VRAM and device names through the SMI interface rather than PCI enumeration.

### Can ODS run on systems with multiple different GPU vendors installed simultaneously?

Currently, ODS selects a single `GPU_BACKEND` based on detection priority (NVIDIA, then AMD, then Apple). For mixed-GPU systems, the first detected vendor takes precedence. Multi-GPU support within the same vendor (e.g., two RTX 4090s) is fully supported via the `GPU_COUNT` variable and multi-GPU naming logic.

### What happens if my NVIDIA GPU reports 0 MB of VRAM during detection?

ODS interprets zero or `[N/A]` VRAM values as indicators of unified memory architectures like NVIDIA Grace or certain virtualized environments (**line 561-573**). It falls back to using system RAM as the VRAM value and sets `GPU_MEMORY_TYPE=unified`, ensuring container limits are still enforced correctly.

### Does ODS support AMD integrated graphics (APUs) without dedicated VRAM?

Yes. The AMD detection branch reads both `mem_info_vram_total` and `mem_info_gtt_total` to calculate available graphics memory (**line 446-452**). For APUs, it identifies the lack of dedicated VRAM and sets `GPU_MEMORY_TYPE=unified`, allowing the stack to run on Ryzen integrated graphics with appropriate memory constraints.