# How ODS Handles Different GPU Backends: NVIDIA, AMD, Apple Silicon, and Intel Arc Detection

> Discover how ODS detects and supports NVIDIA, AMD, Apple Silicon, and Intel Arc GPU backends. Learn about environment variable normalization and Docker Compose integration for seamless startup.

- Repository: [Osmantic/ODS](https://github.com/Osmantic/ODS)
- Tags: internals
- Published: 2026-08-30

---

**ODS detects GPU vendors through a layered hardware-discovery pipeline in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh), normalizes findings into environment variables like `GPU_BACKEND` and `GPU_VRAM`, and merges vendor-specific Docker Compose overlays to ensure compatible service startup.**

The Osmantic/ODS (Open-Source Distributed Stack) repository orchestrates AI workloads across heterogeneous hardware by abstracting GPU specifics behind a unified detection interface. Understanding how ODS handles different GPU backends allows operators to deploy on discrete NVIDIA cards, unified-memory Apple Silicon devices, and Intel Arc systems without manual configuration. The detection logic runs early in the installation process to configure the correct service manifests and backend contracts.

## The Detection Pipeline in detection.sh

The central entry point for GPU discovery is the `detect_gpu` function in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh). This script executes a vendor-ordered series of checks to identify the hardware platform, characterize memory architecture, and populate standardized variables used throughout the stack.

### Apple Silicon Detection

Apple Silicon is detected first via the `detect_apple` helper in [`ods/scripts/detect-hardware.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/detect-hardware.sh) with a fallback inside `detect_gpu`. When the script identifies macOS ARM64 architecture, it sets `GPU_BACKEND="apple"` and calculates VRAM as a portion of system unified memory. For Apple Silicon, the installer later loads [`docker-compose.apple.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.apple.yml) through the [`resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/resolve-compose-stack.sh) script.

### NVIDIA GPU Discovery

For NVIDIA hardware, `detect_gpu` scans sysfs for PCI vendor ID `0x10de`. If sysfs entries are absent but the host runs WSL2, the function falls back to executing `nvidia-smi`. When present, the parser extracts the GPU name, device ID, and VRAM size directly from the SMI output.

Unified-memory NVIDIA GPUs—such as Blackwell architecture cards—trigger a special condition when `nvidia-smi` reports `[N/A]` for memory. In this case, the detection logic substitutes system RAM for `GPU_VRAM` and sets `GPU_MEMORY_TYPE="unified"`.

### Intel Arc Support

Intel Arc detection greps `lspci` output for the pattern `VGA.*Intel.*Arc` while confirming vendor ID `0x8086` and known Arc device ID ranges. Because Intel Arc drivers expose local memory differently than discrete cards, the script checks for `lmem_total_bytes` in sysfs to determine available VRAM. When successful, `GPU_BACKEND` is set to `"intel"`.

### AMD GPU Identification

AMD detection scans `/sys/class/drm/*/device/vendor` for `0x1002`. The logic distinguishes between discrete GPUs, APUs, and mixed-mode systems by analyzing the relationship between VRAM and GTT (Graphics Translation Table) size reported in sysfs. Based on this analysis, the script sets `GPU_BACKEND="amd"` and classifies the memory type as `discrete`, `unified`, or `mixed`.

## Normalized Environment Variables

After detection completes, [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) exports a consistent set of variables consumed by downstream components such as [`classify-hardware.sh`](https://github.com/Osmantic/ODS/blob/main/classify-hardware.sh) and `load_backend_contract`:

```bash
GPU_BACKEND       # one of: nvidia | amd | intel | apple | jetson | cpu

GPU_NAME          # human-readable model string

GPU_VRAM          # total VRAM in MB (or system RAM for unified memory)

GPU_COUNT         # number of detected GPUs

GPU_MEMORY_TYPE   # discrete | unified | mixed | none

GPU_DEVICE_ID     # PCI device ID (or L4T release for Jetson)

```

These variables drive compose-file selection and backend-contract loading without requiring subsequent scripts to implement vendor-specific parsing logic.

## Backend-Specific Compose Overlays

The [`ods/scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/resolve-compose-stack.sh) script consumes the `GPU_BACKEND` variable to merge the base configuration with vendor-specific overlays. The core stack defined in [`docker-compose.base.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.base.yml) is combined with overlays such as [`docker-compose.nvidia.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.nvidia.yml), [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml), or [`installers/macos/docker-compose.macos.yml`](https://github.com/Osmantic/ODS/blob/main/installers/macos/docker-compose.macos.yml) depending on the detected hardware.

For example, when `gpu_backend == "apple"`, the script injects the Apple-specific overlay:

```bash
elif gpu_backend == "apple":
    APPLE_OVERLAY = "installers/macos/docker-compose.macos.yml"

```

This ensures that services incompatible with Apple Silicon unified memory are disabled unless explicitly enabled via the `--gpu-backend apple` flag.

## Backend Contracts and Configuration

Following hardware detection, the `load_backend_contract` function (also in [`detection.sh`](https://github.com/Osmantic/ODS/blob/main/detection.sh)) loads a JSON configuration from `config/backends/<backend>.json`. These contract files define per-backend defaults including model tier mappings, LLM engine selection, and GPU-layer limits. For instance, [`config/backends/nvidia.json`](https://github.com/Osmantic/ODS/blob/main/config/backends/nvidia.json) contains CUDA-specific optimizations while [`config/backends/apple.json`](https://github.com/Osmantic/ODS/blob/main/config/backends/apple.json) specifies Metal Performance Shaders (MPS) parameters.

## Special Handling for Jetson and Unified Memory

**Jetson (NVIDIA ARM)** devices are detected as a separate `jetson` backend rather than generic NVIDIA. The detection logic identifies these via L4T (Linux for Tegra) release files and treats all memory as unified, using system RAM for `GPU_VRAM` calculations.

**Unified-memory GPUs**—including Apple Silicon, Jetson, and certain NVIDIA Blackwell configurations—trigger fallback logic that disables GPU-dependent services unless the user explicitly forces GPU mode. This prevents runtime errors on systems where traditional VRAM allocation does not exist.

## Summary

- **Detection Entry Point:** The `detect_gpu` function in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) orchestrates vendor discovery via sysfs, `lspci`, and `nvidia-smi`.
- **Vendor Identification:** Apple uses `detect_apple` in [`ods/scripts/detect-hardware.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/detect-hardware.sh); NVIDIA uses PCI ID `0x10de`; Intel uses `0x8086` with `lmem_total_bytes`; AMD uses `0x1002` with VRAM/GTT analysis.
- **Variable Normalization:** Six standard environment variables (`GPU_BACKEND`, `GPU_NAME`, `GPU_VRAM`, `GPU_COUNT`, `GPU_MEMORY_TYPE`, `GPU_DEVICE_ID`) abstract hardware specifics.
- **Compose Selection:** [`ods/scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/resolve-compose-stack.sh) merges [`docker-compose.base.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.base.yml) with backend-specific overlays based on the detected vendor.
- **Configuration Contracts:** JSON files in `config/backends/` provide per-vendor service parameters and tier mappings.

## Frequently Asked Questions

### How can I manually test GPU detection before running the full installer?

You can source the detection library and run the function interactively to inspect the populated variables. From the repository root, execute:

```bash
source ods/installers/lib/detection.sh
detect_gpu
echo "Detected: $GPU_BACKEND ($GPU_NAME) with ${GPU_VRAM}MB"

```

This prints the backend, model name, and memory without triggering the full installation process.

### Does ODS support Intel Arc GPUs with limited driver support?

Yes. The detection logic in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) specifically checks for `lmem_total_bytes` in sysfs to determine Intel Arc VRAM, accommodating drivers that do not report memory through standard legacy interfaces. If the field is absent, the script gracefully degrades to CPU-only mode.

### What happens when ODS detects a unified-memory GPU like Apple Silicon?

When `GPU_MEMORY_TYPE` is set to `unified` (detected for Apple Silicon, Jetson, or NVIDIA Blackwell), ODS defaults to CPU-only service configurations to avoid compatibility issues. To force GPU utilization on these platforms, explicitly pass the `--gpu-backend` flag matching your hardware (e.g., `--gpu-backend apple`).

### Where are the backend-specific Docker Compose overlays stored?

Vendor-specific overlays reside in the `ods/` directory root as [`docker-compose.nvidia.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.nvidia.yml), [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml), [`docker-compose.intel.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.intel.yml), and within `ods/installers/macos/` for [`docker-compose.macos.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.macos.yml). The [`ods/scripts/classify-hardware.sh`](https://github.com/Osmantic/ODS/blob/main/ods/scripts/classify-hardware.sh) and [`resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/resolve-compose-stack.sh) scripts select the appropriate file based on the `GPU_BACKEND` variable determined during detection.