How ODS Handles Different GPU Backends: NVIDIA, AMD, Apple Silicon, and Intel Arc Detection

ODS detects GPU vendors through a layered hardware-discovery pipeline in ods/installers/lib/detection.sh, normalizes findings into environment variables like GPU_BACKEND and GPU_VRAM, and merges vendor-specific Docker Compose overlays to ensure compatible service startup.

The Osmantic/ODS (Open-Source Distributed Stack) repository orchestrates AI workloads across heterogeneous hardware by abstracting GPU specifics behind a unified detection interface. Understanding how ODS handles different GPU backends allows operators to deploy on discrete NVIDIA cards, unified-memory Apple Silicon devices, and Intel Arc systems without manual configuration. The detection logic runs early in the installation process to configure the correct service manifests and backend contracts.

The Detection Pipeline in detection.sh

The central entry point for GPU discovery is the detect_gpu function in ods/installers/lib/detection.sh. This script executes a vendor-ordered series of checks to identify the hardware platform, characterize memory architecture, and populate standardized variables used throughout the stack.

Apple Silicon Detection

Apple Silicon is detected first via the detect_apple helper in ods/scripts/detect-hardware.sh with a fallback inside detect_gpu. When the script identifies macOS ARM64 architecture, it sets GPU_BACKEND="apple" and calculates VRAM as a portion of system unified memory. For Apple Silicon, the installer later loads docker-compose.apple.yml through the resolve-compose-stack.sh script.

NVIDIA GPU Discovery

For NVIDIA hardware, detect_gpu scans sysfs for PCI vendor ID 0x10de. If sysfs entries are absent but the host runs WSL2, the function falls back to executing nvidia-smi. When present, the parser extracts the GPU name, device ID, and VRAM size directly from the SMI output.

Unified-memory NVIDIA GPUs—such as Blackwell architecture cards—trigger a special condition when nvidia-smi reports [N/A] for memory. In this case, the detection logic substitutes system RAM for GPU_VRAM and sets GPU_MEMORY_TYPE="unified".

Intel Arc Support

Intel Arc detection greps lspci output for the pattern VGA.*Intel.*Arc while confirming vendor ID 0x8086 and known Arc device ID ranges. Because Intel Arc drivers expose local memory differently than discrete cards, the script checks for lmem_total_bytes in sysfs to determine available VRAM. When successful, GPU_BACKEND is set to "intel".

AMD GPU Identification

AMD detection scans /sys/class/drm/*/device/vendor for 0x1002. The logic distinguishes between discrete GPUs, APUs, and mixed-mode systems by analyzing the relationship between VRAM and GTT (Graphics Translation Table) size reported in sysfs. Based on this analysis, the script sets GPU_BACKEND="amd" and classifies the memory type as discrete, unified, or mixed.

Normalized Environment Variables

After detection completes, ods/installers/lib/detection.sh exports a consistent set of variables consumed by downstream components such as classify-hardware.sh and load_backend_contract:

GPU_BACKEND       # one of: nvidia | amd | intel | apple | jetson | cpu

GPU_NAME          # human-readable model string

GPU_VRAM          # total VRAM in MB (or system RAM for unified memory)

GPU_COUNT         # number of detected GPUs

GPU_MEMORY_TYPE   # discrete | unified | mixed | none

GPU_DEVICE_ID     # PCI device ID (or L4T release for Jetson)

These variables drive compose-file selection and backend-contract loading without requiring subsequent scripts to implement vendor-specific parsing logic.

Backend-Specific Compose Overlays

The ods/scripts/resolve-compose-stack.sh script consumes the GPU_BACKEND variable to merge the base configuration with vendor-specific overlays. The core stack defined in docker-compose.base.yml is combined with overlays such as docker-compose.nvidia.yml, docker-compose.amd.yml, or installers/macos/docker-compose.macos.yml depending on the detected hardware.

For example, when gpu_backend == "apple", the script injects the Apple-specific overlay:

elif gpu_backend == "apple":
    APPLE_OVERLAY = "installers/macos/docker-compose.macos.yml"

This ensures that services incompatible with Apple Silicon unified memory are disabled unless explicitly enabled via the --gpu-backend apple flag.

Backend Contracts and Configuration

Following hardware detection, the load_backend_contract function (also in detection.sh) loads a JSON configuration from config/backends/<backend>.json. These contract files define per-backend defaults including model tier mappings, LLM engine selection, and GPU-layer limits. For instance, config/backends/nvidia.json contains CUDA-specific optimizations while config/backends/apple.json specifies Metal Performance Shaders (MPS) parameters.

Special Handling for Jetson and Unified Memory

Jetson (NVIDIA ARM) devices are detected as a separate jetson backend rather than generic NVIDIA. The detection logic identifies these via L4T (Linux for Tegra) release files and treats all memory as unified, using system RAM for GPU_VRAM calculations.

Unified-memory GPUs—including Apple Silicon, Jetson, and certain NVIDIA Blackwell configurations—trigger fallback logic that disables GPU-dependent services unless the user explicitly forces GPU mode. This prevents runtime errors on systems where traditional VRAM allocation does not exist.

Summary

• Detection Entry Point: The detect_gpu function in ods/installers/lib/detection.sh orchestrates vendor discovery via sysfs, lspci, and nvidia-smi.
• Vendor Identification: Apple uses detect_apple in ods/scripts/detect-hardware.sh; NVIDIA uses PCI ID 0x10de; Intel uses 0x8086 with lmem_total_bytes; AMD uses 0x1002 with VRAM/GTT analysis.
• Variable Normalization: Six standard environment variables (GPU_BACKEND, GPU_NAME, GPU_VRAM, GPU_COUNT, GPU_MEMORY_TYPE, GPU_DEVICE_ID) abstract hardware specifics.
• Compose Selection: ods/scripts/resolve-compose-stack.sh merges docker-compose.base.yml with backend-specific overlays based on the detected vendor.
• Configuration Contracts: JSON files in config/backends/ provide per-vendor service parameters and tier mappings.

Frequently Asked Questions

How can I manually test GPU detection before running the full installer?

You can source the detection library and run the function interactively to inspect the populated variables. From the repository root, execute:

source ods/installers/lib/detection.sh
detect_gpu
echo "Detected: $GPU_BACKEND ($GPU_NAME) with ${GPU_VRAM}MB"

This prints the backend, model name, and memory without triggering the full installation process.

Does ODS support Intel Arc GPUs with limited driver support?

Yes. The detection logic in ods/installers/lib/detection.sh specifically checks for lmem_total_bytes in sysfs to determine Intel Arc VRAM, accommodating drivers that do not report memory through standard legacy interfaces. If the field is absent, the script gracefully degrades to CPU-only mode.

What happens when ODS detects a unified-memory GPU like Apple Silicon?

When GPU_MEMORY_TYPE is set to unified (detected for Apple Silicon, Jetson, or NVIDIA Blackwell), ODS defaults to CPU-only service configurations to avoid compatibility issues. To force GPU utilization on these platforms, explicitly pass the --gpu-backend flag matching your hardware (e.g., --gpu-backend apple).

Where are the backend-specific Docker Compose overlays stored?

Vendor-specific overlays reside in the ods/ directory root as docker-compose.nvidia.yml, docker-compose.amd.yml, docker-compose.intel.yml, and within ods/installers/macos/ for docker-compose.macos.yml. The ods/scripts/classify-hardware.sh and resolve-compose-stack.sh scripts select the appropriate file based on the GPU_BACKEND variable determined during detection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}