How ODS Handles GPU Detection for NVIDIA, Intel, AMD, and ARM Hardware

ODS implements a hierarchical hardware detection algorithm in ods/installers/lib/detection.sh that probes for Jetson ARM devices, validates discrete NVIDIA GPUs via sysfs PCI vendor IDs, detects Intel Arc and AMD accelerators, and exports normalized environment variables for downstream orchestration.

The Osmantic/ODS (Open Data Services) installer relies on robust GPU detection to automatically configure the appropriate inference backend for diverse hardware environments. Understanding how ODS handles GPU detection across different platforms ensures administrators can predict behavior on edge devices, workstations, and cloud instances. The detection logic resides in a POSIX-compliant shell library that prioritizes hardware validation over tooling presence to eliminate false positives.

The Core Detection Architecture

The detection routine follows a strict priority order to prevent misclassification. It begins with a conservative CPU-only assumption and progressively tests for specific hardware signatures in ods/installers/lib/detection.sh. A macOS-specific wrapper at ods/installers/macos/lib/detection.sh forwards calls to this canonical script, ensuring consistent behavior across Unix-like platforms.

All probes export standardized environment variables—including GPU_BACKEND, GPU_MEMORY_TYPE, GPU_VRAM, GPU_NAME, and GPU_DEVICE_ID—for consumption by ods/installers/lib/tier-map.sh and Docker Compose configuration scripts.

NVIDIA Jetson and ARM-Based Detection

For NVIDIA Jetson devices, the detector performs multi-factor verification between lines 276-332.

The script searches for:

• L4T (Linux for Tegra) release files
• Device-tree markers
• The presence of /sys/devices/gpu.0

When identified, the detector sets GPU_BACKEND="jetson" and treats the accelerator as unified memory. Unlike discrete GPUs, VRAM calculation derives from total system RAM rather than dedicated video memory.

Discrete NVIDIA GPU Detection on Linux and WSL2

For discrete NVIDIA cards, the detection logic between lines 335-398 implements a two-stage validation to avoid false positives from systems with drivers installed but lacking actual hardware.

Sysfs PCI Validation

The script scans /sys/class/drm/card*/device/vendor for the PCI vendor ID 0x10de. This hardware-level check confirms physical GPU presence before invoking userspace tools.

WSL2 Compatibility

On Windows Subsystem for Linux, the sysfs tree may be absent. The detector falls back to checking the kernel release string for "microsoft" or "wsl" patterns, then uses nvidia-smi for device enumeration.

Unified Memory Detection

When nvidia-smi returns [N/A] for memory statistics—indicating Grace or Blackwell architectures with unified memory—the script maps system RAM to GPU_VRAM using the same logic applied to Jetson devices.

Intel Arc GPU Detection

Intel Arc GPU support is handled between lines 400-410. The detector executes lspci to identify VGA controllers with Intel signatures, then validates vendor ID 0x8086 against specific device ID ranges:

• 0x56a0 through 0x56c1
• 0x5690 through 0x569f

Upon successful validation, the script exports GPU_BACKEND="intel" and GPU_MEMORY_TYPE="discrete".

AMD GPU Support

Following the same architectural pattern, AMD detection inspects /dev/kfd and /dev/dri for kernel-mode driver devices. When present, the detector classifies the hardware as GPU_BACKEND="amd" with discrete memory allocation. This implementation ensures ODS can leverage ROCm-compatible accelerators without vendor-specific userspace dependencies during the initial probe.

Graceful Fallback to CPU-Only Mode

If all hardware probes fail, the detection logic at lines 519-525 emits a warning and retains the initial GPU_BACKEND="cpu" fallback. This guarantees installer continuity on servers without accelerators or in restricted containers, allowing the stack to deploy with CPU-based inference.

Consuming Detection Results in Installer Phases

Downstream components source the detection library to configure runtime environments.

Bash Integration

Installation phases invoke the detection routine directly:

source "$SCRIPT_DIR/lib/detection.sh"
detect_gpu || exit 1

echo "Detected backend: $GPU_BACKEND"
echo "GPU name: $GPU_NAME"
echo "VRAM (MiB): $GPU_VRAM"

Python Service Integration

The Dashboard API at ods/extensions/services/dashboard-api/helpers.py reads exported variables via standard library functions:

import os

gpu_backend = os.getenv("GPU_BACKEND", "cpu")
gpu_name = os.getenv("GPU_NAME", "CPU")
gpu_vram = int(os.getenv("GPU_VRAM", "0"))

print(f"Backend: {gpu_backend}, GPU: {gpu_name}, VRAM: {gpu_vram} MiB")

Docker Compose Selection

The resolve-compose-stack.sh script uses the backend variable to include hardware-specific overlays:

if [[ "$GPU_BACKEND" == "nvidia" ]]; then
    COMPOSE_FILES+=("$BASE_DIR/docker-compose.nvidia.yml")
elif [[ "$GPU_BACKEND" == "amd" ]]; then
    COMPOSE_FILES+=("$BASE_DIR/docker-compose.amd.yml")
fi

Summary

• ODS implements GPU detection in ods/installers/lib/detection.sh using a hierarchical probe strategy that starts with CPU fallback and validates specific hardware signatures.
• Jetson devices are identified via L4T files and /sys/devices/gpu.0 between lines 276-332, using system RAM as VRAM for unified memory architectures.
• Discrete NVIDIA GPUs require PCI vendor ID 0x10de validation in sysfs between lines 335-398, with WSL2 fallback to nvidia-smi, and special handling for Grace/Blackwell unified memory indicators.
• Intel Arc detection validates PCI vendor ID 0x8086 against device ID ranges 0x56a0-0x56c1 and 0x5690-0x569f at lines 400-410.
• AMD GPUs are detected via /dev/kfd and /dev/dri kernel device nodes with discrete memory classification.
• All results export as environment variables for consumption by tier-mapping scripts and Docker Compose configuration, with CPU fallback guaranteed at lines 519-525.

Frequently Asked Questions

How does ODS prevent false positives when NVIDIA drivers are installed but no GPU is present?

The detector validates physical hardware by reading /sys/class/drm/card*/device/vendor for PCI vendor ID 0x10de before invoking nvidia-smi. This sysfs check ensures the kernel recognizes actual NVIDIA hardware, preventing misconfiguration on systems with only driver utilities installed.

What happens when ODS runs on a Grace or Blackwell architecture with unified memory?

When nvidia-smi reports [N/A] for memory statistics—indicating unified memory architectures like Grace or Blackwell—the detection script treats the GPU as unified memory and derives GPU_VRAM from total system RAM, identical to the Jetson detection logic implemented in lines 335-398.

Can ODS detect GPUs inside Windows Subsystem for Linux (WSL2)?

Yes. When the sysfs PCI tree is unavailable, the script detects WSL2 environments by matching "microsoft" or "wsl" in the kernel release string, then falls back to nvidia-smi for GPU enumeration and capability extraction.

Where does ODS store the detected GPU information for downstream services?

The detect_gpu function exports standardized environment variables including GPU_BACKEND, GPU_NAME, GPU_VRAM, GPU_DEVICE_ID, and GPU_MEMORY_TYPE. These variables are consumed by ods/installers/lib/tier-map.sh for model selection and scripts/resolve-compose-stack.sh for Docker Compose overlay inclusion.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}