# How Project N.O.M.A.D. Automatically Detects and Utilizes NVIDIA GPUs for AI Acceleration

> Project N.O.M.A.D. automatically detects and utilizes NVIDIA GPUs for AI acceleration. Learn how to leverage your NVIDIA hardware with this innovative solution.

- Repository: [Crosstalk Solutions/project-nomad](https://github.com/Crosstalk-Solutions/project-nomad)
- Tags: how-to-guide
- Published: 2026-03-16

---

**Yes, Project N.O.M.A.D. automatically detects NVIDIA GPUs by querying the Docker daemon for the NVIDIA runtime and executing `nvidia-smi` inside the Ollama container to verify GPU accessibility.**

Project N.O.M.A.D. (Neural Operations Management and Deployment) is an open-source AI assistant platform that streamlines running large language models via Ollama in Docker containers. One of its key features is the ability to automatically detect and utilize NVIDIA GPUs for hardware-accelerated inference, eliminating manual configuration for supported hardware.

## How Project N.O.M.A.D. Detects NVIDIA GPUs

The detection mechanism operates in two distinct stages to ensure accurate hardware identification and container compatibility.

### Stage 1: Docker Runtime Detection via `_detectGPUType()`

The system first queries the Docker daemon to determine if the **NVIDIA Container Toolkit** is properly installed. In [`admin/app/services/docker_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/docker_service.ts), the `_detectGPUType()` method inspects the runtime configuration:

```typescript
const dockerInfo = await this.docker.info()
const runtimes = dockerInfo.Runtimes || {}
if ('nvidia' in runtimes) {
  // NVIDIA runtime detected → GPU is potentially usable
}

```

If the `nvidia` runtime is present, the system confirms that the host NVIDIA driver and container toolkit are installed, allowing GPU pass-through to containers. If the runtime is missing but `lspci` detects an NVIDIA device, the code flags `toolkitMissing` so the UI can advise installing the NVIDIA Container Toolkit.

### Stage 2: In-Container Verification with `getNvidiaSmiInfo()`

Once the runtime is confirmed, the system verifies actual GPU accessibility inside the Ollama container. Located in [`admin/app/services/system_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/system_service.ts), the `getNvidiaSmiInfo()` method executes `nvidia-smi` within the container and parses the CSV output:

```typescript
if (!graphics.controllers || graphics.controllers.length === 0) {
  const runtimes = dockerInfo.Runtimes || {}
  if ('nvidia' in runtimes) {
    gpuHealth.hasNvidiaRuntime = true
    const nvidiaInfo = await this.getNvidiaSmiInfo()
    if (Array.isArray(nvidiaInfo)) {
      graphics.controllers = nvidiaInfo.map(gpu => ({
        model: gpu.model,
        vendor: gpu.vendor,
        vram: gpu.vram,
      }))
      gpuHealth.status = 'ok'
      gpuHealth.ollamaGpuAccessible = true
    }
  }
}

```

The `getNvidiaSmiInfo()` method runs `nvidia-smi --query-gpu=name,memory.total` inside the Ollama container, strips non-printable characters, splits the CSV rows, and returns a clean array of `{vendor, model, vram}` objects.

## Handling Detection Failures and Edge Cases

The system includes comprehensive error handling for various failure scenarios:

- **Ollama container not running**: `getNvidiaSmiInfo()` returns `'OLLAMA_NOT_FOUND'` and sets `gpuHealth.status` to `'ollama_not_installed'`.
- **NVIDIA runtime present but `nvidia-smi` errors**: Returns `'BAD_RESPONSE'` with `gpuHealth.status` set to `'passthrough_failed'`.
- **GPU present but NVIDIA Container Toolkit missing**: `_detectGPUType()` returns `{type: 'none', toolkitMissing: true}`, allowing the UI to surface installation warnings.
- **No GPU detected**: `gpuHealth.type` is set to `'none'` with `gpuHealth.status` as `'ok'` (indicating no GPU is required).

## Implementation Examples

### Checking GPU Status via API

You can expose GPU detection status through an API endpoint using the `SystemService`:

```typescript
import SystemService from '#app/services/system_service'

// Example: an API route that returns GPU status
router.get('/api/gpu-status', async ({ response }) => {
  const sysInfo = await SystemService.getSystemInfo()
  if (!sysInfo) {
    return response.internalServerError({ error: 'Unable to collect system info' })
  }

  const { gpuHealth, graphics } = sysInfo
  response.ok({
    detected: gpuHealth.status,
    hasNvidiaRuntime: gpuHealth.hasNvidiaRuntime ?? false,
    ollamaAccessible: gpuHealth.ollamaGpuAccessible ?? false,
    gpus: graphics.controllers?.map(c => ({
      model: c.model,
      vendor: c.vendor,
      vram: c.vram,
    })) ?? [],
  })
})

```

**Example response when an NVIDIA GPU is available:**

```json
{
  "detected": "ok",
  "hasNvidiaRuntime": true,
  "ollamaAccessible": true,
  "gpus": [
    { "model": "GeForce RTX 3080", "vendor": "NVIDIA", "vram": 10240 }
  ]
}

```

### Displaying GPU Information in the Frontend

Use this React component to display GPU status in your UI:

```tsx
import { useEffect, useState } from 'react'
import axios from 'axios'

export function GpuInfo() {
  const [info, setInfo] = useState<any>(null)

  useEffect(() => {
    axios.get('/api/gpu-status')
      .then(r => setInfo(r.data))
      .catch(() => setInfo({ error: 'Failed to fetch GPU info' }))
  }, [])

  if (!info) return <p>Loading…</p>
  if (info.error) return <p>{info.error}</p>

  return (
    <div>
      <h3>GPU Detection</h3>
      <p>Status: {info.detected}</p>
      {info.gpus.length > 0 ? (
        <ul>
          {info.gpus.map((g: any, i: number) => (
            <li key={i}>{g.vendor} {g.model} – {g.vram} MiB</li>
          ))}
        </ul>
      ) : (
        <p>No GPU detected.</p>
      )}
    </div>
  )
}

```

### Manual GPU Detection in Scripts

For administrative scripts or debugging, invoke the low-level helper directly:

```typescript
import SystemService from '#app/services/system_service'

async function listNvidiaGpus() {
  const sys = new SystemService(/* DockerService instance */)
  const info = await sys.getNvidiaSmiInfo()
  console.log('Raw NVIDIA info:', info)
}
listNvidiaGpus()

```

## Key Source Files and Functions

The GPU detection system is implemented across several critical files in the `admin` directory:

- **[`admin/app/services/docker_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/docker_service.ts)** – Contains `_detectGPUType()` which queries the Docker daemon for NVIDIA runtimes and performs host-level GPU detection via `lspci`.
- **[`admin/app/services/system_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/app/services/system_service.ts)** – Houses `getNvidiaSmiInfo()` for executing `nvidia-smi` inside containers and the GPU fallback logic within `getSystemInfo()` that orchestrates the full detection pipeline.
- **[`admin/types/system.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/types/system.ts)** – Defines the `GpuHealthStatus` interface specifying the shape of GPU health reports including `status`, `hasNvidiaRuntime`, and `ollamaGpuAccessible` fields.
- **[`admin/constants/service_names.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/admin/constants/service_names.ts)** – Stores the `OLLAMA` constant used to identify the target container for GPU queries.

## Summary

- **Project N.O.M.A.D. automatically detects NVIDIA GPUs** through a two-stage process involving Docker runtime inspection and in-container `nvidia-smi` execution.
- **The `_detectGPUType()` method** in [`docker_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/docker_service.ts) verifies NVIDIA Container Toolkit installation by checking for the `nvidia` runtime in Docker daemon info.
- **`getNvidiaSmiInfo()` in [`system_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/system_service.ts)** executes GPU queries inside the Ollama container, parsing CSV output to extract model names, vendors, and VRAM capacity.
- **Comprehensive error handling** covers missing containers, toolkit installation failures, and passthrough errors, surfacing detailed status via the `gpuHealth` object.
- **GPU information is exposed through the system API**, enabling frontend components to conditionally enable GPU-accelerated AI workloads based on hardware availability.

## Frequently Asked Questions

### Does Project N.O.M.A.D. support AMD GPUs?

Currently, Project N.O.M.A.D. primarily implements automated detection for NVIDIA GPUs through the NVIDIA Container Toolkit and `nvidia-smi`. While the `_detectGPUType()` method in [`docker_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/docker_service.ts) contains logic to identify AMD GPUs via `lspci`, the automated container integration and GPU health reporting are optimized for NVIDIA hardware. AMD GPU support would require similar runtime detection mechanisms for ROCm or other AMD container toolkits.

### What are the requirements for NVIDIA GPU detection?

For Project N.O.M.A.D. to automatically detect and utilize an NVIDIA GPU, the host system must have the **NVIDIA driver** installed, the **NVIDIA Container Toolkit** configured, and the Docker daemon must expose the `nvidia` runtime. The Ollama container must also be running, as the system executes `nvidia-smi` inside this specific container to verify GPU accessibility. If the toolkit is missing but a GPU is present, the system flags `toolkitMissing` in the health status.

### How can I troubleshoot GPU detection failures?

If Project N.O.M.A.D. fails to detect your NVIDIA GPU, check the `gpuHealth` field in the system information API response. Status values like `'ollama_not_installed'` indicate the Ollama container isn't running, while `'passthrough_failed'` suggests the NVIDIA runtime is present but `nvidia-smi` returned an error. Verify that `docker info` shows the `nvidia` runtime and that `nvidia-smi` works on the host. The `_detectGPUType()` method in [`docker_service.ts`](https://github.com/Crosstalk-Solutions/project-nomad/blob/main/docker_service.ts) also logs when the toolkit is missing but hardware is detected.

### Can I use Project N.O.M.A.D. without a GPU?

Yes, Project N.O.M.A.D. functions entirely on CPU-based inference when no GPU is detected. The `gpuHealth` object will report `type: 'none'` and `status: 'ok'`, indicating that while no GPU is present, the system is operating normally. The Ollama service will automatically fall back to CPU execution for model inference, though performance will be significantly slower compared to GPU acceleration.