# ODS Hardware Tier Default Models: CPU, AMD, NVIDIA, and Apple Silicon

> Discover ODS hardware tier default models. Learn which CPU, AMD, NVIDIA, and Apple Silicon LLM models are automatically assigned via JSON config.

- Repository: [Osmantic/ODS](https://github.com/Osmantic/ODS)
- Tags: architecture
- Published: 2026-08-30

---

**Open Data Stack (ODS) automatically selects default LLM models based on four hardware tiers—assigning `llama2-7b` to CPU and Apple Silicon, `llama2-13b` to AMD GPUs, and `mixtral-8x7b` to NVIDIA GPUs—via JSON configuration files in `ods/config/backends/`.**

When deploying local language models, understanding the **default models for each ODS hardware tier** ensures optimal performance without manual tuning. The Osmantic/ODS repository uses a tier-detection system that probes your hardware during installation and maps detected capabilities to pre-configured model defaults stored in backend-specific JSON files.

## Hardware Tier Overview and Default Models

ODS categorizes host machines into four distinct hardware classes. Each tier maps to a specific backend JSON file that defines the default model and available alternatives.

| Hardware Tier | Detection Method | Default Model | Memory Profile |
|---------------|------------------|---------------|----------------|
| **CPU** | No GPU detected | `llama2-7b` (GGUF) | Optimized for ≥4GB RAM |
| **AMD** | ROCm/AMD GPU detected | `llama2-13b` (GGUF) | Balanced quality and VRAM usage |
| **NVIDIA** | CUDA GPU detected | `mixtral-8x7b` (GGUF) | High-performance multi-GPU |
| **Apple** | Apple Silicon (M-series) detected | `llama2-7b` (GGUF) | Neural Engine optimized |

The first entry in the `models` array within each backend JSON determines the default. For example, in [`ods/config/backends/nvidia.json`](https://github.com/Osmantic/ODS/blob/main/ods/config/backends/nvidia.json), the array position `[0]` specifies `mixtral-8x7b` as the automatic selection for NVIDIA hardware.

## Backend Configuration Files

ODS stores tier definitions in `ods/config/backends/`. Each file contains a `models` array where the first element serves as the default for that hardware class.

### CPU Tier Configuration

For CPU-only machines, [`ods/config/backends/cpu.json`](https://github.com/Osmantic/ODS/blob/main/ods/config/backends/cpu.json) specifies lightweight models that run efficiently without GPU acceleration:

```json
{
  "backend": "cpu",
  "models": [
    "llama2-7b",
    "phi-2",
    "tinyllama"
  ],
  "format": "gguf"
}

```

### AMD GPU Tier Configuration

AMD GPU hosts use [`ods/config/backends/amd.json`](https://github.com/Osmantic/ODS/blob/main/ods/config/backends/amd.json), defaulting to a mid-size model that leverages ROCm capabilities:

```json
{
  "backend": "amd",
  "models": [
    "llama2-13b",
    "llama2-7b",
    "mistral-7b"
  ],
  "format": "gguf"
}

```

### NVIDIA GPU Tier Configuration

High-performance NVIDIA systems reference [`ods/config/backends/nvidia.json`](https://github.com/Osmantic/ODS/blob/main/ods/config/backends/nvidia.json), which prioritizes the Mixtral mixture-of-experts model:

```json
{
  "backend": "nvidia",
  "models": [
    "mixtral-8x7b",
    "llama2-13b",
    "codellama-13b"
  ],
  "format": "gguf"
}

```

### Apple Silicon Tier Configuration

Apple devices use [`ods/config/backends/apple.json`](https://github.com/Osmantic/ODS/blob/main/ods/config/backends/apple.json), sharing the lightweight default with CPU tiers but optimized for the Apple Neural Engine:

```json
{
  "backend": "apple",
  "models": [
    "llama2-7b",
    "phi-2",
    "mistral-7b"
  ],
  "format": "gguf"
}

```

## Hardware Detection and Model Selection

ODS determines your hardware tier through a detection pipeline involving two critical shell scripts in the installer library.

### Detection Mechanism

The [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) script probes system hardware using `nvidia-smi`, `roc-smi`, and `sysctl` commands. It exports the `ODS_BACKEND` environment variable as `cpu`, `amd`, `nvidia`, or `apple` based on detected capabilities.

Subsequently, [`ods/installers/lib/tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/tier-map.sh) reads `ODS_BACKEND` and loads the corresponding JSON from `ods/config/backends/${ODS_BACKEND}.json`. This script extracts the first model from the `models` array and sets it as the active default for the installation session.

## Programmatically Accessing Default Models

You can retrieve the current hardware tier's default model programmatically using the CLI or Python SDK.

### Bash CLI Method

Query the active backend and parse the default model using standard shell tools:

```bash
#!/bin/bash

# Retrieve detected backend (cpu, amd, nvidia, apple)

backend=$(ods-cli env get ODS_BACKEND)

# Extract first model from tier configuration

default_model=$(jq -r '.models[0]' \
  "/opt/ods/ods/config/backends/${backend}.json")

echo "Active hardware tier: ${backend}"
echo "Default model: ${default_model}"

```

### Python Implementation

For dashboard extensions or API services, use the following pattern implemented in [`ods/extensions/services/dashboard-api/helpers.py`](https://github.com/Osmantic/ODS/blob/main/ods/extensions/services/dashboard-api/helpers.py):

```python
import json
import os
from pathlib import Path

def get_default_model() -> str:
    """
    Retrieves the default model for the current ODS hardware tier.
    Returns the first entry from the backend's models array.
    """
    backend = os.getenv("ODS_BACKEND", "cpu")
    config_path = (
        Path(__file__).parents[3] / "ods" / "config" / "backends" / f"{backend}.json"
    )
    
    with config_path.open() as f:
        data = json.load(f)
    
    # First element in models array is the default

    return data["models"][0]

# Usage

print(f"Default model for current tier: {get_default_model()}")

```

## Customizing Default Models

To change the default model for a specific tier, reorder the `models` array in the corresponding backend JSON file so your preferred model appears first.

Using `jq` to modify the NVIDIA tier default:

```bash

# Promote llama3-8b to default while preserving other models

jq '.models = ["llama3-8b"] + [.models[] | select(. != "llama3-8b")]' \
  ods/config/backends/nvidia.json > tmp.json \
  && mv tmp.json ods/config/backends/nvidia.json

```

Alternatively, create a custom [`model-library.json`](https://github.com/Osmantic/ODS/blob/main/model-library.json) in your ODS root directory. This file overrides the standard backend configurations without modifying the core repository files, allowing you to persist custom defaults across updates.

## Summary

- **ODS hardware tiers** are defined in `ods/config/backends/` with separate JSON files for `cpu`, `amd`, `nvidia`, and `apple` backends.
- **Default models** are determined by the first element in each backend's `models` array: `llama2-7b` (CPU/Apple), `llama2-13b` (AMD), and `mixtral-8x7b` (NVIDIA).
- **Hardware detection** occurs via [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh), which sets the `ODS_BACKEND` environment variable consumed by [`ods/installers/lib/tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/tier-map.sh).
- **Programmatic access** requires reading the backend JSON and extracting the zeroth index of the `models` list using `jq` or Python's `json` module.
- **Customization** involves reordering the JSON array or providing a [`model-library.json`](https://github.com/Osmantic/ODS/blob/main/model-library.json) override file.

## Frequently Asked Questions

### How does ODS determine which hardware tier to use?

ODS runs [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh) during installation, which checks for NVIDIA GPUs via `nvidia-smi`, AMD GPUs via `roc-smi`, and Apple Silicon via `sysctl`. The script sets the `ODS_BACKEND` environment variable to `cpu`, `amd`, `nvidia`, or `apple`, which [`tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/tier-map.sh) then uses to select the appropriate configuration file from `ods/config/backends/`.

### Can I override the default model without modifying JSON files?

Yes. Set the `ODS_DEFAULT_MODEL` environment variable before running the installer or dashboard services. When present, this variable takes precedence over the first element in the backend's `models` array, though the JSON configuration remains unchanged on disk.

### What model format does ODS use for hardware tier defaults?

All default models specified in `ods/config/backends/*.json` use the **GGUF** format (Georgi Gerganov Universal Format). This quantized format provides efficient inference across CPU and GPU backends while maintaining compatibility with the [`llama.cpp`](https://github.com/Osmantic/ODS/blob/main/llama.cpp) inference engine used by ODS services.

### Where are the hardware detection scripts located?

The detection logic resides in [`ods/installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/detection.sh), while the tier-to-model mapping logic is implemented in [`ods/installers/lib/tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/ods/installers/lib/tier-map.sh). These scripts are invoked during the `ods install` process and by the dashboard API helpers when initializing model serving endpoints.