System Requirements for ODS: From Lightweight Setups to High-End Workstations

ODS scales from low-end machines with 4GB RAM running 2B-parameter models to high-end workstations with dual RTX 4090s handling 32B models and voice agents, automatically detecting your hardware tier during installation.

Osmantic/ODS is an open-source framework designed to accommodate diverse hardware configurations, from minimal bootstrap environments to enterprise-grade multi-user deployments. Understanding the system requirements for ODS ensures you select the appropriate model size and feature set for your infrastructure. The installer automatically evaluates your hardware against three defined tiers before proceeding with setup.

ODS Hardware Tiers Explained

ODS categorizes hardware into three distinct tiers based on GPU VRAM, system RAM, and storage capacity. The detection logic in installers/lib/detection.sh automatically assigns your machine to the appropriate category during installation.

Lightweight Tier (Bootstrap)

Configuration: Any GPU or CPU-only, 4GB+ RAM, 15GB free storage Use Case: Running Qwen3.5 2B bootstrap models for initialization and testing

This tier requires no dedicated GPU and functions on minimal hardware. The installer selects this mode automatically when detect_vram_gb returns zero or minimal values. This configuration is suitable for development environments and initial system validation only.

Minimum Tier (Comfortable Operation)

Configuration: RTX 3060 12GB or RTX 4060 8GB, 32GB RAM, 500GB NVMe SSD Use Case: 7B-14B parameter models for basic inference tasks

Twelve gigabytes of VRAM comfortably accommodates 7B-14B models with standard context windows. The 32GB system RAM prevents swapping during model loading, while the NVMe SSD ensures rapid checkpoint access. This tier targets single-user daily operation without voice pipeline features.

Recommended Tier (Production Use)

Configuration: RTX 4070 Ti Super 16GB or RTX 4090 24GB, 64GB RAM, 1TB NVMe SSD Use Case: 32B parameter models, full voice pipelines, and multi-user concurrency

Sixteen gigabytes enables 32B models with reduced context, while 24GB VRAM supports full context windows plus the complete voice processing pipeline. Systems with 48GB+ VRAM (e.g., 2× RTX 4090s) can host multiple models simultaneously or serve concurrent users according to the hardware matrix in ods/docs/HARDWARE-GUIDE.md.

How ODS Validates Your Hardware

The ODS installer performs comprehensive pre-flight validation through installers/phases/04-requirements.sh before allowing installation to proceed.

Automated Detection Process

The installer sources detection functions from installers/lib/detection.sh to evaluate:

• GPU Backend: detect_gpu identifies CUDA-capable devices
• VRAM Capacity: detect_vram_gb returns available GPU memory
• System Memory: detect_ram_gb validates RAM requirements
• CPU Resources: detect_cores checks processor capabilities
• Port Availability: Validates network port accessibility

If any requirement falls below the selected tier's threshold, the installer halts and provides specific remediation guidance.

Manual Hardware Detection

You can manually audit your system using the detection library before running the full installer:

#!/usr/bin/env bash

# Detect and display key hardware specs

source "$(dirname "$0")/../installers/lib/detection.sh"

echo "Operating System: $(detect_os)"
echo "GPU Backend: $(detect_gpu || echo 'none')"
echo "VRAM (GB): $(detect_vram_gb)"
echo "RAM (GB): $(detect_ram_gb)"
echo "CPU Cores: $(detect_cores)"

Save this as check-hardware.sh and execute it to preview which tier the installer will select. The wrapper script scripts/detect-hardware.sh provides similar functionality with formatted output for debugging purposes.

VRAM Requirements by Model Size

Understanding VRAM allocation helps you select appropriate model configurations:

• 12GB VRAM: Supports 7B-14B parameter models comfortably
• 16GB VRAM: Accommodates 32B models with reduced context windows
• 24GB VRAM: Runs 32B models with full context plus voice pipeline processing
• 48GB+ VRAM: Enables multi-model hosting and concurrent user sessions

According to the source code in ods/docs/FAQ.md#hardware, these allocations account for model weights, KV cache, and overhead from the voice processing pipeline when enabled.

Storage and Memory Specifications

Beyond GPU requirements, ODS demands specific storage characteristics:

• Lightweight: 15GB free space for bootstrap artifacts and temporary model caching
• Minimum: 500GB NVMe SSD for 7B-14B model storage and checkpoint operations
• Recommended: 1TB NVMe SSD to house multiple 32B parameter variants and voice model components

System RAM requirements scale with model size due to background processing and embedding caches. The 32GB minimum tier prevents performance degradation from memory pressure, while the 64GB recommended tier supports aggressive caching strategies for production workloads.

Summary

• ODS supports three hardware tiers: Lightweight (4GB RAM, CPU), Minimum (12GB VRAM, 32GB RAM), and Recommended (16-24GB VRAM, 64GB RAM)
• Automatic detection occurs via functions in installers/lib/detection.sh including detect_vram_gb and detect_ram_gb
• Pre-flight validation in installers/phases/04-requirements.sh prevents installation on underspecified hardware
• VRAM requirements scale from 12GB for 7B-14B models to 24GB for 32B models with voice pipelines
• Storage requirements range from 15GB for bootstrap to 1TB NVMe for full production deployments

Frequently Asked Questions

Can I run ODS without a GPU?

Yes, the Lightweight tier supports CPU-only inference for the 2B-parameter bootstrap model. The detect_gpu function returns 'none' when no CUDA device is present, automatically triggering CPU fallback mode suitable for testing and initialization as documented in ods/docs/HARDWARE-GUIDE.md.

Why does the installer check port availability?

The pre-flight script in installers/phases/04-requirements.sh validates network port accessibility to prevent conflicts with existing services. If ports required by ODS voice pipelines or API endpoints are occupied, the installer halts to avoid runtime failures. You must free the conflicting ports or configure ODS to use alternative ports in the environment configuration.

How do I manually override the automatic tier selection?

While the installer automatically selects tiers based on detect_vram_gb output, you can manually specify a tier by setting environment variables before installation. Refer to ods/docs/HARDWARE-GUIDE.md for manual tier selection flags that bypass the automatic detection logic in installers/lib/detection.sh.

Is dual GPU configuration supported for 48GB+ VRAM setups?

Yes, configurations with 2× RTX 4090 cards (48GB total VRAM) support multiple concurrent models and multi-user deployments. The detection script aggregates VRAM across visible CUDA devices, and the hardware guide confirms support for model parallelism across multiple GPUs in the Recommended tier according to the specifications in ods/docs/FAQ.md#hardware.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}