# ODS | Osmantic | Knowledge Base | Instagit

Turn your PC, Mac, or Linux box into an AI server.  LLM inference, chat UI, voice, agents, workflows, RAG, and image generation.

GitHub Stars: 5.2k

Repository: https://github.com/Osmantic/ODS

---

## Articles

### [How to Use ODS with NVIDIA GPUs: Automatic Detection and Configuration Guide](/Osmantic/ODS/how-to-use-ods-with-nvidia-gpus)

Learn how to use ODS with NVIDIA GPUs. ODS automatically detects and configures Docker GPU passthrough for seamless integration. Get started today.

- Tags: how-to-guide
- Published: 2026-09-02

### [How to Use ODS with AMD GPUs: Automated ROCm Setup and Configuration Guide](/Osmantic/ODS/how-to-use-ods-with-amd-gpus)

Easily use ODS with AMD GPUs. Our guide automates ROCm setup and configuration, simplifying GPU device node management for Docker containers. Get started now!

- Tags: how-to-guide
- Published: 2026-09-02

### [What Models Are Available for Each ODS Tier? Complete LLM Hardware Mapping](/Osmantic/ODS/what-models-are-available-for-each-ods-tier)

Discover which LLM models map to each ODS tier from cloud APIs to local GGUF. Find the perfect quantized LLM for your hardware setup and explore the official tier mappings.

- Tags: api-reference
- Published: 2026-09-02

### [How ODS Maps Hardware Capabilities to Deployment Tiers: A Technical Deep Dive](/Osmantic/ODS/how-does-ods-map-hardware-to-tiers)

Learn how ODS maps hardware capabilities to deployment tiers with its two-step pipeline. Discover how GPU analysis sets environment variables for LLM model selection and context limits.

- Tags: deep-dive
- Published: 2026-09-02

### [How ODS Performs GPU Detection on Linux and macOS: A Deep Dive into the Source Code](/Osmantic/ODS/how-does-ods-perform-gpu-detection)

Discover how ODS performs GPU detection on Linux and macOS by examining Bash scripts, sysfs, and sysctl data. Learn about the source code and fallback mechanisms for hardware detection.

- Tags: deep-dive
- Published: 2026-09-02

### [How ODS Handles Errors and Interrupts During Installation: A Complete Technical Guide](/Osmantic/ODS/how-does-ods-handle-errors-and-interrupts)

Learn how ODS handles errors and interrupts during installation. Discover its robust error handling with strict Bash settings, custom traps, and phase-aware cleanup for deterministic failure states.

- Tags: how-to-guide
- Published: 2026-09-02

### [How ODS Installation Phases Are Executed: Sequential Orchestration in Osmantic/ODS](/Osmantic/ODS/how-are-ods-installation-phases-executed)

Discover how ODS installation phases are executed sequentially. Learn about the orchestration process in Osmantic/ODS and ensure successful deployments.

- Tags: internals
- Published: 2026-09-02

### [Understanding the File Header Convention in ODS Installer Modules](/Osmantic/ODS/what-is-the-file-header-convention-in-ods-installer)

Learn the ODS installer module file header convention. Discover the three-section format Purpose, Expects, and Provides for clear script intent, environment variables, and exported outputs.

- Tags: how-to-guide
- Published: 2026-09-02

### [How the ODS Installer Library (installers/lib) Works: Architecture and Deep Dive](/Osmantic/ODS/how-does-the-ods-installer-library-work)

Explore the ODS installer library architecture and deep dive into how installers/lib handles hardware detection, tier-to-model mapping, and package abstraction for intelligent deployments.

- Tags: architecture
- Published: 2026-09-02

### [Understanding the ODS Installer Directory Structure: A Complete Guide](/Osmantic/ODS/what-is-the-ods-installer-directory-structure)

Explore the ODS installer directory structure. Learn about the core orchestrator, reusable libraries, and sequential installation phases in this comprehensive guide.

- Tags: how-to-guide
- Published: 2026-09-02

### [How to Configure ODS for Cloud APIs and Local LLM Servers](/Osmantic/ODS/how-to-configure-ods-api-usage)

Configure ODS to use cloud APIs or local LLM servers by setting ODS_MODE and LLM_BACKEND in your .env file. Rerun the installer to update your Docker Compose stack.

- Tags: how-to-guide
- Published: 2026-09-01

### [Understanding the ODS Service Communication Flow: A Complete Architecture Guide](/Osmantic/ODS/what-is-the-ods-service-communication-flow)

Discover the ODS service communication flow. Learn how ODS routes traffic through localhost containers for seamless UI, LLM, and service interaction within its Docker Compose architecture.

- Tags: architecture
- Published: 2026-09-01

### [How to Customize ODS with User Extensions: A Complete Guide to Service Extensions](/Osmantic/ODS/how-to-customize-ods-with-user-extensions)

Customize ODS with user extensions. Learn to create self-contained service extensions with manifest.yaml and Docker Compose for seamless integration. A complete guide for Osmantic ODS.

- Tags: how-to-guide
- Published: 2026-09-01

### [How ODS Docker Compose Layering Works: From Base to GPU Overlays](/Osmantic/ODS/how-does-the-ods-docker-compose-layering-work)

Discover how ODS Docker Compose layering dynamically builds your stack by merging base and overlay files for GPU, multi-GPU, cloud, and user extensions. Learn more today!

- Tags: internals
- Published: 2026-09-01

### [How ODS Handles Different GPU Backends: NVIDIA, AMD, and Apple Silicon Detection](/Osmantic/ODS/how-does-ods-handle-different-gpu-backends)

ODS automatically detects NVIDIA, AMD, and Apple Silicon GPUs during installation. Learn how ODS configures your system for optimal GPU performance.

- Tags: internals
- Published: 2026-09-01

### [ODS Architecture Overview: Complete Technical Guide to the Open-Source Distributed Stack](/Osmantic/ODS/what-is-the-ods-architecture-overview)

Explore the ODS architecture overview, a complete technical guide to the Open-Source Distributed Stack. Learn about its layered Docker-Compose setup for local end-to-end AI workflows on a single machine.

- Tags: architecture
- Published: 2026-09-01

### [How to Use ODS for Image Generation: Stable Diffusion and ComfyUI Setup](/Osmantic/ODS/how-to-use-ods-for-image-generation)

Learn to use ODS for image generation with Stable Diffusion and ComfyUI. Set up AI services locally via convenient REST APIs without manual port configuration. Get started with Osmantic/ODS today.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Configure RAG with ODS: Enable Vector Search and Embeddings](/Osmantic/ODS/how-to-configure-rag-with-ods)

Configure RAG with ODS for powerful vector search and embeddings. Easily install with the --rag flag and customize integrations via environment variables in your .env file.

- Tags: how-to-guide
- Published: 2026-09-01

### [How to Set Up the ODS Chat Interface: Installation and Access Guide](/Osmantic/ODS/how-to-set-up-ods-chat-interface)

Learn how to set up the ODS chat interface easily. Install and access ODS chat on port 3000 or your local network with our quick guide.

- Tags: getting-started
- Published: 2026-09-01

### [How to Deploy LLM Inference Locally with ODS: Complete Setup Guide](/Osmantic/ODS/how-to-deploy-llm-inference-locally-with-ods)

Deploy LLM inference locally with ODS. This guide shows how to use the Osmantic Deployment System for a simple, single-command setup with auto-detected hardware and Open WebUI.

- Tags: how-to-guide
- Published: 2026-09-01

### [ODS Hardware Support for AI Deployment: Complete GPU and CPU Compatibility Guide](/Osmantic/ODS/what-hardware-does-ods-support-for-ai-deployment)

Discover ODS hardware support for AI deployment. Explore compatibility with NVIDIA, AMD, Intel Arc, and Apple Silicon GPUs and CPUs on Linux, Windows, and macOS. Optimize AI effortlessly.

- Tags: how-to-guide
- Published: 2026-09-01

### [System Requirements for ODS: From Lightweight Setups to High-End Workstations](/Osmantic/ODS/what-are-the-system-requirements-for-ods)

Discover ODS system requirements. Scale from 4GB RAM setups to dual RTX 4090 workstations. ODS automatically detects hardware for optimal performance.

- Tags: getting-started
- Published: 2026-09-01

### [How to Install Osmantic Deployment System (ODS): Complete Setup Guide](/Osmantic/ODS/how-to-install-osmantic-deployment-system-ods)

Install Osmantic Deployment System ODS effortlessly on Linux macOS or Windows with Docker. Our guide covers GPU detection and AI stack configuration for a seamless setup.

- Tags: getting-started
- Published: 2026-09-01

### [What Is the Role of the `installers/lib/` Directory in ODS?](/Osmantic/ODS/what-is-the-role-of-the-installers-lib-directory-in-ods)

Discover the role of the installers/lib/ directory in ODS. This layer contains Bash scripts for constants, hardware detection, and utilities, sourced by install-core.sh.

- Tags: internals
- Published: 2026-08-30

### [How to Force a Specific Hardware Tier During ODS Installation](/Osmantic/ODS/how-to-force-a-specific-hardware-tier-during-ods-installation)

Control ODS hardware tiers during installation using the CLI flag or environment variable. Force specific tiers and bypass auto GPU detection for precise control.

- Tags: how-to-guide
- Published: 2026-08-30

### [ODS Hardware Tier Default Models: CPU, AMD, NVIDIA, and Apple Silicon](/Osmantic/ODS/what-are-the-default-models-for-each-ods-hardware-tier)

Discover ODS hardware tier default models. Learn which CPU, AMD, NVIDIA, and Apple Silicon LLM models are automatically assigned via JSON config.

- Tags: architecture
- Published: 2026-08-30

### [How ODS Manages Docker Compose Configurations for Different GPU Types](/Osmantic/ODS/how-does-ods-manage-docker-compose-configurations-for-different-gpu-types)

Discover how ODS manages Docker Compose for diverse GPU types. Learn about its layered architecture and automatic configuration merging for AMD, NVIDIA, and Apple Silicon.

- Tags: how-to-guide
- Published: 2026-08-30

### [ODS Dashboard Frontend and Backend Technologies: Complete Stack Breakdown](/Osmantic/ODS/what-are-the-frontend-and-backend-technologies-used-in-the-ods-dashboard)

Discover the ODS dashboard's full tech stack. Explore the React 18 frontend with Vite and Tailwind CSS, plus the Python FastAPI backend powered by Uvicorn for AI deployment.

- Tags: architecture
- Published: 2026-08-30

### [How to Configure ODS to Use an External LLM Endpoint (Ollama or LM Studio)](/Osmantic/ODS/how-to-configure-ods-to-use-an-external-llm-endpoint-like-ollama-or-lm-studio)

Configure ODS to use external LLMs like Ollama or LM Studio. Learn how ODS routes LLM traffic to OpenAI-compatible endpoints for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-30

### [How ODS Ensures Installation Robustness with `set -euo pipefail`](/Osmantic/ODS/how-does-ods-ensure-installation-robustness-with-set-euo-pipefail)

Learn how ODS ensures installation robustness using set euo pipefail. Discover how this Bash strict mode setting prevents errors, catches undefined variables, and improves script reliability.

- Tags: how-to-guide
- Published: 2026-08-30

### [What Is the Purpose of the install-core.sh Script in ODS?](/Osmantic/ODS/what-is-the-purpose-of-the-install-core-sh-script-in-ods)

Discover the purpose of the install-core.sh script in ODS. This script orchestrates the Osmantic ODS installer, initializing the environment and executing sequential installation phases to deploy the full ODS stack.

- Tags: how-to-guide
- Published: 2026-08-30

### [How to Manage the ODS Stack Using the `ods-cli`: Command Reference Guide](/Osmantic/ODS/how-can-i-manage-the-ods-stack-using-the-ods-cli)

Master ODS stack management with the ods-cli command reference guide. Install, configure, and maintain your Open-Source Distributed Stack effortlessly. Get started today.

- Tags: command-reference
- Published: 2026-08-30

### [What Is Bootstrap Mode in ODS Installation? Complete Technical Guide](/Osmantic/ODS/what-is-the-bootstrap-mode-in-ods-installation)

Understand Bootstrap mode for ODS installation. Get ODS running quickly with a lightweight model while the full version downloads in the background. Start chatting instantly.

- Tags: how-to-guide
- Published: 2026-08-30

### [How to Use ODS in Offline or Air-Gapped Mode](/Osmantic/ODS/how-to-use-ods-in-offline-or-air-gapped-mode)

Learn how to use ODS in offline or air-gapped mode with the --offline flag. Enable local RAG and pre-download models for zero-connectivity environments. Get started with secure, offline operation today.

- Tags: how-to-guide
- Published: 2026-08-30

### [How ODS Handles Different GPU Backends: NVIDIA, AMD, Apple Silicon, and Intel Arc Detection](/Osmantic/ODS/how-does-ods-handle-different-gpu-backends-nvidia-amd-apple-silicon-intel-arc)

Discover how ODS detects and supports NVIDIA, AMD, Apple Silicon, and Intel Arc GPU backends. Learn about environment variable normalization and Docker Compose integration for seamless startup.

- Tags: internals
- Published: 2026-08-30

### [Known Issues with Qwen3 Coder Next on Unified Memory in ODS](/Osmantic/ODS/what-are-the-known-issues-with-qwen3-coder-next-on-unified-memory-in-ods)

Discover known issues with Qwen3 Coder Next on unified memory in ODS. Learn how ODS prevents critical failures by rerouting to a stable model.

- Tags: known-issues
- Published: 2026-08-30

### [How to Configure ODS to Use Google Gemma 4 Models: A Complete Guide](/Osmantic/ODS/how-to-configure-ods-to-use-google-gemma-4-models)

Learn how to configure Osmantic ODS to use Google Gemma 4 models by setting the MODEL_PROFILE environment variable. Follow this guide for a seamless setup.

- Tags: how-to-guide
- Published: 2026-08-30

### [Which LLM Models Does ODS Support by Default? Local and Cloud Providers Explained](/Osmantic/ODS/which-llm-models-does-ods-support-by-default)

Discover which LLM models ODS supports by default. Explore local Llama-Server, Lemonade, and cloud providers like GPT-4o, Claude Sonnet, and MiniMax with our guide.

- Tags: api-reference
- Published: 2026-08-30

### [Understanding the ODS Hardware Tier Mapping System: Automatic LLM Selection for GPU and CPU Classes](/Osmantic/ODS/what-is-the-ods-hardware-tier-mapping-system)

Discover the ODS hardware tier mapping system. Automatically select LLMs for GPU and CPU classes by translating hardware capabilities into specific configurations. Learn more about this declarative shell library.

- Tags: deep-dive
- Published: 2026-08-30

### [How to Enable or Disable Specific Features Like Voice or RAG in ODS Installation](/Osmantic/ODS/how-to-enable-or-disable-specific-features-like-voice-or-rag-in-ods-installation)

Easily enable or disable ODS features like voice and RAG using CLI commands or environment variables. Customize your ODS installation effectively.

- Tags: how-to-guide
- Published: 2026-08-30

### [Command-Line Options for the ODS Installer: Complete Flag Reference](/Osmantic/ODS/what-are-the-command-line-options-for-the-ods-installer)

Explore over 30 ODS installer command-line flags like --dry-run and --tier to customize installation hardware detection LLM integration and service deployment for your needs.

- Tags: api-reference
- Published: 2026-08-30

### [How ODS Handles GPU Detection for NVIDIA, Intel, AMD, and ARM Hardware](/Osmantic/ODS/how-does-ods-handle-gpu-detection-for-different-hardware)

Discover how ODS handles GPU detection for NVIDIA, Intel, AMD, and ARM hardware. Our script identifies and exports normalized environment variables for seamless integration and orchestration.

- Tags: how-to-guide
- Published: 2026-08-30

### [What Are the 13 Installation Phases in ODS? Complete Guide to the I3 Pipeline](/Osmantic/ODS/what-are-the-13-installation-phases-in-ods)

Understand the 13 installation phases in ODS. This guide details the I3 pipeline from system checks to final validation for seamless ODS deployment.

- Tags: how-to-guide
- Published: 2026-08-30

### [How the ODS Installer Orchestrates the Installation Process: A 13-Phase Bash Pipeline](/Osmantic/ODS/how-does-the-ods-installer-orchestrate-the-installation-process)

Discover how the ODS installer orchestrates its 13-phase Bash pipeline for robust and secure installations. Learn about its pre-flight checks, error handling, and atomic locking.

- Tags: internals
- Published: 2026-08-30

### [Core Components of the ODS Architecture: A Technical Deep Dive into Osmantic/ODS](/Osmantic/ODS/what-are-the-core-components-of-the-ods-architecture)

Explore the core components of the Osmantic/ODS architecture. Discover its layered compose model, core services mesh, and modular installer for deploying GPU AI services locally.

- Tags: deep-dive
- Published: 2026-08-30

### [How to Install Osmantic Deployment System (ODS) Locally: A Complete Setup Guide](/Osmantic/ODS/how-to-install-osmantic-deployment-system-ods-locally)

Install Osmantic Deployment System ODS locally with our complete guide. Clone the repo and run the install script for a full AI stack deployment on Linux macOS or Windows.

- Tags: how-to-guide
- Published: 2026-08-30