# Core Components of the ODS Architecture: A Technical Deep Dive into Osmantic/ODS > Explore the core components of the Osmantic/ODS architecture. Discover its layered compose model, core services mesh, and modular installer for deploying GPU AI services locally. - Repository: [Osmantic/ODS](https://github.com/Osmantic/ODS) - Tags: deep-dive - Published: 2026-08-30 --- **TLDR:** The Osmantic/ODS (Open-Source Data Stack) is a two-layer system comprising a 13-phase installer wrapper and a core product built on a **Layered Compose Model**, **Core Services** mesh, and **modular Installer Architecture** that deploys GPU-accelerated AI services to localhost. ODS provides a self-hosted AI infrastructure that binds all services to `127.0.0.1` for security. According to the [`ARCHITECTURE.md`](https://github.com/Osmantic/ODS/blob/main/ARCHITECTURE.md) specification in the Osmantic/ODS repository, the system uses Docker Compose overlays to support heterogeneous hardware from NVIDIA GPUs to Apple Silicon. Understanding the core components of the ODS architecture reveals how it maintains modularity across its deterministic installer pipeline and service orchestration. ## Layered Compose Model The **Layered Compose Model** enables hardware-agnostic deployment through file composition rather than monolithic configuration. At the base, [`docker-compose.base.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.base.yml) defines immutable core services including `llama-server`, `open-webui`, and `dashboard-api`. Hardware acceleration is injected via **GPU overlay files** ([`docker-compose.nvidia.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.nvidia.yml), [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml), [`docker-compose.apple.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.apple.yml), [`docker-compose.cpu.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.cpu.yml)). These replace container images and runtime settings based on the detected backend. Optional capabilities enter through **extension compose files** located in `extensions/services/*/compose*.yaml`. Services like `comfyui`, `n8n`, and `openclaw` are added without modifying core definitions. The **resolver script** ([`scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/scripts/resolve-compose-stack.sh)) merges these layers into a final deployable stack. It discovers enabled extensions, selects the appropriate GPU overlay, and outputs a unified compose file. ```bash # Manually resolve and inspect the final Docker Compose stack scripts/resolve-compose-stack.sh > full-stack.yml docker compose -f full-stack.yml config ``` ## Core Services and Localhost Service Mesh The **core services** layer exposes twenty-two specialized containers, each binding to `127.0.0.1` to preserve local-only access. These communicate via HTTP/REST or OpenAI-compatible APIs. Key infrastructure components include: - **llama-server** (port 8080): The GPU-accelerated LLM inference engine - **litellm** (port 4000): OpenAI-compatible proxy enabling hybrid cloud fallback - **dashboard-api** (port 3002): FastAPI backend exposing setup, agents, and privacy endpoints - **qdrant** (port 6333): Vector database for RAG pipelines - **embeddings** (port 8090): HuggingFace TEI for vector generation User-facing interfaces include **open-webui** (port 3000) for chat and **dashboard** (port 3001) for system control. Voice processing runs through **whisper** (port 9000) and **tts/kokoro** (port 8880), while **comfyui** (port 8188) handles media generation. Additional services like **perplexica** (port 3004) for deep research, **langfuse** (port 3006) for observability, and **opencode** (port 3003) as a web IDE complete the stack. The **gateway** (`litellm`) routes OpenAI-style requests to `llama-server` and can fallback to cloud APIs when local models are insufficient. ```bash # Verify that core services are healthy after startup ods-cli health ``` ## Installer Architecture The **installer architecture** implements a deterministic 13-phase pipeline orchestrated by [`install-core.sh`](https://github.com/Osmantic/ODS/blob/main/install-core.sh). It strictly separates **pure functions** from **imperative side effects** to ensure testability. **Pure-function libraries** reside in `installers/lib/` and handle GPU detection, tier mapping, and service registry lookups without side effects. These include [`detection.sh`](https://github.com/Osmantic/ODS/blob/main/detection.sh), [`tier-map.sh`](https://github.com/Osmantic/ODS/blob/main/tier-map.sh), and [`service-registry.sh`](https://github.com/Osmantic/ODS/blob/main/service-registry.sh). **Sequential phases** live in `installers/phases/` as numbered scripts ([`01-preflight.sh`](https://github.com/Osmantic/ODS/blob/main/01-preflight.sh) through [`13-summary.sh`](https://github.com/Osmantic/ODS/blob/main/13-summary.sh)). Each phase consumes state from previous steps via exported environment variables. Critical phases include: 1. **Preflight**: OS and dependency validation 2. **Detection**: GPU hardware detection and tier assignment 3. **Features**: Interactive selection of voice, RAG, and media components 4. **Docker**: Installation of Docker Engine, Compose, and NVIDIA toolkit 5. **Images**: Pulling container images for the detected hardware tier 6. **Services**: Model file generation, stack launch, and health verification ```bash # Install ODS (runs the full 13-phase installer pipeline) curl -fsSL https://raw.githubusercontent.com/Osmantic/ODS/main/install.sh -o install.sh bash install.sh ``` ## Summary - **Layered Compose Model**: Uses base, GPU overlay, and extension compose files merged by [`scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/scripts/resolve-compose-stack.sh) to support hardware from CPUs to NVIDIA GPUs. - **Core Services**: Twenty-two localhost-bound services exposing ports 3000-9000+ for LLM inference, RAG, voice, and media generation, coordinated via the `litellm` gateway. - **Installer Architecture**: A 13-phase pipeline in [`install-core.sh`](https://github.com/Osmantic/ODS/blob/main/install-core.sh) separating pure logic (`installers/lib/`) from imperative phases (`installers/phases/`). - **Security Model**: All services bind to `127.0.0.1` only, ensuring local-only access to sensitive AI infrastructure. ## Frequently Asked Questions ### What is the ODS (Open-Source Data Stack)? ODS is a self-hosted AI infrastructure platform maintained by Osmantic that deploys a complete stack of large language model services, RAG components, and automation tools to your local machine. According to the repository's [`ARCHITECTURE.md`](https://github.com/Osmantic/ODS/blob/main/ARCHITECTURE.md), it is designed as a two-layer system combining an installer wrapper with deployable core services that run entirely on localhost. ### How does ODS handle different GPU backends? ODS uses a **Layered Compose Model** where [`docker-compose.base.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.base.yml) defines core services and hardware-specific overlays ([`docker-compose.nvidia.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.nvidia.yml), [`docker-compose.amd.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.amd.yml), [`docker-compose.apple.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.apple.yml), [`docker-compose.cpu.yml`](https://github.com/Osmantic/ODS/blob/main/docker-compose.cpu.yml)) modify container runtimes and images. The [`scripts/resolve-compose-stack.sh`](https://github.com/Osmantic/ODS/blob/main/scripts/resolve-compose-stack.sh) resolver automatically selects the appropriate overlay during installation based on the detection logic in [`installers/lib/detection.sh`](https://github.com/Osmantic/ODS/blob/main/installers/lib/detection.sh). ### How do I add optional services to the ODS stack? Optional services are added via the **extensions directory** at `extensions/services/*/`. Each extension contains a [`manifest.yaml`](https://github.com/Osmantic/ODS/blob/main/manifest.yaml) service contract and compose files that the resolver merges into the final stack. You enable these during the **Features** phase of installation or by manually configuring the resolver to include specific extension paths before running `docker compose up -d`. ### Is ODS suitable for production environments? ODS is architected for **local-only deployment** with all services binding to `127.0.0.1`, making it ideal for development workstations and secure local inference rather than exposed production infrastructure. While the deterministic 13-phase installer and layered compose structure provide reliability, the localhost binding and lack of built-in remote access controls mean additional networking layers would be required for internet-facing production use.