How to Install Osmantic Deployment System (ODS) Locally: A Complete Setup Guide

To install Osmantic Deployment System (ODS) locally, clone the repository and run the install.sh (Linux/macOS) or install.ps1 (Windows) script, which automatically detects your hardware, merges the appropriate Docker Compose configurations, and deploys the full AI stack.

Osmantic Deployment System (ODS) is a self-contained AI stack that bundles LLM inference, chat interfaces, voice agents, and RAG capabilities into Docker containers. According to the Osmantic/ODS source code, the installation process is orchestrated by a Bash-based phase system that handles hardware detection, service resolution, and container deployment without manual configuration.

Prerequisites

Before installing ODS locally, ensure your system meets the baseline requirements. The installer performs automatic detection, but you need Docker and Docker Compose installed and running. The system supports NVIDIA GPUs, AMD GPUs, Apple Silicon, or CPU-only deployments.

ODS requires sufficient disk space for container images (Llama-cpp, FastAPI backend, React frontend) and model weights. The ods/installers/lib/detection.sh script validates available resources during the pre-flight phase.

Installation Steps

Clone the Repository

Start by cloning the ODS repository to your local machine. This provides access to the installer scripts and configuration files.

git clone https://github.com/Osmantic/ODS.git
cd ODS

Run the Installer

Execute the appropriate installer for your operating system. For Linux and macOS, use the Bash installer:

bash install.sh

For Windows environments, use the PowerShell installer:

.\install.ps1

The installer calls ods/install-core.sh to orchestrate the deployment. This script sets the INSTALL_PHASE variable for error reporting and executes three major phases defined in the ods/installers/phases/ directory.

Verify the Deployment

After installation completes, run the built-in diagnostic tool to verify service health:

bash ods/scripts/ods-doctor.sh

This script checks connectivity to core services including the dashboard at http://127.0.0.1:8080, the LLM inference server, and API endpoints.

How the ODS Installer Works

The ODS installation process follows a modular architecture implemented across several key source files. Understanding this flow helps troubleshoot issues or customize the deployment.

Phase 1: Pre-flight Checks

Located in ods/installers/phases/01-preflight.sh, this phase executes hardware detection and validation. The ods/installers/lib/detection.sh library identifies your GPU type (NVIDIA, AMD, Apple Silicon) and operating system. The ods/installers/lib/tier-map.sh utility then maps this hardware to an ODS "tier" configuration (CPU, AMD, NVIDIA, or Apple).

These libraries are designed as pure functions without side effects, ensuring consistent detection across environments.

Phase 2: Service Resolution

The ods/scripts/resolve-compose-stack.sh script dynamically merges Docker Compose configurations. It starts with ods/docker-compose.base.yml (containing core services like llama-server, dashboard, and dashboard-api) and merges the appropriate GPU-specific overlay:

• docker-compose.nvidia.yml for NVIDIA GPU acceleration
• docker-compose.amd.yml for AMD GPU support
• docker-compose.apple.yml for Apple Silicon

The resolver also processes enabled extensions by reading manifest.yaml files from ods/extensions/services/. For example, the dashboard API service includes ods/extensions/services/dashboard-api/manifest.yaml to inject its compose fragments into the final stack.

The merged output is written to ods/.ods-compose.yml, which represents the complete deployment configuration.

Phase 3: Deployment and Health Checks

The final phase, located in ods/installers/phases/13-summary.sh, pulls container images and starts the stack using docker compose up -d. Post-install health checks verify that all containers reach a running state and that network connectivity exists between the React frontend (in ods/extensions/services/dashboard/) and the FastAPI backend.

Manual Deployment Options

If you prefer manual control over the installation, you can bypass the installer and work directly with the resolved compose file:

# Generate the compose file manually

bash ods/scripts/resolve-compose-stack.sh

# Deploy using the merged configuration

docker compose -f ods/.ods-compose.yml up -d

This approach allows you to inspect ods/.ods-compose.yml before deployment or modify environment variables for custom configurations.

Troubleshooting

If the installer fails, check the INSTALL_PHASE variable in the error output to identify which phase encountered the problem. Common issues include:

• GPU detection failures: Verify drivers are installed for NVIDIA or AMD hardware
• Port conflicts: Ensure ports 8080 and 8000 are available for the dashboard and API
• Permission errors: Run the installer with appropriate user permissions for Docker

Consult ods/docs/INSTALL-TROUBLESHOOTING.md in the repository for specific fixes to common installation problems.

Summary

• ODS deploys a complete AI stack via Docker containers using automated hardware detection.
• The installer (install.sh/install.ps1) orchestrates three phases: pre-flight checks, service resolution, and deployment.
• Service resolution merges docker-compose.base.yml with GPU-specific overlays and extension manifests to generate ods/.ods-compose.yml.
• Hardware detection occurs in ods/installers/lib/detection.sh and maps to tiers via ods/installers/lib/tier-map.sh.
• Verify installations using ods/scripts/ods-doctor.sh to check service health and connectivity.

Frequently Asked Questions

Does ODS support CPU-only installations?

Yes. If ods/installers/lib/detection.sh finds no compatible GPU, the installer defaults to a CPU-only configuration using the base Docker Compose file without GPU overlays. The tier mapping logic assigns a CPU tier that deploys llama-server without GPU acceleration flags.

Can I install ODS on Windows?

Yes. The repository includes install.ps1, a PowerShell installer that provides equivalent functionality to the Bash script. Windows installations require Docker Desktop with WSL2 integration enabled. The PowerShell script handles path translation and executes the same phase-based installation logic.

Where are the Docker Compose configurations stored?

The base configuration lives in ods/docker-compose.base.yml, while GPU-specific extensions reside in files like ods/docker-compose.nvidia.yml. During installation, ods/scripts/resolve-compose-stack.sh merges these into a single file at ods/.ods-compose.yml. You can inspect this generated file to see the exact container configuration being deployed.

How do I add or remove services after installation?

ODS uses an extension system where each service (like dashboard-api) includes a manifest.yaml file in ods/extensions/services/. To modify services, edit the relevant manifest files and rerun the resolver script (ods/scripts/resolve-compose-stack.sh), then restart the stack with docker compose -f ods/.ods-compose.yml up -d.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}