How ODS Docker Compose Layering Works: From Base to GPU Overlays

ODS Docker Compose layering merges a base configuration file with conditionally selected overlay files—GPU-specific, multi-GPU, cloud, Apple, and user extensions—to dynamically generate the final Docker Compose stack via the resolve-compose-stack.sh script.

The Osmantic/ODS project implements a sophisticated Docker Compose layering system that automatically adapts to your hardware and deployment needs. By combining a core base file with optional overlays, ODS supports everything from single-GPU workstations to multi-GPU servers and Apple Silicon machines. This article explains the technical architecture behind ODS Docker Compose layering and demonstrates how the resolver script assembles your container stack.

The Base Layer and Core Services

Every ODS deployment starts with ods/docker-compose.base.yml, which declares the essential services including llama-server, open-webui, dashboard-api, and model-router. This file serves as the foundation upon which all other configurations are layered. The base definitions remain hardware-agnostic, containing only the core service structures and networking that every deployment requires regardless of GPU backend.

Overlay Selection Logic

The resolution process is driven by ods/scripts/resolve-compose-stack.sh, specifically within its Python section (lines 15-78). This script evaluates runtime arguments, environment variables, and detected hardware to determine which overlay files to append to the base configuration. The script processes conditions in a specific order using if/elif branches, ensuring deterministic stack assembly.

GPU Backend Selection

The resolver first detects your GPU architecture and appends the corresponding overlay file. Available backends include:

• NVIDIA – docker-compose.nvidia.yml (runtime, device limits)
• AMD – docker-compose.amd.yml
• Apple Silicon – docker-compose.apple.yml (host-mode llama-server)
• Intel – docker-compose.intel.yml
• Intel Arc – docker-compose.arc.yml
• CPU-only – docker-compose.cpu.yml (fallback when no GPU is detected)

Deployment Tiers and Cloud Mode

For specialized deployment scenarios, the script applies tier-specific overlays. When running in cloud mode, the resolver includes docker-compose.cloud.yml, which adds remote-provider services. Apple-specific tiers (designated with "AP_" prefixes like AP_PRO) trigger the Apple Silicon overlay in addition to any GPU-specific configurations, ensuring proper host-mode execution on macOS hardware.

Multi-GPU Configurations

When the environment variable GPU_COUNT exceeds 1, the resolver automatically appends docker-compose.multigpu-<backend>.yml to the stack. For example, a dual-NVIDIA setup receives both docker-compose.nvidia.yml and docker-compose.multigpu-nvidia.yml, enabling device enumeration and load distribution across multiple graphics cards.

External LLM Integration

If EXTERNAL_LLM_URL is configured, the resolver substitutes the local LLM service with docker-compose.external-llm.yml. This overlay removes the local llama-server dependency and configures the system to proxy requests to your external inference endpoint.

Custom Profile Overlays

Users can inject arbitrary compose files via the --profile-overlays flag, accepting a comma-separated list of file paths. The resolver validates that these files exist and appends them verbatim to the resolved stack, allowing complete customization of service definitions without modifying core ODS files.

Extension Discovery and Validation

After assembling the core stack, the script scans the extensions/services/ directory for enabled service manifests. Each extension's manifest.yaml may declare:

• A compose_file (primary extension compose)
• GPU-specific variants (compose.<backend>.yaml)
• Mode-specific overlays (compose.local.yaml)

These files are appended to the final list only after passing security validation. This extension system allows third-party services to integrate seamlessly with ODS Docker Compose layering while maintaining hardware-specific optimizations.

User Overrides and Security Scanning

The resolver supports a docker-compose.override.yml in the repository root for persistent user customizations. Before inclusion, both user overrides and extension files pass through _scan_user_compose_content (starting at line 70), which validates the YAML for dangerous directives that could compromise container security. This scanning ensures that user-provided layers cannot inject privileged modes or unauthorized volume mounts.

Running the Resolver

You can manually invoke the resolver to preview which files will compose your stack. The script outputs a space-separated list of -f <file> arguments suitable for Docker Compose consumption.

Run with NVIDIA backend and tier 1:

ods/scripts/resolve-compose-stack.sh \
  --tier 1 \
  --gpu-backend nvidia \
  --gpu-count 1

Output:

-f docker-compose.base.yml -f docker-compose.nvidia.yml

Enable multi-GPU support with two NVIDIA cards:

ods/scripts/resolve-compose-stack.sh \
  --gpu-backend nvidia \
  --gpu-count 2

Output:

-f docker-compose.base.yml -f docker-compose.nvidia.yml -f docker-compose.multigpu-nvidia.yml

Configure for Apple Silicon:

ods/scripts/resolve-compose-stack.sh \
  --tier AP_PRO \
  --gpu-backend apple

Add custom profile overlays:

ods/scripts/resolve-compose-stack.sh \
  --profile-overlays custom1.yml,custom2.yml

When called with --env, the script exports variables for downstream automation:

• COMPOSE_PRIMARY_FILE – The base configuration path
• COMPOSE_FILE_LIST – Space-separated resolved file list
• COMPOSE_FLAGS – Complete -f flag string for Docker Compose

Summary

• ODS Docker Compose layering begins with ods/docker-compose.base.yml as the immutable foundation for core services.
• The resolve-compose-stack.sh script (lines 15-78) evaluates hardware, tier, and user preferences to select appropriate GPU, multi-GPU, cloud, or Apple overlays.
• Extensions integrate through manifest-declared compose files in extensions/services/, validated by _scan_user_compose_content at line 70.
• User customizations via docker-compose.override.yml and --profile-overlays are security-scanned before inclusion.
• The resolver outputs Docker Compose -f flags and environment variables (COMPOSE_FILE_LIST, COMPOSE_FLAGS) for automated deployment pipelines.

Frequently Asked Questions

How does ODS determine which GPU overlay to use?

The resolve-compose-stack.sh script checks the --gpu-backend argument and environment variables to select from hardware-specific files like docker-compose.nvidia.yml or docker-compose.apple.yml. If no GPU is detected, it falls back to docker-compose.cpu.yml.

Can I run ODS Docker Compose layering with multiple GPUs?

Yes, set --gpu-count to a value greater than 1 when running the resolver. This triggers the inclusion of docker-compose.multigpu-<backend>.yml, which configures Docker to recognize and utilize multiple graphics cards simultaneously.

Where should I place custom Docker Compose modifications?

Place persistent customizations in docker-compose.override.yml at the repository root. For temporary or scenario-specific changes, use the --profile-overlays flag followed by comma-separated file paths. Both methods undergo security scanning before application.

What security measures protect ODS Docker Compose layering?

The resolver validates all user-provided and extension files through _scan_user_compose_content, which checks for dangerous directives that could enable container escapes or unauthorized host access. This ensures that overlay files cannot compromise the host system regardless of their origin.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}