How Docker Compose Files Are Merged and Resolved for the Dream Server Stack

Dream Server programmatically merges Docker Compose files by selecting a base configuration, layering GPU-specific overlays, injecting validated extension fragments, and appending user overrides to generate a deterministic docker compose command.

The Dream Server stack relies on a sophisticated resolution pipeline to assemble container configurations dynamically. Located in dream-server/scripts/resolve-compose-stack.sh, the resolver script orchestrates how Docker Compose files are merged and resolved for the Dream Server stack based on hardware capabilities, user preferences, and security constraints.

Understanding the Compose Resolution Pipeline

The resolution process implemented in resolve-compose-stack.sh (lines 12-38) accepts flags such as --gpu-backend, --tier, --profile-overlays, --gpu-count, and --env to determine which files to include. The script constructs a prioritized list of Compose files, transforming them into -f flags for the Docker Compose CLI.

Step-by-Step Merge Process

Input Arguments and Configuration

The script begins by parsing command-line arguments that drive the selection logic. Key parameters include:

• --gpu-backend: Specifies the GPU type (nvidia, amd, apple, intel, arc, or cpu)
• --tier: Hardware tier configuration
• --profile-overlays: Custom comma-separated overlay files
• --gpu-count: Number of GPUs for multi-GPU setups
• --env: Output mode that prints environment variables instead of raw flags

Base Compose Selection Logic

The core selection logic (lines 91-140) uses a conditional ladder to determine the base configuration:

• Profile Overlays: If --profile-overlays is provided and all files exist, these become the full list with the last overlay designated as the primary file.
• Apple Silicon: Uses docker-compose.base.yml + docker-compose.apple.yml
• CPU-only: Uses docker-compose.base.yml + docker-compose.cpu.yml
• AMD GPUs: Uses docker-compose.base.yml + docker-compose.amd.yml
• Intel/ARC/SYCL: Uses docker-compose.base.yml + docker-compose.arc.yml or docker-compose.intel.yml
• Nvidia (default): Uses docker-compose.base.yml + docker-compose.nvidia.yml
• Fallback: Plain docker-compose.yml if no matches occur

Multi-GPU and Extension Handling

When --gpu-count exceeds 1 and docker-compose.multigpu.yml exists, the script appends this overlay (lines 44-47).

For extensions, the resolver scans extensions/services/ for manifest.yaml files to identify compose_file paths. It validates that files remain within the extension directory, then adds:

• The base compose file specified in the manifest
• GPU-specific overlays (compose.<backend>.yaml)
• Mode-specific overlays (compose.local.yaml)

Security Scanning for User Extensions

User-installed extensions in data/user-extensions/ undergo rigorous validation through the _scan_user_compose_content helper function. This security layer rejects dangerous directives including:

• Privileged mode containers
• Host network bindings
• Mounting /var/run/docker.sock
• Other potentially harmful configurations

Override Files and Flag Construction

The script checks for docker-compose.override.yml in the repository root, scanning it for security issues before inclusion. Finally, it constructs the output:

• Standard mode: Prints the -f flag string directly
• Environment mode (--env): Exports three variables:
  • COMPOSE_PRIMARY_FILE: The primary compose file
  • COMPOSE_FILE_LIST: Comma-separated list of all files
  • COMPOSE_FLAGS: Complete -f flags string

Practical Usage Examples

Resolve compose files for an Nvidia tier-1 system:

./dream-server/scripts/resolve-compose-stack.sh \
    --gpu-backend nvidia \
    --tier 1 \
    --gpu-count 1

Typical output:

-f docker-compose.base.yml -f docker-compose.nvidia.yml -f extensions/services/dashboard-api/compose.yaml -f extensions/services/dashboard-api/compose.nvidia.yaml

Request environment-mode output for an AMD multi-GPU setup:

./dream-server/scripts/resolve-compose-stack.sh \
    --gpu-backend amd \
    --tier SH_LARGE \
    --gpu-count 2 \
    --env

Sample environment variables printed:

COMPOSE_PRIMARY_FILE="docker-compose.amd.yml"
COMPOSE_FILE_LIST="docker-compose.base.yml,docker-compose.amd.yml,extensions/services/dashboard-api/compose.yaml,extensions/services/dashboard-api/compose.amd.yaml,docker-compose.multigpu.yml"
COMPOSE_FLAGS="-f docker-compose.base.yml -f docker-compose.amd.yml -f extensions/services/dashboard-api/compose.yaml -f extensions/services/dashboard-api/compose.amd.yaml -f docker-compose.multigpu.yml"

Adding a custom profile overlay:

./dream-server/scripts/resolve-compose-stack.sh \
    --profile-overlays "docker-compose.base.yml,profile/low-latency.yml" \
    --env

Resulting variables will list both overlay files, with low-latency.yml becoming the primary compose file.

Summary

• The resolver script dream-server/scripts/resolve-compose-stack.sh orchestrates how Docker Compose files are merged and resolved for the Dream Server stack through a layered selection process.
• Base configurations are selected based on GPU backend, with specific files for Nvidia, AMD, Intel, Apple Silicon, and CPU-only deployments.
• Multi-GPU systems automatically include docker-compose.multigpu.yml when available.
• Extension discovery validates manifest.yaml files and applies backend-specific overlays securely.
• User extensions undergo content scanning via _scan_user_compose_content to prevent privilege escalation.
• The script outputs either raw -f flags or environment variables (COMPOSE_FILE_LIST, COMPOSE_FLAGS) depending on the --env flag.

Frequently Asked Questions

What determines which Docker Compose files are selected?

The selection depends on the --gpu-backend flag and hardware detection logic in resolve-compose-stack.sh. Nvidia GPUs default to docker-compose.nvidia.yml, while AMD, Intel, Apple Silicon, and CPU-only systems use their respective overlay files. Custom profiles specified via --profile-overlays take precedence over automatic detection.

How does Dream Server handle multiple GPUs?

When --gpu-count is greater than 1, the script automatically appends docker-compose.multigpu.yml to the stack (lines 44-47), provided the file exists in the repository root.

Are user-installed extensions safe to use?

Yes, the script implements security scanning through the _scan_user_compose_content function, which examines compose files in data/user-extensions/ for dangerous configurations like privileged mode, host network access, or Docker socket mounts before inclusion.

Can I provide my own Compose overrides?

Yes, placing a docker-compose.override.yml file in the repository root allows custom configurations. The script scans this file for security issues and appends it to the final flag list, ensuring user customizations are merged last in the resolution chain.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}