Best Practices for Deploying a RAG System with Docker Compose: Production-Ready Patterns

Deploy a complete agentic Retrieval-Augmented Generation (RAG) stack using a single docker compose up command with loosely-coupled services, explicit health checks, and environment-driven configuration.

The jamwithai/production-agentic-rag-course repository provides a reference architecture for running a production-grade RAG pipeline entirely within Docker Compose. By splitting the system into specialized containers—API, vector stores, inference engines, and observability tools—the stack achieves horizontal scalability and operational resilience without orchestration complexity. This guide extracts the deployment patterns from compose.yml to help you launch secure, maintainable RAG infrastructure.

Architecture Overview

The compose.yml file defines ten distinct services orchestrated across a unified Docker network. Each container handles a specific concern, from FastAPI request serving to vector search and LLM inference.

• api (lines 3-13): FastAPI entry point built from the repository root (build: .). Exposes port 8000 and declares explicit dependencies on postgres, opensearch, and redis. A custom healthcheck (lines 15-20) polls http://localhost:8000/api/v1/health every 30 seconds to ensure readiness before downstream services start.

• postgres (lines 53-66): Runs postgres:16-alpine as the SQL metadata store. Persists data via the named volume postgres_data, surviving container recreation.

• opensearch (lines 53-76): Single-node OpenSearch cluster (opensearchproject/opensearch:2.19.0) with memory limits configured. Includes a curl-based healthcheck to verify node status before the API attempts to index documents.

• redis (lines 35-49): Cache and pub-sub layer using redis:7-alpine with appendonly persistence enabled. Critical for Langfuse tracing and streaming response buffering.

• ollama (lines 37-52): LLM inference server running ollama/ollama:0.11.2 on port 11434. Pulls models on first run and stores weights in the ollama_data volume.

• airflow (lines 99-136): Scheduled ingestion pipelines for Arxiv papers and PDF parsing. Built from ./airflow/Dockerfile and mounts the repository's src directory as a volume to share service code with the API.

• langfuse-web & langfuse-worker: Observability layer running langfuse:3 images. The web interface handles UI requests while the background worker processes traces, both requiring dedicated Postgres, ClickHouse, Redis, and MinIO instances defined later in the file.

• clickhouse (lines 76-94): Analytics database (clickhouse/clickhouse-server:24.8-alpine) backing Langfuse metrics aggregation.

• minio (lines 48-66): S3-compatible object storage for Langfuse asset persistence.

All services attach to the rag-network bridge network (declared at lines 32-34, 51-53, 78-80, and throughout), enabling DNS resolution via service names like postgres or opensearch rather than hardcoded IPs.

Network Isolation and Service Discovery

The stack leverages a single custom bridge network called rag-network. This design isolates the RAG infrastructure from the host network while permitting zero-trust communication between containers using their service names as hostnames. The API container references postgres:5432 and opensearch:9200 directly, eliminating the need for environment variable juggling of IP addresses.

For production hardening, this network topology ensures that sensitive databases like ClickHouse and Postgres accept connections only from within the rag-network namespace, reducing the attack surface exposed to the host.

Configuration Management with Environment Files

Secrets and host-specific parameters live exclusively in a .env file, never committed to version control. The repository ships .env.example as a template defining required variables such as POSTGRES_PASSWORD, LANGFUSE_SALT, and LANGFUSE_ENCRYPTION_KEY.

In compose.yml (lines 21-31), the API service injects these values via the env_file directive:

services:
  api:
    env_file:
      - .env
    environment:
      - DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD}@postgres:5432/rag_db

Pin exact image tags in the Compose file—redis:7-alpine rather than redis:latest—to prevent accidental upgrades that could break compatibility with the OpenSearch client or Langfuse workers.

Health Checks and Startup Sequencing

Robust dependency management prevents race conditions during startup. The API service uses depends_on with condition: service_healthy (lines 9-14), ensuring it waits until Postgres, OpenSearch, and Redis report healthy status before binding to port 8000.

The healthcheck implementation for the API service (lines 15-20) uses Python's urllib to verify endpoint responsiveness:

healthcheck:
  test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8000/api/v1/health')\""]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s

OpenSearch and Redis define similar probes, allowing Docker Compose to restart containers that fail to initialize within the specified grace period.

Data Persistence Strategy

Named volumes ensure that vector indexes, conversation history, and LLM weights survive container destruction. The compose.yml defines dedicated volumes for each stateful service (lines 71-79, 124-127):

• postgres_data for SQL metadata
• opensearch_data for vector indexes
• ollama_data for downloaded model weights
• redis_data for append-only cache files

When running docker compose down -v, the -v flag removes these volumes along with containers, providing a clean slate for testing but requiring caution in production environments.

Deployment Workflow

Execute the following commands from the repository root to deploy the stack:

# 1. Prepare environment variables

cp .env.example .env

# Edit .env to set POSTGRES_PASSWORD, LANGFUSE_SALT, and LANGFUSE_ENCRYPTION_KEY

# 2. Build and launch all services in detached mode

docker compose up --build -d

# 3. Verify service health

docker compose ps
docker compose logs api --tail 100

# 4. Restart individual services after code changes

docker compose restart api

# 5. Complete teardown including persistent data

docker compose down -v

The --build flag ensures the API and Airflow images compile from their respective Dockerfiles (./Dockerfile and ./airflow/Dockerfile), incorporating the latest code from src/services/.

Observability and Monitoring

The deployment includes production observability without additional instrumentation. Access the Langfuse tracing UI at http://localhost:3001 to view RAG retrieval steps, token usage, and latency metrics stored in ClickHouse. OpenSearch Dashboards available at http://localhost:5601 provide insights into vector index health and query performance.

Airflow DAGs located in airflow/dags/arxiv_ingestion/*.py handle scheduled document ingestion, automatically pulling papers, parsing PDFs via src/services/pdf_parser/factory.py, and embedding vectors into OpenSearch. Monitor these pipelines through the Airflow web interface to ensure data freshness for the RAG retriever.

Summary

• Service decomposition: Split the RAG pipeline into ten specialized containers (API, Postgres, OpenSearch, Redis, Ollama, Airflow, Langfuse, ClickHouse, MinIO) defined in compose.yml for independent scaling and maintenance.
• Health-driven startup: Implement depends_on with condition: service_healthy and custom healthchecks (lines 15-20) to eliminate race conditions between the API and its backing stores.
• Immutable tags: Pin specific image versions (e.g., opensearchproject/opensearch:2.19.0, redis:7-alpine) to ensure reproducible builds across environments.
• Secret isolation: Store credentials exclusively in .env (templated by .env.example) and inject them via env_file declarations, keeping sensitive data out of version control.
• Persistent volumes: Use named volumes (postgres_data, opensearch_data, ollama_data) to preserve state across container restarts, with clear backup strategies for production data.
• Unified networking: Attach all services to rag-network for automatic DNS resolution and inter-service communication without hardcoded IP addresses.

Frequently Asked Questions

How do I update environment variables without rebuilding the entire stack?

Modify the .env file in the repository root, then restart only the affected service using docker compose restart <service_name>. For the API specifically, run docker compose restart api to reload environment injections without affecting the Postgres or OpenSearch containers. Note that changes to variables used in volume mounts or build-time arguments require a full docker compose up --build to take effect.

What is the purpose of the rag-network defined in compose.yml?

The rag-network (declared at lines 32-34 and referenced throughout the file) creates an isolated bridge network enabling DNS-based service discovery. Containers communicate using service names as hostnames—e.g., the API connects to postgres:5432 and opensearch:9200—without exposing database ports to the host machine. This network isolation improves security by ensuring that sensitive services like ClickHouse and MinIO accept connections only from within the Docker network namespace.

How do I back up the Postgres and OpenSearch data volumes?

Use temporary helper containers to archive volume contents before running docker compose down -v. For Postgres: docker run --rm -v production-agentic-rag-course_postgres_data:/data -v $(pwd)/backup:/backup alpine tar czf /backup/postgres.tar.gz -C /data . For OpenSearch, pause indexing and snapshot the opensearch_data volume similarly. Restore by extracting the archive back into a freshly created volume before starting the stack.

Why does the API service depend on health checks rather than just service availability?

The depends_on condition service_healthy (lines 9-14) ensures the API starts only after Postgres, OpenSearch, and Redis complete initialization—such as creating indexes or loading cache warm-ups—not just when their containers exist. This prevents connection errors during FastAPI startup when databases are still bootstrapping. The healthcheck defined at lines 15-20 specifically validates that the FastAPI application has bound to port 8000 and responds to HTTP requests before Docker marks the container as healthy.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}