How to Configure and Run the CMF Server with a PostgreSQL Backend

To run the CMF server with a PostgreSQL backend, set the five POSTGRES_* environment variables in a .env file and execute docker compose -f docker-compose-server.yml up -d; the orchestration ensures the database passes health checks before the FastAPI service initializes its async SQLAlchemy engine.

The CMF (Continuous Machine Learning Framework) server from the hewlettpackard/cmf repository persists ML metadata in a PostgreSQL database when the relational backend is enabled. The entire configuration is driven by environment variables consumed by both the Docker Compose orchestration and the server code, enabling a seamless containerized deployment.

Architecture Overview

The deployment consists of three primary components wired together via docker-compose-server.yml.

PostgreSQL Container

The stack uses a standard postgres:13 image defined in docker-compose-server.yml (lines 18–27). This container stores artifact and execution tables accessed by the server through SQLAlchemy. A named volume mounts the data directory to $(CMF_DATA_DIR)/postgres_data on the host, ensuring persistence across restarts.

CMF Server Container

The CMF server is a FastAPI service that implements the CMF REST API. In server/app/db/dbconfig.py (lines 22–26), the server constructs an async SQLAlchemy engine from the environment variables using the URL pattern:

postgresql+asyncpg://{user}:{password}@{host}:{port}/{db}

This engine is initialized when the container starts, provided the database is healthy.

Orchestration and Health Checks

The docker-compose-server.yml file (lines 74–77) defines a depends_on condition that delays the server startup until the PostgreSQL container reports a healthy status. The health check uses pg_isready to verify the database is accepting connections before the FastAPI application attempts to connect.

Python Client Integration

The client-side library reads the same environment variables through cmflib.utils.helper_functions.get_postgres_config() (lines 75–88). This ensures that scripts and notebooks using the cmflib package connect to the same PostgreSQL instance as the server.

Configuration Steps

1. Define Environment Variables

Create a .env file in the repository root (or copy env-example). The five required variables for the PostgreSQL backend are:

POSTGRES_HOST=postgres          # Docker service name resolves via internal network

POSTGRES_USER=myuser
POSTGRES_PASSWORD=mypassword
POSTGRES_PORT=5432
POSTGRES_DB=mlmd

The env-example file in the repository root demonstrates all optional and required variables, including UI and MCP settings.

2. Configure Data Persistence

(Optional) Set CMF_DATA_DIR to a host-side directory where Docker will store the PostgreSQL data volume and other artifacts:

CMF_DATA_DIR=~/cmf_data

3. Launch the Stack

From the repository root, start the services:

docker compose -f docker-compose-server.yml up -d

Docker Compose will:

• Pull the postgres:13 image and create the postgres-data volume.
• Execute the health check (pg_isready) until the database is ready.
• Start the cmf-server container only after the health check passes, guaranteeing a live connection.

To start only the database and server (excluding the UI and Nginx), run:

docker compose -f docker-compose-server.yml up -d postgres cmf-server

4. Verify the Deployment

Test the FastAPI root endpoint:

curl http://localhost:8080/api/

The expected response is {"cmf-server"}. The server logs will also display the resolved local addresses, confirming connectivity to the postgres host.

5. Access the Web Interface (Optional)

If you started the full stack including the UI service, open a browser to http://localhost/. The React front-end communicates with the server via REACT_APP_CMF_API_URL, which defaults to the Nginx proxy address.

Connecting the Python Client to PostgreSQL

The cmflib library can interact directly with the same PostgreSQL store using the PostgresStore implementation in cmflib/store/postgres.py. Load the connection details from the .env file and initialize the client:

from cmflib.utils.helper_functions import get_postgres_config
from cmflib.cmf import Cmf

# Parse the same environment variables used by the server

pg_cfg = get_postgres_config()

# Initialize client with PostgreSQL backend

cmf = Cmf(
    base_url="http://localhost:8080/api",
    store_type="postgres",
    store_kwargs=pg_cfg  # Dict with host, port, user, password, dbname

)

# Query metadata

pipelines = cmf.list_pipelines()
print("Available pipelines:", pipelines)

The store_kwargs dictionary returned by get_postgres_config() maps exactly to the parameters expected by PostgresStore, ensuring the client and server share a consistent configuration source.

Summary

• The CMF server uses environment variables (POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_PORT, POSTGRES_DB) to construct an async PostgreSQL connection string in server/app/db/dbconfig.py.
• docker-compose-server.yml orchestrates the PostgreSQL container and CMF server with a health-check dependency to prevent startup race conditions.
• Data persists to the host filesystem via the CMF_DATA_DIR variable, which maps to a Docker named volume.
• The Python client library reads identical variables via cmflib.utils.helper_functions.get_postgres_config() to connect to the same database instance.
• Verification is performed by curling http://localhost:8080/api/ or checking the server logs for successful PostgreSQL connection messages.

Frequently Asked Questions

What PostgreSQL version does CMF require?

The docker-compose-server.yml file specifies the postgres:13 image. While the server code in server/app/db/dbconfig.py uses SQLAlchemy and asyncpg for compatibility, the tested and supported version in the hewlettpackard/cmf repository is PostgreSQL 13.

Can I use an external PostgreSQL instance instead of the container?

Yes. Set the POSTGRES_HOST variable in your .env file to the external hostname or IP address, and adjust POSTGRES_PORT if necessary. Ensure the CMF server container can reach the host network or expose the necessary ports, and verify that the health check logic in your orchestration accommodates external dependencies.

How does the server handle database migrations and table creation?

The FastAPI application imports init_db() from the database module and executes it on startup, as seen in server/app/main.py. This creates the required tables if they do not exist, using the async SQLAlchemy engine constructed from the environment variables.

Why does the server fail to start with a "Connection refused" error?

This typically occurs when the CMF server container starts before PostgreSQL is ready to accept connections. The default docker-compose-server.yml includes a depends_on condition with a health check using pg_isready to prevent this. If running containers individually, ensure the database is fully initialized before launching the server, or implement a wait script that polls the PostgreSQL port.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}