Leveraging the GitHub Actions CI/CD Workflow for Building LabNow AI Images: A Complete Guide

The LabNow AI foundation repository automates Docker image builds for data science and AI environments through a single GitHub Actions workflow that uses matrix jobs, profile-driven Dockerfiles, and a Bash helper script to build, tag, and publish images to container registries.

The labnow-ai/lab-foundation repository provides a fully automated CI/CD pipeline for constructing comprehensive Docker images used in AI, data science, and development workflows. Leveraging the GitHub Actions CI/CD workflow for building LabNow AI images ensures every container is built reproducibly, tagged consistently with Git SHA suffixes, and synchronized across multiple registries. The pipeline orchestrates builds for base "atom" images, multi-language "core" images, and specialized utility containers through the coordinated execution of .github/workflows/build-docker.yml and the tool.sh helper script.

Architecture of the LabNow AI CI/CD Pipeline

The pipeline architecture separates concerns between workflow orchestration, image definition, and build utility functions. This separation allows developers to modify image contents without changing CI logic, or adjust registry targets without altering Dockerfiles.

Workflow Definition and Triggers

The central workflow file .github/workflows/build-docker.yml declares three primary triggers that initiate the build process. Any push to main that modifies non-Markdown files automatically starts the pipeline, ensuring images stay current with source changes. Pull requests to main trigger identical build sequences, enabling validation of image builds before code merges. Additionally, the workflow_dispatch event allows manual execution from the GitHub Actions tab with customizable inputs.

The workflow defines environment variables at the job level and uses a matrix strategy to parallelize builds across different image profiles. Each job specifies dependencies—for example, the job-core build requires successful completion of job-base—ensuring the foundational Ubuntu Noble "atom" image exists before constructing language-specific layers.

Image Naming and Registry Management

Image tagging logic resides in tool.sh, which generates namespace variables used across all build steps. The script defines IMG_PREFIX_SRC and IMG_PREFIX_DST to distinguish between base image sources and destination publishing registries, while TAG_SUFFIX creates immutable tags based on the short Git SHA. This naming convention ensures every built image carries a unique, traceable identifier derived from the exact commit that produced it.

Core Components of the Build System

Three primary components handle the technical execution of container builds: the Bash helper script, the Dockerfile hierarchy, and parameterized build arguments.

The tool.sh Helper Script

The tool.sh script in the repository root functions as the build system's standard library. It defines three critical Bash functions used by every job:

• build_image – Wraps docker build with standardized registry prefixes, tag suffixes, and support for --build-arg parameters
• alias_image – Creates additional tags for existing images (for example, mapping python-3.12 to base)
• push_image – Authenticates to the destination registry using DOCKER_REGISTRY_USERNAME and DOCKER_REGISTRY_PASSWORD secrets, then pushes images matching a specified keyword (defaulting to "second" unless overridden)

Each job in the workflow begins by sourcing this script: source ./tool.sh loads the namespace variables and makes these functions available to subsequent commands.

Dockerfile Families

The repository maintains two primary Dockerfile families that establish an inheritance pattern:

docker_atom/Dockerfile constructs the minimal "atom" base image containing Ubuntu Noble and essential system utilities. All other images derive from this foundation, ensuring consistent operating system layers and reducing overall build time through Docker layer caching.

docker_core/Dockerfile builds higher-level development environments by installing language-specific toolchains. It accepts build arguments like ARG_PROFILE_PYTHON and ARG_PROFILE_R to conditionally install Python, R, Java, Go, Julia, or LaTeX distributions based on the target use case. This parameterization allows a single Dockerfile to generate specialized variants (datascience, torch, NLP, computer vision) without code duplication.

Profile-Driven Build Arguments

The build system implements a profile-driven architecture where jobs pass --build-arg values to enable specific language toolchains. For example, the core job passes multiple profiles simultaneously:

--build-arg "ARG_PROFILE_PYTHON=base,datascience,mkl,database,nlp,cv,chem,tf2,torch" \
--build-arg "ARG_PROFILE_R=base,datascience" \
--build-arg "ARG_PROFILE_NODEJS=base" \
--build-arg "ARG_PROFILE_JAVA=base,maven"

These arguments trigger conditional installation steps within docker_core/Dockerfile, which invokes helper scripts from docker_core/work/script-setup.sh (and related variants) to install conda environments, system packages, and language runtimes.

Build Process Flow

The pipeline executes through six distinct phases for every image:

1. Workspace preparation – actions/checkout@v4 retrieves the repository source
2. Environment setup – Sourcing tool.sh initializes REGISTRY_SRC, REGISTRY_DST, and TAG_SUFFIX variables
3. Base resolution – The build pulls the atom image from REGISTRY_SRC as the foundation layer
4. Image compilation – build_image executes Docker with profile-specific build arguments, generating tags suffixed with the Git SHA
5. Registry push – push_image authenticates to REGISTRY_DST and uploads images containing the default keyword
6. Dependency tracking – Jobs declare needs relationships (e.g., job-core depends on job-base) to maintain build order

Docker-Kit Synchronization

After all primary images build successfully, the workflow executes a job-docker_kit job that runs a specialized synchronization utility. This job builds the docker-kit image from docker_docker_kit/Dockerfile, which contains the Python script run_jobs.py.

The synchronizer mirrors all newly built images to a secondary registry defined by the DOCKER_MIRROR_REGISTRY environment variable. It authenticates using a JSON credentials file constructed from repository secrets, ensuring high availability by maintaining copies across geographically distributed or organizationally separate registries.

Practical Examples

Manually Triggering the Workflow

Navigate to the Actions tab in the labnow-ai/lab-foundation repository, select the build-docker-images workflow, and click Run workflow. This initiates the full pipeline without requiring a code commit, useful for rebuilding images after base image security updates.

Inspecting Built Images Locally

After the workflow completes, verify image contents by pulling and interrogating the container:

docker pull quay.io/labnow/core:latest

docker run --rm quay.io/labnow/core:latest bash -c 'python --version && R --version && julia --version'

This command validates that the profile-driven build successfully installed the expected language runtimes.

Running the Docker-Kit Synchronizer Locally

Debug registry synchronization issues by executing the synchronizer outside CI:

export REGISTRY_DST=quay.io
export REGISTRY_SRC=quay.io
export DOCKER_MIRROR_REGISTRY=quay.io/mirror
export AUTH_FILE_CONTENT='{"username":"user","password":"token"}'

source ./tool.sh && build_image docker-kit latest docker_docker_kit/Dockerfile && push_image docker-kit

docker run --rm \
  -v "$(pwd)":/tmp \
  -w /tmp \
  ${IMG_PREFIX_DST:-labnow}/docker-kit \
  python /opt/utils/image-syncer/run_jobs.py \
  --auth-file=/tmp/.github/workflows/auth.json

Adding a New Language Profile

Extend the build system to support new toolchains by modifying docker_core/Dockerfile to accept a new ARG_PROFILE_RUST parameter, then add a dedicated job:

job-rust-nightly:
  name: 'rust-nightly'
  needs: job-base
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - run: |
        source ./tool.sh
        build_image rust-nightly latest docker_core/Dockerfile \
          --build-arg "ARG_PROFILE_RUST=nightly"
        push_image

Commit and push to trigger automatic building and publishing of the new image variant.

Summary

• The .github/workflows/build-docker.yml file orchestrates all image builds through a matrix of parameterized jobs triggered by pushes, pull requests, or manual dispatch.
• The tool.sh script provides reusable Bash functions (build_image, push_image) and generates registry-specific naming variables including Git SHA-based tags.
• Two-tier image architecture: docker_atom/Dockerfile creates the Ubuntu Noble base, while docker_core/Dockerfile installs language profiles via ARG_PROFILE_* build arguments.
• Profile-driven builds allow a single Dockerfile to generate specialized variants (Python datascience, R statistics, Java development) through conditional build arguments.
• The docker-kit job runs run_jobs.py from docker_docker_kit/Dockerfile to synchronize images to a mirror registry defined by DOCKER_MIRROR_REGISTRY.

Frequently Asked Questions

How does the LabNow AI workflow handle image tagging and versioning?

The workflow generates immutable tags using the short Git SHA through the TAG_SUFFIX variable defined in tool.sh. Every image receives a unique tag derived from the commit that triggered the build, ensuring complete traceability between source code and container artifacts. The alias_image function can create additional semantic tags (like latest or python-3.12) for human-friendly consumption while maintaining the SHA-tagged version for reproducibility.

What triggers the GitHub Actions workflow in the lab-foundation repository?

The workflow triggers on three events: any push to the main branch (excluding Markdown-only changes), pull requests targeting main, and manual workflow_dispatch events from the GitHub UI. This configuration ensures images rebuild automatically when source code changes, allows pre-merge validation through PR builds, and provides manual execution capability for maintenance or security updates.

How can I customize which programming languages or tools are included in a LabNow AI image?

Modify the --build-arg parameters passed to the build_image function in the workflow job definition. The docker_core/Dockerfile accepts ARG_PROFILE_* arguments (such as ARG_PROFILE_PYTHON, ARG_PROFILE_R, or ARG_PROFILE_JAVA) that trigger installation scripts in docker_core/work/script-setup*.sh. Each argument accepts comma-separated profile names (e.g., base,datascience,torch) that conditionally install specific conda environments, system packages, and language runtimes.

What is the purpose of the docker-kit synchronization step?

The docker-kit job mirrors successfully built images from the primary registry (REGISTRY_DST) to a secondary mirror registry (DOCKER_MIRROR_REGISTRY). It executes a Python script (run_jobs.py) from within the docker-kit container to replicate images across organizational boundaries or geographic regions. This synchronization ensures high availability and provides redundancy for critical AI and data science environments deployed through the LabNow AI foundation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}