CI/CD Pipeline for Colibri: Complete Guide to Automated Builds and Multi-Platform Testing

The CI/CD pipeline for Colibri is defined in .github/workflows/ci.yml and executes 16+ parallel jobs across Linux, macOS, Windows, and ARM64 runners to build the C engine, validate GPU backends (CUDA, HIP, Vulkan), run memory sanitizers, and verify token-exactness through oracle tests on every push or pull request targeting the dev or main branches.

The JustVugg/colibri repository utilizes a comprehensive GitHub Actions workflow to ensure code quality across its multi-language codebase. This CI/CD pipeline for Colibri orchestrates builds for the core C engine, Python bindings, React-based Web UI, and Docker containers while maintaining strict memory safety standards through automated sanitizers and deterministic testing.

Pipeline Configuration and Triggers

Event Triggers and Security Permissions

According to the source code in .github/workflows/ci.yml lines 4‑8, the workflow triggers on pull_request and push events targeting the dev or main branches. This ensures every proposed change and merged commit undergoes validation.

Security is enforced through read-only tokens. Lines 9‑13 restrict permissions to contents: read, preventing compromised steps from pushing tags or releases. This principle of least privilege protects the artifact supply chain.

Core Build Jobs in .github/workflows/ci.yml

Engine Compilation and C Test Suite

The engine job runs on ubuntu-latest and builds the foundation of the project. It executes make colibri inkling to compile the core C engine and the Inkling binary, followed by make test-c to run the C test suite. This job, defined in lines 15‑24, serves as the primary gate for C code correctness.

Multi-Platform Build Matrix

The engines-all-platforms job employs a matrix strategy across Linux, macOS, and Windows runners (lines 48‑66). It loops through the engine list—including colibri, glm53, inkling, and others—to verify that the codebase compiles on every major platform. On macOS, this includes a Metal-specific build, while Windows includes a backend-loader contract test.

GPU Backend Validation

Colibri validates three GPU compute backends through dedicated jobs:

• engine-cuda-syntax: Installs CUDA 12.6.2 via Jimver/cuda-toolkit@v0.2.19 and compiles backend_cuda.cu with nvcc -c to ensure syntactic correctness (lines 12‑42).
• engine-hip-syntax: Runs inside a Docker container (rocm/dev-ubuntu-24.04:6.2) and compiles the HIP backend using hipcc with HIP=1 and HIP_ARCH=gfx1100. It verifies PIC/PIE compatibility using readelf (lines 23‑57).
• vulkan: Installs the Vulkan toolchain and compiles with VK=1, validates GLSL to SPIR-V compilation, and initializes the backend on the Lavapipe software driver using COLI_VULKAN=1 (lines 92‑L45).

Container and Web Builds

The docker job builds the minimal docker/Dockerfile.slim image and verifies that the container runs the launcher and imports the gateway module through a sequence of docker run checks (lines 34‑60).

The web job handles the React-based frontend. It installs Node dependencies with npm ci, builds with npm run build, and executes the Vitest suite via npx vitest run (lines 54‑71).

Testing and Quality Assurance Strategy

Memory Safety and Sanitizers

The sanitizers job runs the C test suite under AddressSanitizer (ASan) and UndefinedBehaviorSanitizer (UBSan) to catch memory corruption and undefined behavior. It executes make -C c test-asan and make -C c fuzz-rans (lines 67‑90), providing early detection of use-after-free and integer overflow bugs.

Token-Exactness Oracle Testing

Deterministic inference is verified through oracle jobs that compare outputs against reference fixtures:

• qwen36-tiny-check and qwen38-tiny-check: Generate tiny-model fixtures and run token-exact checks across multiple cache capacities, repeating the tests under ASan/UBSan (lines 44‑75).
• inkling-oracle: Builds the Inkling engine, generates fixtures using python3 tools/make_tiny_inkling.py, and verifies token output against qwen36_tiny/ref_full.json (lines 93‑L21).

These jobs rely on pinned dependencies specified in c/tools/oracle-requirements.txt and c/tools/requirements-deepseek-v4-tiny.txt.

Performance and Efficiency

The efficiency job runs deterministic telemetry tests, verifying phase presence, I/O-wait behavior, and deterministic token output. It builds colibri, generates the glm_tiny fixture, and executes selected Python unittest cases (lines 43‑L86).

Platform-Specific Validation

Windows CUDA and Python Testing

The windows-cuda-build job verifies MSVC compatibility for the CUDA DLL. Using ilammy/msvc-dev-cmd and the CUDA toolkit, it executes make cuda-dll CUDA_ARCH=sm_80 and make colibri CUDA_DLL=1 (lines 60‑53).

The windows-python-focused job runs a targeted Python test suite under PowerShell to exercise Windows-specific converter and launcher code paths (lines 92‑L10).

ARM64 Architecture Testing

The oracle-arm job runs on ubuntu-24.04-arm to validate NEON optimizations. It builds engines with NEON support (colibri, inkling, kimi_k3, olmoe) and tests integer-kernel exactness on ARM hardware, ensuring parity with x86_64 outputs (lines 112‑L42).

Auxiliary Workflows

Beyond the main CI definition, three auxiliary workflows provide additional automation:

• release.yml: Handles release tagging, artifact uploads, and changelog generation.
• check.yml: Contains supplementary validation steps, including the Linux v4-tiny oracle referenced in the engine-cuda-syntax job (lines 48‑52).
• site.yml: Builds and deploys the project documentation site using Node-based static site generation.

Reproducing CI Steps Locally

You can replicate key CI validation steps on your development machine using the same commands executed in .github/workflows/ci.yml.

Build the Core Engine and Run C Tests

git clone https://github.com/JustVugg/colibri.git
cd colibri/c
make colibri inkling      # Lines 15‑24 equivalent

make test-c               # Runs the C test suite

Validate the Vulkan Backend with Lavapipe

cd colibri/c
sudo apt-get install -y libvulkan-dev glslc mesa-vulkan-drivers
make colibri VK=1         # Compile with Vulkan support

COLI_VULKAN=1 ./colibri   # Initialize on software driver

Execute the Qwen-3.6 Tiny-Oracle Check

cd colibri/c
pip install -r tools/oracle-requirements.txt
make qwen36               # Build the engine binary

COLI_DENSE_I8=0 SNAP=qwen36_tiny_c ./qwen36 1 8 qwen36_tiny/ref_full.json

Build and Test the Web UI

cd colibri/web
npm ci                    # Install exact dependency tree

npm run build             # Production build

npx vitest run            # Execute test suite (Lines 54‑71)

Build Windows CUDA DLL with MSVC

# In PowerShell with MSVC tools initialized

cd colibri/c
make cuda-dll CUDA_ARCH=sm_80
make colibri CUDA_DLL=1

Summary

• 16+ parallel jobs in .github/workflows/ci.yml provide comprehensive coverage of the C engine, Python bindings, and Web UI.
• Multi-platform matrix includes Linux (x86_64 and ARM64), macOS, and Windows to catch platform-specific regressions.
• GPU backend validation ensures syntax and compilation correctness for CUDA, HIP, and Vulkan without requiring physical hardware for basic checks.
• Memory safety is enforced through AddressSanitizer and UndefinedBehaviorSanitizer in dedicated CI jobs.
• Token-exactness oracles guarantee deterministic inference across model architectures using tiny fixtures and reference outputs.
• Local replication is supported through standard make commands and npm scripts, matching the CI environment.

Frequently Asked Questions

What triggers the Colibri CI/CD pipeline?

The pipeline triggers on pull_request and push events targeting the dev or main branches, as defined in lines 4‑8 of .github/workflows/ci.yml. The workflow uses a read-only contents: read permission (lines 9‑13) to minimize security exposure during automated builds.

How does Colibri test GPU backends without physical hardware?

The pipeline uses Lavapipe, a Mesa software Vulkan driver, to initialize the Vulkan backend without GPU hardware (vulkan job). For CUDA and HIP, the engine-cuda-syntax and engine-hip-syntax jobs perform compilation-only checks using nvcc -c and hipcc to verify syntax correctness and linking compatibility without executing device code.

What sanitizers does the Colibri CI/CD pipeline use?

The sanitizers job employs AddressSanitizer (ASan) and UndefinedBehaviorSanitizer (UBSan) through the commands make -C c test-asan and make -C c fuzz-rans (lines 67‑90). These tools detect memory leaks, use-after-free errors, and undefined behavior in the C engine before code reaches production.

How can I run the oracle tests locally?

Install the pinned dependencies from c/tools/oracle-requirements.txt, then run make qwen36 or make qwen38 to build the engine binaries. Execute the oracle check by running the binary with the reference fixture path, such as ./qwen36 1 8 qwen36_tiny/ref_full.json, ensuring COLI_DENSE_I8 and SNAP environment variables are set as shown in the qwen36-tiny-check job configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}