CI/CD Pipeline for the Shadowbroker Backend: GitHub Actions Configuration Explained

Yes, the Shadowbroker repository implements a complete CI/CD pipeline using GitHub Actions that automatically validates code quality, runs security scans, executes smoke tests, and publishes multi-architecture Docker images for the backend service.

The Shadowbroker project maintains a robust DevOps workflow defined in the .github/workflows directory. According to the source code in BigBodyCobain/Shadowbroker, every push and pull request to the main branch triggers automated validation, while successful builds result in containerized artifacts published to the GitHub Container Registry.

Continuous Integration Workflow Structure

The primary CI logic resides in .github/workflows/ci.yml. This workflow orchestrates the backend validation through a series of sequential jobs designed to catch issues before they reach production.

Secret Scanning and Security Validation

The pipeline begins with a security gate defined in the backend job. Before any code analysis occurs, the workflow executes backend/scripts/scan-secrets.sh to detect potential credential leaks or sensitive data exposure.

# Run the same secret scan locally that CI executes

bash backend/scripts/scan-secrets.sh --all

This step ensures that no accidental commits of API keys, tokens, or passwords proceed to subsequent stages.

Python Environment and Dependency Management

The backend targets Python 3.11 and utilizes the modern uv package manager for dependency resolution. The CI workflow installs uv, then synchronizes the environment using the frozen lockfile to guarantee reproducible builds:

# Install development dependencies exactly as CI does

uv sync --frozen --group dev

This command references backend/pyproject.toml to install both runtime and development dependencies, ensuring that linting tools and test frameworks are available for subsequent steps.

Linting and Code Quality Enforcement

Static analysis runs via ruff for linting and black for code formatting validation. These tools enforce consistent style across the codebase without manual intervention:

# Execute linting and format checks

uv run ruff check .
uv run black --check .

If either check fails, the workflow terminates immediately, preventing non-compliant code from merging into the main branch.

Smoke Testing with Pytest

The final CI stage executes a comprehensive test suite using pytest. The backend job specifically targets mesh functionality and release verification through the following test modules:

• backend/tests/mesh/test_mesh_node_bootstrap_runtime.py
• backend/tests/mesh/test_mesh_infonet_sync_support.py
• backend/tests/mesh/test_mesh_canonical.py
• backend/tests/mesh/test_mesh_merkle.py
• backend/tests/test_release_helper.py

# Run the exact smoke tests executed in CI

uv run pytest \
  backend/tests/mesh/test_mesh_node_bootstrap_runtime.py \
  backend/tests/mesh/test_mesh_infonet_sync_support.py \
  backend/tests/mesh/test_mesh_canonical.py \
  backend/tests/mesh/test_mesh_merkle.py \
  backend/tests/test_release_helper.py -v --tb=short

These tests validate mesh bootstrapping, network synchronization, canonical validation, and Merkle tree integrity—critical components of the Shadowbroker backend architecture.

Continuous Delivery and Container Publishing

Once the CI gate succeeds, the .github/workflows/docker-publish.yml workflow triggers to build and distribute deployment artifacts.

Multi-Architecture Docker Builds

The pipeline constructs backend images from backend/Dockerfile using a build matrix that targets both linux/amd64 and linux/arm64 architectures. This multi-platform support ensures compatibility across diverse deployment environments, from cloud servers to ARM-based edge devices.

# Build the backend image locally using the same Dockerfile as CI

docker build -f backend/Dockerfile -t shadowbroker-backend:dev .

GitHub Container Registry Integration

Successfully built images receive tags corresponding to the repository state and push multi-arch manifests to the GitHub Container Registry (GHCR). The backend image publishes as ghcr.io/bigbodycobain/shadowbroker-backend (following the standard GHCR naming convention), making it immediately available for production deployments or further staging environments.

Local Development and CI Replication

Developers can replicate the entire CI pipeline locally to debug issues before committing. The following sequence mirrors the automated workflow exactly:

# Clone and enter the repository

git clone https://github.com/BigBodyCobain/Shadowbroker.git
cd Shadowbroker

# Install uv if not present

curl -LsSf https://astral.sh/uv/install.sh | sh

# Execute security scan

bash backend/scripts/scan-secrets.sh --all

# Setup Python environment

cd backend
uv sync --frozen --group dev

# Validate code quality

uv run ruff check .
uv run black --check .

# Execute test suite

uv run pytest \
  backend/tests/mesh/test_mesh_node_bootstrap_runtime.py \
  backend/tests/mesh/test_mesh_infonet_sync_support.py \
  backend/tests/mesh/test_mesh_canonical.py \
  backend/tests/mesh/test_mesh_merkle.py \
  backend/tests/test_release_helper.py -v --tb=short

Summary

• The Shadowbroker backend uses GitHub Actions defined in .github/workflows/ci.yml and .github/workflows/docker-publish.yml to automate validation and deployment.
• Security scanning runs first via backend/scripts/scan-secrets.sh to prevent credential leaks.
• Python 3.11 and uv manage dependencies in a reproducible environment using backend/pyproject.toml.
• Code quality gates enforce standards through ruff linting and black formatting checks.
• Smoke testing covers critical mesh operations including bootstrapping, synchronization, and Merkle validation.
• Multi-arch Docker images (amd64/arm64) publish automatically to GHCR after successful CI completion.

Frequently Asked Questions

What triggers the Shadowbroker backend CI pipeline?

The workflow triggers on every push and pull request targeting the main branch. This ensures that all proposed changes undergo secret scanning, linting, and testing before code review or merge operations complete.

How does the backend handle dependency management in CI?

The pipeline uses the uv package manager with the --frozen flag to install dependencies exactly as specified in the lockfile. The command uv sync --frozen --group dev installs both production and development dependencies from backend/pyproject.toml, ensuring consistent environments across CI runners and local development machines.

Can I run the CI checks locally before pushing?

Yes. You can execute the identical sequence locally by installing uv, running bash backend/scripts/scan-secrets.sh --all, executing uv sync --frozen --group dev, then running uv run ruff check ., uv run black --check ., and the specific pytest commands targeting the mesh test suite and release helper tests.

Where are the Docker images published?

After successful CI completion, the .github/workflows/docker-publish.yml workflow pushes multi-architecture images to the GitHub Container Registry at ghcr.io/bigbodycobain/shadowbroker-backend. These images support both linux/amd64 and linux/arm64 architectures for broad deployment compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}