What Verification Scripts Are Run in CI: Complete Guide to diagram-design Testing

The cathrynlavery/diagram-design repository executes over 30 Python verification scripts from the scripts/ directory in its CI pipeline, following a verify-<feature>.py naming convention and paired with corresponding test-verify-<feature>.py test modules to validate diagram generation, asset handling, and plugin packaging.

The continuous integration (CI) pipeline for the diagram-design repository relies on a comprehensive suite of Python verification scripts to ensure code quality. These scripts are orchestrated by .github/workflows/ci.yml and validate everything from diagram rendering to plugin packaging. Understanding what verification scripts are run in CI and their standardized structure is essential for contributors debugging pipeline failures or extending the codebase.

CI Workflow Configuration

According to the diagram-design source code, the CI workflow defined in .github/workflows/ci.yml iterates over the Python files in the scripts/ directory. During each pipeline run, the workflow executes these scripts using the Python interpreter to ensure that diagram generation, asset handling, and plugin packaging work correctly. Any script that exits with a non-zero status immediately halts the pipeline, ensuring regressions are caught before merge.

Comprehensive Script Inventory

While most verification scripts follow the verify-<feature>.py pattern, the suite includes auxiliary tools using lint-*, build-*, and fix-* prefixes for specialized tasks. The following tables detail the primary verification scripts and their corresponding test modules.

Diagram Type Verification

These scripts validate specific diagram implementations against reference outputs:

Diagram Type	Verification Script	Test Module
Beeswarm	scripts/verify-beeswarm.py	scripts/test-verify-beeswarm.py
Bubble	scripts/verify-bubble.py	scripts/test-verify-bubble.py
Dumbbell	scripts/verify-dumbbell.py	scripts/test-verify-dumbbell.py
Motion	scripts/verify-motion.py	scripts/test-verify-motion.py
Polar	scripts/verify-polar.py	scripts/test-verify-polar.py
Ridgeline	scripts/verify-ridgeline.py	scripts/test-verify-ridgeline.py
Sankey	scripts/verify-sankey.py	scripts/test-verify-sankey.py
Slopegraph	scripts/verify-slopegraph.py	scripts/test-verify-slopegraph.py
Treemap	scripts/verify-treemap.py	scripts/test-verify-treemap.py
Waterfall	scripts/verify-waterfall.py	scripts/test-verify-waterfall.py

Import and Integration Validation

These verify third-party diagram format support:

• scripts/verify-mermaid-import.py – Validates Mermaid diagram imports (no paired test module).
• scripts/verify-excalidraw-import.py – Ensures Excalidraw file compatibility, tested by scripts/test-verify-excalidraw-import.py.
• scripts/verify-drawio-import.py – Checks Draw.io import functionality, tested by scripts/test-verify-drawio-import.py.

Asset and Build Verification

Scripts that check visual assets, skin files, and build artifacts:

• scripts/verify-screenshot-freshness.py – Ensures screenshots match current renders; paired with scripts/test-self-check.py.
• scripts/verify-skin-polarity.py – Validates skin configuration; tested by scripts/test-verify-skin-polarity.py.
• scripts/verify-geometry.py – Checks geometric calculations; tested by scripts/test-verify-geometry.py.
• scripts/lint-render.py – Performs rendering verification (no separate test module).
• scripts/lint-skin.py – Validates skin file syntax (no separate test module).

Plugin and Packaging Verification

These ensure the plugin release process works correctly:

• scripts/verify-plugin-package.py – Validates plugin package structure; tested by scripts/test-plugin-package.py.
• scripts/verify-bump.py – Tests version bumping logic; paired with scripts/test-verify-bump.py.
• scripts/verify-doctor.py – Runs diagnostic checks; tested by scripts/test-verify-doctor.py.
• scripts/verify-docs-sync.py – Ensures documentation synchronization; tested by scripts/test-verify-docs-sync.py.
• scripts/bump-plugin-version.py – Handles version increments (no separate test module).

Utility and Maintenance Scripts

Supporting scripts for build processes and data integrity:

• scripts/verify-block-registry.py – Validates block registration; tested by scripts/test-verify-block-registry.py.
• scripts/verify-sequence-oauth.py – Checks OAuth flow diagrams; tested by scripts/test-verify-sequence-oauth.py.
• scripts/verify-semantic-motion.py – Validates motion semantics; tested by scripts/test-verify-semantic-motion.py.
• scripts/fix-mojibake.py – Corrects character encoding; tested by scripts/test-fix-mojibake.py.
• scripts/build-readme-thumbs.py – Generates README thumbnails; tested by scripts/test-build-readme-thumbs.py.
• scripts/build-icons.py – Handles icon building; tested by scripts/test-build-icons-quoted-attributes.py and scripts/test-build-icons-devicon.py.
• scripts/screenshot_catalog.py – Generates screenshot catalogs (no separate test module).

Standard Script Architecture

As implemented in the scripts directory, each verification script follows a uniform four-part structure to ensure maintainability and predictable CI behavior.

1. Imports

Scripts begin with standard library modules and repository-specific helpers:

import os
import json
import subprocess

# Repository-specific diagram utilities

2. Helper Functions

These functions locate diagram assets, invoke rendering tools, or compare generated output against reference files:

def locate_diagram_assets(pattern):
    """Find diagram files matching the verification pattern."""
    return [f for f in os.listdir('assets') if pattern in f]

def compare_output(generated, reference_path):
    """Compare generated output against stored reference file."""
    with open(reference_path, 'r') as ref:
        return generated == ref.read()

3. Main Verification Logic

Each script implements an entry point that runs checks and prints success or failure status:

if __name__ == "__main__":
    assets = locate_diagram_assets("waterfall")
    if not assets:
        print("FAIL: No waterfall assets found")
        exit(1)
    print("PASS: Waterfall verification complete")
    exit(0)

4. Return Codes

CI interprets exit codes strictly. A return value of 0 indicates success, while any non-zero exit status causes the pipeline to stop and report the issue. This design allows scripts to fail fast upon detecting invalid diagram generation or corrupted assets.

Testing the Verification Suite

The paired test modules in the scripts/ directory typically use pytest to import the verification functions and provide isolated unit tests for edge cases. For example, while scripts/verify-waterfall.py performs end-to-end validation, scripts/test-verify-waterfall.py tests individual helper functions with mock inputs to ensure consistent behavior across different environments.

Some scripts like scripts/test-lint-a11y.py serve dual purposes as both verification tools and test implementations, checking accessibility compliance without requiring a separate paired module.

Summary

• The diagram-design CI pipeline executes Python verification scripts from the scripts/ directory via .github/workflows/ci.yml.
• Most scripts follow the verify-<feature>.py naming convention with corresponding test-verify-<feature>.py modules, while auxiliary tools use lint-*, build-*, or fix-* prefixes.
• The verification suite covers diagram types (waterfall, treemap, sankey), import formats (Mermaid, Excalidraw, Draw.io), asset freshness, skin validation, and plugin packaging.
• Each script uses a four-part structure: imports, helper functions, main entry point with if __name__ == "__main__":, and explicit exit codes.
• Non-zero exit statuses immediately halt CI, preventing broken builds from merging while pytest-based test modules provide additional unit testing coverage.

Frequently Asked Questions

What triggers the verification scripts to run in CI?

The .github/workflows/ci.yml workflow file configures GitHub Actions to iterate over the scripts/ directory and execute verification scripts with the Python interpreter. Every commit and pull request triggers this workflow, which runs the complete suite to ensure diagram generation and asset handling remain functional before code merges.

How do I add a new verification script to the CI pipeline?

Create a Python file named verify-<feature>.py in the scripts/ directory following the standard four-part structure. Ensure the script exits with code 0 on success and non-zero on failure. Optionally add a corresponding test-verify-<feature>.py module using pytest for unit testing. The CI workflow automatically discovers and executes new scripts matching the verification patterns on the next run.

Why do some verification scripts lack corresponding test modules?

Certain scripts like scripts/verify-mermaid-import.py and utility scripts such as scripts/screenshot_catalog.py perform standalone validation or function as build tools where verification is inherent in execution success. These either validate external dependencies where mocking is impractical, or they execute deterministic build processes that do not require separate unit testing.

What happens when a verification script detects a failure?

When any script exits with a non-zero status code, the CI pipeline immediately stops and reports the failure. This exit code propagates from the Python process to the GitHub Actions runner, blocking the pull request merge and alerting contributors to specific issues in diagram generation, asset corruption, or packaging logic detected by the affected verification script.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}