Canonical vs Supplementary Artifacts in the OpenMontage Pipeline Stage Contract

Canonical artifacts are schema‑validated outputs declared in a stage’s produces: manifest that serve as contractual requirements for downstream stages, while supplementary artifacts are optional files stored in the same checkpoint without validation or pipeline obligations.

In the OpenMontage video generation framework, each pipeline stage operates under a strict contract defined by its stage‑director skill and YAML manifest. Understanding the distinction between canonical and supplementary artifacts is essential for building reliable, interdependent stages that correctly hand off data to subsequent processing steps according to the checkpoint-protocol.md specifications.

Understanding the Pipeline Stage Contract

Every OpenMontage stage consists of a stage‑director skill (a Markdown file) and a pipeline manifest (YAML). The manifest contains a produces: list that explicitly names the canonical artifacts the stage must emit. This list forms the technical contract that the next stage depends on to begin execution.

The checkpoint system stores all artifacts—both canonical and supplementary—under the checkpoint["artifacts"] dictionary. However, only canonical entries undergo strict validation and presence checks before the pipeline proceeds, as detailed in docs/ARCHITECTURE.md (lines 72‑87).

What Are Canonical Artifacts?

Canonical artifacts are the primary, schema‑validated outputs that drive the OpenMontage pipeline forward. These artifacts are strictly defined and enforced by the lib/checkpoint.py module:

• Declared in the manifest: Each stage lists its canonical outputs in the produces: array of its YAML definition (e.g., script, asset_manifest).
• Schema validation: Every canonical artifact must conform to a JSON‑Schema definition located in schemas/artifacts/*.schema.json.
• Checkpoint storage: Validated artifacts are written to checkpoint["artifacts"][<name>] by the write_checkpoint function.
• Pipeline dependency: The next stage reads the previous stage’s canonical artifact to determine its input, making these outputs contractual requirements.

The pipeline validates these artifacts against their schemas before saving the checkpoint. If validation fails, the stage cannot mark itself complete, preventing invalid data from reaching downstream stages.

What Are Supplementary Artifacts?

Supplementary artifacts represent auxiliary data that aids human review or debugging but carries no contractual obligation for pipeline progression:

• Undeclared outputs: Any file stored under checkpoint["artifacts"] that does not appear in the produces: list is considered supplementary.
• No schema enforcement: These artifacts are treated as opaque blobs—paths, binaries, or unstructured JSON—without validation against schemas/artifacts/*.schema.json.
• Optional presence: The pipeline does not require supplementary artifacts to consider a stage complete. The Backlot board UI explicitly warns when canonical artifacts are missing but ignores absent supplementary files, as implemented in backlot/ui/board.js (lines 233‑236).
• Typical examples: Preview video clips (preview.mp4), thumbnail images, intermediate render‑stills, B‑roll footage, and cost reports.

These items are primarily useful for the Backlot board’s visual review interface or optional downstream tools that do not participate in the core pipeline contract.

Key Differences Between Artifact Types

The distinction between these artifact types determines how the OpenMontage pipeline treats checkpoint data:

Aspect	Canonical Artifact	Supplementary Artifact
Manifest Declaration	Listed in produces: array	Not declared in manifest
Schema Validation	Validated against schemas/artifacts/*.schema.json	No JSON‑Schema validation
Pipeline Requirement	Required for next stage execution	Optional for stage completion
Checkpoint Validation	Validated by lib/checkpoint.py before save	Stored as opaque data
UI Enforcement	Board shows missing‑artifact error if absent	No UI warnings for absence
Primary Use Case	Data handoff between stages	Human review and debugging
Example Names	script, asset_manifest	preview.mp4, cost_log.json

Implementing Artifacts in Code

When writing checkpoints, developers combine both artifact types into a single dictionary, but only canonical entries receive schema validation.

from lib.checkpoint import write_checkpoint

# Canonical artifact must match schemas/artifacts/script.schema.json

canonical_artifact = {
    "script": {
        "sections": [...],
        "metadata": {...}
    }
}

# Supplementary items have no schema requirements

supplementary = {
    "preview.mp4": "projects/vid123/assets/preview.mp4",
    "cost_log.json": "projects/vid123/cost_log.json"
}

# Both stored in checkpoint, but only 'script' is validated

write_checkpoint(
    pipeline_dir="/projects/vid123",
    project_name="vid123",
    stage_name="script",
    status="completed",
    artifacts={**canonical_artifact, **supplementary},
)

The pipeline manifest explicitly declares which outputs are canonical:

# pipeline_defs/example.yaml

stages:
  - name: script
    skill: pipelines/example/script-director
    produces: [script]          # ← canonical artifact name only

    tools_available: [tts_selector, image_selector]

The stage‑director skill documentation clarifies the distinction for AI agents:

<!-- pipelines/example/script-director.md -->

# Script Director

You must output a **canonical `script` artifact** (see schema).  
Additionally, you may generate a **preview video** (`preview.mp4`) for human review, 
but this file is *supplementary* and not required for the next stage.

Summary

• Canonical artifacts are declared in the produces: manifest array, validated against JSON‑Schemas in schemas/artifacts/, and required for pipeline progression.
• Supplementary artifacts are optional checkpoint entries without schema validation, used primarily for debugging and the Backlot UI.
• The lib/checkpoint.py module enforces canonical artifact schemas before allowing checkpoint completion.
• The Backlot board (backlot/ui/board.js) alerts users only when canonical artifacts are missing from a checkpoint.

Frequently Asked Questions

How does OpenMontage validate canonical artifacts?

The write_checkpoint function in lib/checkpoint.py validates every canonical artifact against its corresponding JSON‑Schema in schemas/artifacts/*.schema.json before persisting the checkpoint. If validation fails, the stage cannot complete, preventing invalid data from reaching downstream stages.

Can downstream stages access supplementary artifacts?

While supplementary artifacts are accessible in the checkpoint dictionary, stages should not depend on them for core logic. They exist for optional processing, debugging, or human review in the Backlot interface. Only canonical artifacts provide the contractual guarantee necessary for reliable stage‑to‑stage communication.

What happens if a canonical artifact is missing?

The Backlot board UI, specifically in backlot/ui/board.js (lines 233‑236), detects missing canonical artifacts and displays an error warning. The pipeline considers the stage incomplete until all produces: entries are present and valid in the checkpoint.

Where are artifact schemas defined in the repository?

Canonical artifact schemas are defined in schemas/artifacts/*.schema.json. Each JSON‑Schema specifies the required structure for artifact types like script or asset_manifest, ensuring type safety across stage boundaries as documented in docs/ARCHITECTURE.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}