How to Add a New Pipeline to OpenMontage: Complete Guide to YAML, Director, and Schema

Adding a new pipeline to OpenMontage requires creating a YAML manifest file in the pipeline_defs/ directory that defines metadata, skills, orchestration settings, and production stages validated against pipeline_manifest.schema.json.

OpenMontage treats every production workflow as a declarative pipeline manifest written in YAML. The system automatically discovers new pipelines at startup by reading all files in pipeline_defs/ and validating them against the JSON Schema in schemas/pipelines/pipeline_manifest.schema.json. This architecture allows you to extend OpenMontage’s capabilities without modifying core application code.

Understanding the OpenMontage Pipeline Architecture

OpenMontage uses a purely declarative approach to workflow definition. When the application initializes, pipeline_loader.load_pipeline() reads each YAML file and executes jsonschema.validate against the schema definition. The loader caches manifests using an LRU cache for performance, making pipelines immediately available to the orchestration engine and UI. Each manifest acts as a Director Schema that instructs the executive-producer orchestrator how to execute creative stages.

Step-by-Step Guide to Adding a New Pipeline

Step 1: Create the Pipeline Manifest File

Create a new file named <pipeline>.yaml inside the pipeline_defs/ directory. The filename (without extension) becomes the unique pipeline identifier used throughout the system. For example, creating pipeline_defs/my-pipeline.yaml registers "my-pipeline" as a selectable workflow.

Step 2: Define Top-Level Metadata

Populate the required metadata fields that control UI presentation and system behavior:

• name: Human-readable pipeline title
• version: Semantic version string (e.g., "1.0")
• description: Brief explanation of the pipeline's purpose
• category: Classification for UI organization (e.g., "generated", "live-action")
• stability: Release status ("alpha", "beta", or "stable")
• extensions: Boolean flags enabling custom capabilities (custom_scripts, custom_playbooks, custom_skills, custom_tools)

These fields are enforced by pipeline_manifest.schema.json and determine which extension features the pipeline may access.

Step 3: Declare Required Skills and Extensions

Specify the director skills your pipeline requires under the required_skills key. These paths point to instruction-driven skill implementations invoked by the orchestration engine:

required_skills:
  - pipelines/my-pipeline/idea-director
  - pipelines/my-pipeline/script-director
  - pipelines/my-pipeline/compose-director

If your pipeline uses custom assets, set the corresponding extension flags to true in the extensions block and place supporting files under pipelines/<pipeline>/.

Step 4: Configure Orchestration Settings

Define how the executive-producer loop manages execution under the orchestration key:

• mode: Always set to executive-producer for standard workflows
• skill: Path to the executive producer skill implementation
• budget_default_usd: Default budget cap per run
• max_revisions_per_stage: Limit on iterative revisions
• max_send_backs: Maximum return-to-previous-stage operations

The orchestrator reads these values to drive the production loop according to the constraints defined in your manifest.

Step 5: Declare Compatible Playbooks

List recommended and allowed playbooks under compatible_playbooks. These declarations help the UI filter user choices and validate selections:

compatible_playbooks:
  recommended:
    - clean-professional

Step 6: Define Production Stages

Map out your creative workflow by defining stages under the stages array. Each stage requires:

• name: Stage identifier (e.g., "idea", "script", "scene_plan")
• skill: Path to the director skill controlling this stage
• produces: List of artifact types generated
• tools_available: Union of required, optional, and fallback tools
• checkpoint_required: Boolean for persistence requirements
• human_approval_default: Default approval gate setting
• review_focus: Human-readable criteria for reviewers
• success_criteria: Automated validation checkpoints
• required_artifacts_in / optional_artifacts_in: Input dependencies

Step 7: Add Optional Sub-Stages

For complex workflows, declare sub-stages inside a stage's sub_stages array. Sub-stages support conditional gating via the condition field and maintain their own tools_available and review_focus configurations, allowing granular control over specific workflow branches.

Step 8: Validation and Loading

When the server starts, the system automatically validates your YAML against schemas/pipelines/pipeline_manifest.schema.json. Any schema violation aborts loading and logs an error. Upon successful validation, pipeline_loader.list_pipelines() automatically includes your new manifest, making it selectable via the API and UI without additional registration steps.

Pipeline Manifest Code Example

Here is a complete, production-ready manifest demonstrating the declarative structure:

name: my-pipeline
version: "1.0"
description: >-
  Simple example pipeline that stitches a static image onto a background video.
category: generated
stability: beta
extensions:
  custom_scripts: false
  custom_playbooks: false
  custom_skills: false
  custom_tools: false

required_skills:
  - pipelines/my-pipeline/idea-director
  - pipelines/my-pipeline/script-director
  - pipelines/my-pipeline/compose-director

orchestration:
  mode: executive-producer
  skill: pipelines/my-pipeline/executive-producer
  budget_default_usd: 0.10
  max_revisions_per_stage: 2
  max_send_backs: 2

compatible_playbooks:
  recommended:
    - clean-professional

stages:
  - name: idea
    skill: pipelines/my-pipeline/idea-director
    produces:
      - brief
    tools_available: []
    checkpoint_required: true
    human_approval_default: true
    review_focus:
      - Brief matches intended concept
    success_criteria:
      - Schema‑valid brief artifact

  - name: compose
    skill: pipelines/my-pipeline/compose-director
    required_artifacts_in:
      - brief
    produces:
      - final_video
    tools_available:
      - video_compose
    required_tools:
      - video_compose
    checkpoint_required: true
    human_approval_default: false
    review_focus:
      - Output video is playable
    success_criteria:
      - Final video file exists

Accessing Your New Pipeline Programmatically

Use the pipeline loader to programmatically inspect and invoke your new workflow:

from openmontage.lib.pipeline_loader import load_pipeline, list_pipelines

# Verify the new pipeline appears in the registry

print(list_pipelines())          # → ['talking-head', ..., 'my-pipeline']

# Load and inspect the manifest

manifest = load_pipeline("my-pipeline")
print(manifest["stages"][0]["skill"])   # → pipelines/my-pipeline/idea-director

The load_pipeline() function returns the parsed YAML as a Python dictionary with full access to stage configurations, orchestration settings, and skill paths.

Summary

• OpenMontage discovers pipelines automatically by scanning pipeline_defs/ for YAML files at startup.
• Each manifest must validate against schemas/pipelines/pipeline_manifest.schema.json using jsonschema.validate.
• The filename (minus extension) becomes the pipeline identifier referenced by list_pipelines().
• Declarative configuration includes metadata (name, version, stability), orchestration settings, and stage definitions with skill assignments.
• Custom assets require enabling extension flags and placing files under pipelines/<pipeline>/.
• The pipeline_loader.py module handles caching and exposure to the API layer.

Frequently Asked Questions

What file format does OpenMontage use for pipeline definitions?

OpenMontage uses YAML for all pipeline definitions. Each manifest must follow the structure defined in pipeline_manifest.schema.json, which enforces required fields like name, version, stages, and orchestration while validating data types and nested structures.

Where does OpenMontage store pipeline manifests?

Pipeline manifests reside in the pipeline_defs/ directory at the project root. The pipeline_loader.py module reads all .yaml files from this location during initialization. Custom skills and extensions referenced by the manifest should be stored under pipelines/<pipeline-name>/.

Does adding a new pipeline require code changes?

No. Adding a new pipeline is purely declarative—you only need to create the YAML manifest file. Code changes are only necessary if your pipeline requires custom skills, scripts, or tools not already present in the system, and even then, you simply enable the appropriate extensions flags in the manifest rather than modifying core logic.

How does OpenMontage validate pipeline configurations?

Validation occurs automatically when the application starts. The pipeline_loader.load_pipeline() function parses each YAML file and runs jsonschema.validate against schemas/pipelines/pipeline_manifest.schema.json. Schema violations cause immediate loading aborts with descriptive error messages, ensuring only valid configurations enter the execution environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}