# How to Add a New Pipeline to OpenMontage: Complete Guide to YAML, Director, and Schema

> Learn how to add a new pipeline to OpenMontage. This guide covers YAML, Director, and Schema steps for defining and validating your pipeline manifests.

- Repository: [Calesthio/OpenMontage](https://github.com/calesthio/OpenMontage)
- Tags: how-to-guide
- Published: 2026-08-30

---

**Adding a new pipeline to OpenMontage requires creating a YAML manifest file in the `pipeline_defs/` directory that defines metadata, skills, orchestration settings, and production stages validated against [`pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_manifest.schema.json).**

OpenMontage treats every production workflow as a **declarative pipeline manifest** written in YAML. The system automatically discovers new pipelines at startup by reading all files in `pipeline_defs/` and validating them against the JSON Schema in [`schemas/pipelines/pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/schemas/pipelines/pipeline_manifest.schema.json). This architecture allows you to extend OpenMontage’s capabilities without modifying core application code.

## Understanding the OpenMontage Pipeline Architecture

OpenMontage uses a purely declarative approach to workflow definition. When the application initializes, `pipeline_loader.load_pipeline()` reads each YAML file and executes `jsonschema.validate` against the schema definition. The loader caches manifests using an LRU cache for performance, making pipelines immediately available to the orchestration engine and UI. Each manifest acts as a **Director Schema** that instructs the executive-producer orchestrator how to execute creative stages.

## Step-by-Step Guide to Adding a New Pipeline

### Step 1: Create the Pipeline Manifest File

Create a new file named `<pipeline>.yaml` inside the `pipeline_defs/` directory. The filename (without extension) becomes the unique pipeline identifier used throughout the system. For example, creating [`pipeline_defs/my-pipeline.yaml`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_defs/my-pipeline.yaml) registers "my-pipeline" as a selectable workflow.

### Step 2: Define Top-Level Metadata

Populate the required metadata fields that control UI presentation and system behavior:

- **name**: Human-readable pipeline title
- **version**: Semantic version string (e.g., "1.0")
- **description**: Brief explanation of the pipeline's purpose
- **category**: Classification for UI organization (e.g., "generated", "live-action")
- **stability**: Release status ("alpha", "beta", or "stable")
- **extensions**: Boolean flags enabling custom capabilities (`custom_scripts`, `custom_playbooks`, `custom_skills`, `custom_tools`)

These fields are enforced by [`pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_manifest.schema.json) and determine which extension features the pipeline may access.

### Step 3: Declare Required Skills and Extensions

Specify the director skills your pipeline requires under the `required_skills` key. These paths point to instruction-driven skill implementations invoked by the orchestration engine:

```yaml
required_skills:
  - pipelines/my-pipeline/idea-director
  - pipelines/my-pipeline/script-director
  - pipelines/my-pipeline/compose-director

```

If your pipeline uses custom assets, set the corresponding extension flags to `true` in the `extensions` block and place supporting files under `pipelines/<pipeline>/`.

### Step 4: Configure Orchestration Settings

Define how the executive-producer loop manages execution under the `orchestration` key:

- **mode**: Always set to `executive-producer` for standard workflows
- **skill**: Path to the executive producer skill implementation
- **budget_default_usd**: Default budget cap per run
- **max_revisions_per_stage**: Limit on iterative revisions
- **max_send_backs**: Maximum return-to-previous-stage operations

The orchestrator reads these values to drive the production loop according to the constraints defined in your manifest.

### Step 5: Declare Compatible Playbooks

List recommended and allowed playbooks under `compatible_playbooks`. These declarations help the UI filter user choices and validate selections:

```yaml
compatible_playbooks:
  recommended:
    - clean-professional

```

### Step 6: Define Production Stages

Map out your creative workflow by defining stages under the `stages` array. Each stage requires:

- **name**: Stage identifier (e.g., "idea", "script", "scene_plan")
- **skill**: Path to the director skill controlling this stage
- **produces**: List of artifact types generated
- **tools_available**: Union of required, optional, and fallback tools
- **checkpoint_required**: Boolean for persistence requirements
- **human_approval_default**: Default approval gate setting
- **review_focus**: Human-readable criteria for reviewers
- **success_criteria**: Automated validation checkpoints
- **required_artifacts_in** / **optional_artifacts_in**: Input dependencies

### Step 7: Add Optional Sub-Stages

For complex workflows, declare sub-stages inside a stage's `sub_stages` array. Sub-stages support conditional gating via the `condition` field and maintain their own `tools_available` and `review_focus` configurations, allowing granular control over specific workflow branches.

### Step 8: Validation and Loading

When the server starts, the system automatically validates your YAML against [`schemas/pipelines/pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/schemas/pipelines/pipeline_manifest.schema.json). Any schema violation aborts loading and logs an error. Upon successful validation, `pipeline_loader.list_pipelines()` automatically includes your new manifest, making it selectable via the API and UI without additional registration steps.

## Pipeline Manifest Code Example

Here is a complete, production-ready manifest demonstrating the declarative structure:

```yaml
name: my-pipeline
version: "1.0"
description: >-
  Simple example pipeline that stitches a static image onto a background video.
category: generated
stability: beta
extensions:
  custom_scripts: false
  custom_playbooks: false
  custom_skills: false
  custom_tools: false

required_skills:
  - pipelines/my-pipeline/idea-director
  - pipelines/my-pipeline/script-director
  - pipelines/my-pipeline/compose-director

orchestration:
  mode: executive-producer
  skill: pipelines/my-pipeline/executive-producer
  budget_default_usd: 0.10
  max_revisions_per_stage: 2
  max_send_backs: 2

compatible_playbooks:
  recommended:
    - clean-professional

stages:
  - name: idea
    skill: pipelines/my-pipeline/idea-director
    produces:
      - brief
    tools_available: []
    checkpoint_required: true
    human_approval_default: true
    review_focus:
      - Brief matches intended concept
    success_criteria:
      - Schema‑valid brief artifact

  - name: compose
    skill: pipelines/my-pipeline/compose-director
    required_artifacts_in:
      - brief
    produces:
      - final_video
    tools_available:
      - video_compose
    required_tools:
      - video_compose
    checkpoint_required: true
    human_approval_default: false
    review_focus:
      - Output video is playable
    success_criteria:
      - Final video file exists

```

## Accessing Your New Pipeline Programmatically

Use the pipeline loader to programmatically inspect and invoke your new workflow:

```python
from openmontage.lib.pipeline_loader import load_pipeline, list_pipelines

# Verify the new pipeline appears in the registry

print(list_pipelines())          # → ['talking-head', ..., 'my-pipeline']

# Load and inspect the manifest

manifest = load_pipeline("my-pipeline")
print(manifest["stages"][0]["skill"])   # → pipelines/my-pipeline/idea-director

```

The `load_pipeline()` function returns the parsed YAML as a Python dictionary with full access to stage configurations, orchestration settings, and skill paths.

## Summary

- OpenMontage discovers pipelines automatically by scanning `pipeline_defs/` for YAML files at startup.
- Each manifest must validate against [`schemas/pipelines/pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/schemas/pipelines/pipeline_manifest.schema.json) using `jsonschema.validate`.
- The filename (minus extension) becomes the pipeline identifier referenced by `list_pipelines()`.
- Declarative configuration includes metadata (`name`, `version`, `stability`), orchestration settings, and stage definitions with skill assignments.
- Custom assets require enabling extension flags and placing files under `pipelines/<pipeline>/`.
- The [`pipeline_loader.py`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_loader.py) module handles caching and exposure to the API layer.

## Frequently Asked Questions

### What file format does OpenMontage use for pipeline definitions?

OpenMontage uses **YAML** for all pipeline definitions. Each manifest must follow the structure defined in [`pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_manifest.schema.json), which enforces required fields like `name`, `version`, `stages`, and `orchestration` while validating data types and nested structures.

### Where does OpenMontage store pipeline manifests?

Pipeline manifests reside in the `pipeline_defs/` directory at the project root. The [`pipeline_loader.py`](https://github.com/calesthio/OpenMontage/blob/main/pipeline_loader.py) module reads all `.yaml` files from this location during initialization. Custom skills and extensions referenced by the manifest should be stored under `pipelines/<pipeline-name>/`.

### Does adding a new pipeline require code changes?

No. Adding a new pipeline is **purely declarative**—you only need to create the YAML manifest file. Code changes are only necessary if your pipeline requires custom skills, scripts, or tools not already present in the system, and even then, you simply enable the appropriate `extensions` flags in the manifest rather than modifying core logic.

### How does OpenMontage validate pipeline configurations?

Validation occurs automatically when the application starts. The `pipeline_loader.load_pipeline()` function parses each YAML file and runs `jsonschema.validate` against [`schemas/pipelines/pipeline_manifest.schema.json`](https://github.com/calesthio/OpenMontage/blob/main/schemas/pipelines/pipeline_manifest.schema.json). Schema violations cause immediate loading aborts with descriptive error messages, ensuring only valid configurations enter the execution environment.