Backlot Living Storyboard: Real-Time Production Visibility in OpenMontage

The Backlot Living Storyboard is a read-only visual dashboard in OpenMontage that aggregates pipeline state from the projects/<id>/ directory, serving live updates via Server-Sent Events to show exactly where your video production stands.

The Backlot Living Storyboard provides producers and developers with a defensive, non-destructive window into OpenMontage video production pipelines. Unlike tools that modify project state, this board interprets existing files from the projects/ folder and refreshes automatically as the pipeline progresses. It serves as the visual front-end for monitoring stages, assets, and automation health without ever writing to your project directory.

Architecture of the Backlot Living Storyboard

Read-Only Defensive Design

The board operates as a strictly read-only observer. According to backlot/README.md, it never writes to a project directory; it only interprets files that the pipeline produces. When encountering malformed JSON or missing assets, the system degrades gracefully rather than failing, ensuring that observability never interferes with production work.

Live Data Sources

All data derives from the projects/<id>/ folder structure. The system continuously reads:

• Project markers and checkpoints
• scene_plan, script, and asset_manifest files
• Recent event logs (events.jsonl)
• Media renders, snapshots, and music files

Real-Time Updates via Server-Sent Events

The File System Watcher

In backlot/server.py, a background watchfiles watcher monitors the project folder. When it detects filesystem changes, it invalidates the cached project summary and publishes the project ID to all subscribers via Server-Sent Events (SSE) on the /api/project/{id}/events endpoint.

Client-Side Refetching

The frontend subscribes to the SSE stream at /api/project/{id}/events. When the browser receives a change message type, it immediately refetches the current state from /api/project/{id}/state and redraws the storyboard. This push-based mechanism exposes new assets, updated stage statuses, and ongoing tool events without polling overhead.

Building the Board State: load_board_state() Deep Dive

The load_board_state() function in backlot/state.py assembles the complete board representation by aggregating disparate pipeline artifacts.

Stage Rail Construction

The function builds the stage rail from the pipeline manifest (or a fallback list), parsing the current status of each production stage to create the horizontal timeline view.

Storyboard Assembly

It constructs the storyboard by joining scene_plan, script, and asset_manifest files together with live events from the event log. This creates a unified view of the narrative flow alongside production metadata.

Live Flags and Media Discovery

The system discovers media including renders, snapshots, and music files, then adds live flags (live, stalled, generating) based on recent filesystem activity tracked in events.jsonl.

Pipeline Health Detection

Stall Detection

If a stage remains in_progress longer than the configured stall window of 10 minutes, backlot/state.py flags it as stalled. This exposes potentially wedged agents requiring manual intervention or debugging.

Generation Markers

Scenes actively being processed are marked with generating: true based on the most recent start events in events.jsonl, giving immediate visibility into exactly where the automation is currently working.

Practical Implementation: Launching and Querying the Board

Launch the Backlot server from the repository root:

# Launch the server and open the browser

python -m backlot open

# Run the server in the foreground on a specific port

python -m backlot serve --port 4750

Query the live board state programmatically using the Python API:

from backlot.state import load_board_state
from lib.paths import PROJECTS_DIR

project_dir = PROJECTS_DIR / "my-demo-project"
state = load_board_state(project_dir)

print("Current stage:", [s["name"] for s in state["stages"]
      if s["status"] in ("in_progress", "awaiting_human")][0])
print("Storyboard scenes:", len(state["storyboard"]["scenes"]))

Consume the HTTP API for external integrations or debugging:

# Get the JSON representation of the board

curl http://localhost:8000/api/project/my-demo-project/state

# Listen for real-time change events (SSE)

curl -N http://localhost:8000/api/project/my-demo-project/events

Implement frontend updates with the EventSource API:

const evtSource = new EventSource(`/api/project/${projectId}/events`);
evtSource.onmessage = e => {
  const payload = JSON.parse(e.data);
  if (payload.type === 'change') {
    fetch(`/api/project/${projectId}/state`).then(r => r.json()).then(renderBoard);
  }
};

Summary

• The Backlot Living Storyboard provides read-only visibility into OpenMontage video production pipelines, ensuring zero interference with project files.
• Real-time updates flow through Server-Sent Events powered by a watchfiles watcher in backlot/server.py that monitors the projects/<id>/ directory for changes.
• The load_board_state() function in backlot/state.py constructs the complete board by joining scene plans, scripts, asset manifests, and live event logs from events.jsonl.
• Stall detection automatically flags stages stuck in_progress for over 10 minutes, while generation markers indicate active scene processing based on recent start events.
• Developers can consume state via the Python API, HTTP endpoints, or subscribe to live SSE streams for instant UI updates.

Frequently Asked Questions

What makes the Backlot Living Storyboard "read-only"?

The board never writes to the project directory. As implemented in backlot/state.py, it only interprets existing files produced by the pipeline, making it impossible for the observability layer to corrupt or modify production assets. This defensive design ensures that visualizing progress cannot accidentally alter it.

How does the board handle missing or corrupted files?

The defensive architecture specified in backlot/README.md ensures the board degrades gracefully. When encountering malformed JSON or missing assets, the system continues operating and displays available data rather than crashing, maintaining visibility even during partial pipeline failures.

What triggers a real-time update in the browser?

The watchfiles watcher in backlot/server.py detects filesystem changes in projects/<id>/, invalidates the cached summary, and publishes a change event via SSE to /api/project/{id}/events. The browser client receives this push notification and refetches the full state from /api/project/{id}/state to redraw the interface.

How does stall detection identify wedged pipeline agents?

If load_board_state() in backlot/state.py finds a stage with in_progress status lasting longer than the 10-minute stall window (tracked via timestamps in events.jsonl), it sets the stalled flag on that stage. This exposes agents that may have crashed or deadlocked, requiring manual intervention or debugging to clear the pipeline blockage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}