# How the Modly Workflow System Works: Node-Based Execution in lightningpixel/modly

> Discover how Modly's node-based execution engine builds AI pipelines using a DAG. Learn about its deterministic workflow and handling of branching, looping, and interactive pausing.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: internals
- Published: 2026-08-19

---

**The Modly workflow system implements a deterministic, node-based execution engine that orchestrates AI pipelines using a directed acyclic graph (DAG) of `WFNode` and `WFEdge` objects, coordinated by Zustand stores and a topological sorter that handles branching, looping, and interactive pausing via Wait nodes.**

The Modly workflow system, found in the `lightningpixel/modly` repository, provides a visual programming environment for chaining AI model calls, control flow logic, and 3D output nodes. Built on a DAG structure defined in [`electron.d.ts`](https://github.com/lightningpixel/modly/blob/main/electron.d.ts) and managed through dedicated Zustand stores, the engine supports everything from linear processing pipelines to complex branching workflows requiring user intervention at specific points.

## Core Architecture: DAG and State Management

At its foundation, every workflow is a **directed acyclic graph** composed of `WFNode` and `WFEdge` objects. The architecture separates concerns between two primary Zustand stores:

- **`useWorkflowsStore`** ([`src/shared/stores/workflowsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/workflowsStore.ts)): Handles workflow persistence, folder organization, and UI tab state.
- **`useWorkflowRunStore`** ([`src/areas/workflows/workflowRunStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/workflowRunStore.ts)): Controls the execution state machine (`idle`, `running`, `paused`, `done`, `error`) and drives the runner loop.

State changes flow through the graph according to behaviors defined in [`src/areas/workflows/nodeBehaviors.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/nodeBehaviors.ts), which classifies nodes as passthrough, branch starters, scene outputs, or branch consumers.

## Workflow Definition and Persistence

Workflows persist through the Electron IPC bridge exposed in [`workflowsStore.ts`](https://github.com/lightningpixel/modly/blob/main/workflowsStore.ts). On initialization, the store deduplicates workflows by `id`, restores open tabs, and loads folder metadata from `localStorage` under the `FOLDERS_KEY`.

The store exposes CRUD operations via `window.electron.workflows.*` methods:

```ts
// Loading with legacy migration support
await window.electron.workflows.list();
const wf = migrateWorkflow(entry); // Handles legacy format conversion
set({ workflows: list, openIds, activeId, folders });

```

Folder management functions (`addFolder`, `removeFolder`, `setFolderColor`, `toggleFolderBookmark`) allow users to organize workflows in the UI sidebar, while static validation occurs in [`src/areas/workflows/preflight.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/preflight.ts) before execution begins.

## Node Behavior Registry

The execution engine relies on a behavior registry in [`src/areas/workflows/nodeBehaviors.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/nodeBehaviors.ts) to determine how data flows through each node type:

| Behavior | Function | Example Nodes |
|----------|----------|---------------|
| **passthrough** | Data flows unchanged | `waitNode` |
| **branchStarter** | Splits execution into user-driven sub-DAGs | `waitNode` (interactive pause points) |
| **sceneOutput** | Final sink pushing meshes to 3D viewer | `outputNode` |
| **branchConsumer** | Consumes single mesh from branch | Extension nodes |

Helper functions like `isPassthrough`, `isBranchStarter`, `resolveDataSource`, and `nearestUpstreamWaits` enable the runner to analyze graph structure and determine execution paths without executing code.

## Execution Engine and Run Preparation

When `useWorkflowRunStore.run(workflow, extensions)` is invoked, the engine performs preflight initialization:

1. **Global flags**: Sets `_cancel`, `_pauseRequested`, `_liveParams`, and `_resume` references.
2. **Branch identification**: `identifyBranches` performs a DFS topological sort (`topoSort`) and groups nodes under their nearest upstream Wait node using `nearestUpstreamWaits`.
3. **Loop detection**: Constructs `LoopInfo` structures for `while` and `forEach` containers, tracking body node IDs and iteration counters.
4. **Context building**: Creates a `RunContext` object containing the workflow, extensions, Axios client, workspace path, and node output caches.

```ts
const { preExecExtNodes, branches, waitIds, parentWait, ordered } = identifyBranches(workflow);
const ctx: RunContext = { workflow, allExtensions, client, workspaceDir, /* ... */ };

```

### The Execution Loop

The runner processes **pre-execution nodes** (those not owned by a branch) in topological order. For each node, it highlights the active node via `activeNodeId` in the UI and dispatches execution via `executeExtensionNode`, which handles three distinct cases:

- **Iterators** (`forEachNode`): Reads the next file path via `listIteratorFiles` and injects it into `nodeOutputs`.
- **Model nodes**: Builds multipart/form-data requests, streams status from the generation API, and updates `blockProgress`.
- **Process extensions**: Invokes `window.electron.extensions.runProcess` for external binary execution.

After each node, `handleLoopEnd` checks loop conditions to determine whether to pause, repeat, or continue:

```ts
await executeExtensionNode(node, ctx, setRunState);
const jump = await handleLoopEnd(i);
if (jump !== undefined) i = jump - 1;

```

## Branch Handling and Wait Nodes

**Wait nodes** (`waitNode`) serve as interactive branch starters. When execution reaches a Wait node, the runner pauses and updates `useWorkflowRunStore.waitStates`, triggering the UI to display a **Continue** button managed by [`useWaitButton.ts`](https://github.com/lightningpixel/modly/blob/main/useWaitButton.ts).

The `continueRun(waitId)` method executes the selected branch:

1. Marks the wait as `running` and clears downstream outputs (enabling clean retries).
2. Executes branch nodes sequentially.
3. On success, unblocks child waits via `parentWait` mappings and pushes generated meshes to the viewer using `pushBranchSceneMesh`.
4. On failure, marks all descendant nodes as `error` to prevent deadlocks.

> **Source:** [`src/areas/workflows/workflowRunStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/workflowRunStore.ts) – `continueRun` implementation (lines 140-210).

## Pausing, Resuming, and Cancelling

The engine supports granular execution control:

- **Manual loops** (`whileNode`): Use `_pauseRequested`, `_resume`, and `_retry` flags to implement step-through debugging.
- **For-Each boundaries**: `pauseWhile()` pauses at file boundaries during batch processing.
- **Cancellation**: The `cancel()` method sets `_cancel` to true, flushes resume states via `flushResume()`, aborts active generation requests, and resets the store to `idle`.

```ts
cancel() {
  _cancel.current = true;
  flushResume(); // Unblock any waiting While loops
  // Abort generation job...
  set({ runState: 'idle', /* ... */ });
}

```

## UI Integration and React Flow

The visual canvas renders via **React Flow**, with the runner's `activeNodeId` driving visual highlighting of the currently executing node. The [`useWaitButton.ts`](https://github.com/lightningpixel/modly/blob/main/useWaitButton.ts) hook reads `waitStates` to conditionally render the Continue button when paused at a Wait node, bridging the gap between the execution engine and user interaction.

## Practical Implementation Examples

### Creating a Workflow Programmatically

```ts
import { v4 as uuid } from 'uuid';
import type { Workflow, WFNode, WFEdge } from '@shared/types/electron.d';

// Input node
const inputNode: WFNode = {
  id: `input-${uuid()}`,
  type: 'imageNode',
  position: { x: 100, y: 50 },
  data: { inputType: 'image', enabled: true, params: {} },
};

// Model extension
const modelNode: WFNode = {
  id: uuid(),
  type: 'extensionNode',
  position: { x: 300, y: 50 },
  data: { extensionId: 'stable-diffusion/1', enabled: true, params: {} },
};

// Wait (branch point)
const waitNode: WFNode = {
  id: uuid(),
  type: 'waitNode',
  position: { x: 500, y: 50 },
  data: { enabled: true },
};

// Output to 3D viewer
const outputNode: WFNode = {
  id: uuid(),
  type: 'outputNode',
  position: { x: 700, y: 50 },
  data: { enabled: true },
};

// Connect the graph
const edges: WFEdge[] = [
  { id: `e-${inputNode.id}-${modelNode.id}`, source: inputNode.id, target: modelNode.id },
  { id: `e-${modelNode.id}-${waitNode.id}`, source: modelNode.id, target: waitNode.id },
  { id: `e-${waitNode.id}-${outputNode.id}`, source: waitNode.id, target: outputNode.id },
];

const myWorkflow: Workflow = {
  id: uuid(),
  name: 'Image to Model Pipeline',
  description: '',
  nodes: [inputNode, modelNode, waitNode, outputNode],
  edges,
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString(),
};

// Persist via Electron API
await window.electron.workflows.save(myWorkflow);

```

### Running a Workflow from the UI

```tsx
import { useWorkflowRunStore } from '@areas/workflows/workflowRunStore';
import { useExtensionsStore } from '@shared/stores/extensionsStore';

const runWorkflow = async () => {
  const { activeWorkflowId, workflows } = useWorkflowsStore.getState();
  const workflow = workflows.find(w => w.id === activeWorkflowId);
  const extensions = useExtensionsStore.getState().extensions;
  
  if (workflow) {
    await useWorkflowRunStore.getState().run(workflow, extensions);
  }
};

```

### Controlling Execution Flow

```ts
// Pause at the next While boundary
useWorkflowRunStore.getState().pauseWhile();

// Resume or retry current iteration
useWorkflowRunStore.getState().continueWhile();
// OR
useWorkflowRunStore.getState().retryWhile();

```

## Summary

- The **Modly workflow system** uses a DAG of `WFNode` and `WFEdge` objects to represent execution graphs, persisted via `useWorkflowsStore` in [`src/shared/stores/workflowsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/workflowsStore.ts).
- **Node behaviors** in [`src/areas/workflows/nodeBehaviors.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/nodeBehaviors.ts) classify how data flows through passthrough, branch starter, and output nodes.
- The **execution engine** in [`workflowRunStore.ts`](https://github.com/lightningpixel/modly/blob/main/workflowRunStore.ts) performs topological sorting via `identifyBranches`, then processes nodes while managing loops and branches.
- **Wait nodes** create user intervention points where `continueRun` executes selected branches and pushes results to the 3D viewer via `pushBranchSceneMesh`.
- **State management** tracks execution through Zustand stores, supporting cancellation, pausing via `pauseWhile`, and resuming of both manual loops and batch iterations.

## Frequently Asked Questions

### How does Modly handle cyclic dependencies in workflows?

The **Modly workflow system** enforces a directed acyclic graph (DAG) structure through its topological sort implementation in `identifyBranches`. While the data model itself doesn't prevent cycles at the type level, the execution engine assumes acyclicity and performs a DFS topological sort (`topoSort`) before running. If cycles existed, the topological sort would fail to produce a valid execution order, effectively preventing cyclic workflows from running.

### What happens when a node fails during execution?

When `executeExtensionNode` encounters an error, the runner catches the exception and marks the node and its descendants as `error` state. For branches under a Wait node, failure prevents unblocking child waits (`parentWait`), and the UI reflects the error state. Users can then retry the specific branch via the Wait button interface or cancel the entire run using the `cancel()` method, which clears caches and resets the state machine to `idle`.

### Can workflows be imported and exported?

Yes, the `useWorkflowsStore` in [`src/shared/stores/workflowsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/workflowsStore.ts) provides import and export functionality via the Electron IPC bridge (`window.electron.workflows.*`). When loading workflows, the store runs `migrateWorkflow(entry)` to handle legacy formats, deduplicates by `id`, and restores tab and folder state from `localStorage`. This allows users to share workflow JSON files between different Modly installations.

### What is the difference between `whileNode` and `forEachNode` in Modly?

Both implement looping constructs but differ in control flow. The **`whileNode`** creates a manual loop where the runner pauses after each iteration, allowing users to adjust parameters before clicking Continue or Retry via `continueWhile()` and `retryWhile()`. The **`forEachNode`** iterates over a file list automatically, processing each file sequentially via `listIteratorFiles`, though it can still be paused at file boundaries using `pauseWhile()`. While nodes use `_pauseRequested` flags for step-through debugging, while forEach nodes batch process inputs.