How the Modly Workflow System Works: Node-Based Execution in lightningpixel/modly

The Modly workflow system implements a deterministic, node-based execution engine that orchestrates AI pipelines using a directed acyclic graph (DAG) of WFNode and WFEdge objects, coordinated by Zustand stores and a topological sorter that handles branching, looping, and interactive pausing via Wait nodes.

The Modly workflow system, found in the lightningpixel/modly repository, provides a visual programming environment for chaining AI model calls, control flow logic, and 3D output nodes. Built on a DAG structure defined in electron.d.ts and managed through dedicated Zustand stores, the engine supports everything from linear processing pipelines to complex branching workflows requiring user intervention at specific points.

Core Architecture: DAG and State Management

At its foundation, every workflow is a directed acyclic graph composed of WFNode and WFEdge objects. The architecture separates concerns between two primary Zustand stores:

• useWorkflowsStore (src/shared/stores/workflowsStore.ts): Handles workflow persistence, folder organization, and UI tab state.
• useWorkflowRunStore (src/areas/workflows/workflowRunStore.ts): Controls the execution state machine (idle, running, paused, done, error) and drives the runner loop.

State changes flow through the graph according to behaviors defined in src/areas/workflows/nodeBehaviors.ts, which classifies nodes as passthrough, branch starters, scene outputs, or branch consumers.

Workflow Definition and Persistence

Workflows persist through the Electron IPC bridge exposed in workflowsStore.ts. On initialization, the store deduplicates workflows by id, restores open tabs, and loads folder metadata from localStorage under the FOLDERS_KEY.

The store exposes CRUD operations via window.electron.workflows.* methods:

// Loading with legacy migration support
await window.electron.workflows.list();
const wf = migrateWorkflow(entry); // Handles legacy format conversion
set({ workflows: list, openIds, activeId, folders });

Folder management functions (addFolder, removeFolder, setFolderColor, toggleFolderBookmark) allow users to organize workflows in the UI sidebar, while static validation occurs in src/areas/workflows/preflight.ts before execution begins.

Node Behavior Registry

The execution engine relies on a behavior registry in src/areas/workflows/nodeBehaviors.ts to determine how data flows through each node type:

Behavior	Function	Example Nodes
passthrough	Data flows unchanged	waitNode
branchStarter	Splits execution into user-driven sub-DAGs	waitNode (interactive pause points)
sceneOutput	Final sink pushing meshes to 3D viewer	outputNode
branchConsumer	Consumes single mesh from branch	Extension nodes

Helper functions like isPassthrough, isBranchStarter, resolveDataSource, and nearestUpstreamWaits enable the runner to analyze graph structure and determine execution paths without executing code.

Execution Engine and Run Preparation

When useWorkflowRunStore.run(workflow, extensions) is invoked, the engine performs preflight initialization:

1. Global flags: Sets _cancel, _pauseRequested, _liveParams, and _resume references.
2. Branch identification: identifyBranches performs a DFS topological sort (topoSort) and groups nodes under their nearest upstream Wait node using nearestUpstreamWaits.
3. Loop detection: Constructs LoopInfo structures for while and forEach containers, tracking body node IDs and iteration counters.
4. Context building: Creates a RunContext object containing the workflow, extensions, Axios client, workspace path, and node output caches.

const { preExecExtNodes, branches, waitIds, parentWait, ordered } = identifyBranches(workflow);
const ctx: RunContext = { workflow, allExtensions, client, workspaceDir, /* ... */ };

The Execution Loop

The runner processes pre-execution nodes (those not owned by a branch) in topological order. For each node, it highlights the active node via activeNodeId in the UI and dispatches execution via executeExtensionNode, which handles three distinct cases:

• Iterators (forEachNode): Reads the next file path via listIteratorFiles and injects it into nodeOutputs.
• Model nodes: Builds multipart/form-data requests, streams status from the generation API, and updates blockProgress.
• Process extensions: Invokes window.electron.extensions.runProcess for external binary execution.

After each node, handleLoopEnd checks loop conditions to determine whether to pause, repeat, or continue:

await executeExtensionNode(node, ctx, setRunState);
const jump = await handleLoopEnd(i);
if (jump !== undefined) i = jump - 1;

Branch Handling and Wait Nodes

Wait nodes (waitNode) serve as interactive branch starters. When execution reaches a Wait node, the runner pauses and updates useWorkflowRunStore.waitStates, triggering the UI to display a Continue button managed by useWaitButton.ts.

The continueRun(waitId) method executes the selected branch:

1. Marks the wait as running and clears downstream outputs (enabling clean retries).
2. Executes branch nodes sequentially.
3. On success, unblocks child waits via parentWait mappings and pushes generated meshes to the viewer using pushBranchSceneMesh.
4. On failure, marks all descendant nodes as error to prevent deadlocks.

Source: src/areas/workflows/workflowRunStore.ts – continueRun implementation (lines 140-210).

Pausing, Resuming, and Cancelling

The engine supports granular execution control:

• Manual loops (whileNode): Use _pauseRequested, _resume, and _retry flags to implement step-through debugging.
• For-Each boundaries: pauseWhile() pauses at file boundaries during batch processing.
• Cancellation: The cancel() method sets _cancel to true, flushes resume states via flushResume(), aborts active generation requests, and resets the store to idle.

cancel() {
  _cancel.current = true;
  flushResume(); // Unblock any waiting While loops
  // Abort generation job...
  set({ runState: 'idle', /* ... */ });
}

UI Integration and React Flow

The visual canvas renders via React Flow, with the runner's activeNodeId driving visual highlighting of the currently executing node. The useWaitButton.ts hook reads waitStates to conditionally render the Continue button when paused at a Wait node, bridging the gap between the execution engine and user interaction.

Practical Implementation Examples

Creating a Workflow Programmatically

import { v4 as uuid } from 'uuid';
import type { Workflow, WFNode, WFEdge } from '@shared/types/electron.d';

// Input node
const inputNode: WFNode = {
  id: `input-${uuid()}`,
  type: 'imageNode',
  position: { x: 100, y: 50 },
  data: { inputType: 'image', enabled: true, params: {} },
};

// Model extension
const modelNode: WFNode = {
  id: uuid(),
  type: 'extensionNode',
  position: { x: 300, y: 50 },
  data: { extensionId: 'stable-diffusion/1', enabled: true, params: {} },
};

// Wait (branch point)
const waitNode: WFNode = {
  id: uuid(),
  type: 'waitNode',
  position: { x: 500, y: 50 },
  data: { enabled: true },
};

// Output to 3D viewer
const outputNode: WFNode = {
  id: uuid(),
  type: 'outputNode',
  position: { x: 700, y: 50 },
  data: { enabled: true },
};

// Connect the graph
const edges: WFEdge[] = [
  { id: `e-${inputNode.id}-${modelNode.id}`, source: inputNode.id, target: modelNode.id },
  { id: `e-${modelNode.id}-${waitNode.id}`, source: modelNode.id, target: waitNode.id },
  { id: `e-${waitNode.id}-${outputNode.id}`, source: waitNode.id, target: outputNode.id },
];

const myWorkflow: Workflow = {
  id: uuid(),
  name: 'Image to Model Pipeline',
  description: '',
  nodes: [inputNode, modelNode, waitNode, outputNode],
  edges,
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString(),
};

// Persist via Electron API
await window.electron.workflows.save(myWorkflow);

Running a Workflow from the UI

import { useWorkflowRunStore } from '@areas/workflows/workflowRunStore';
import { useExtensionsStore } from '@shared/stores/extensionsStore';

const runWorkflow = async () => {
  const { activeWorkflowId, workflows } = useWorkflowsStore.getState();
  const workflow = workflows.find(w => w.id === activeWorkflowId);
  const extensions = useExtensionsStore.getState().extensions;
  
  if (workflow) {
    await useWorkflowRunStore.getState().run(workflow, extensions);
  }
};

Controlling Execution Flow

// Pause at the next While boundary
useWorkflowRunStore.getState().pauseWhile();

// Resume or retry current iteration
useWorkflowRunStore.getState().continueWhile();
// OR
useWorkflowRunStore.getState().retryWhile();

Summary

• The Modly workflow system uses a DAG of WFNode and WFEdge objects to represent execution graphs, persisted via useWorkflowsStore in src/shared/stores/workflowsStore.ts.
• Node behaviors in src/areas/workflows/nodeBehaviors.ts classify how data flows through passthrough, branch starter, and output nodes.
• The execution engine in workflowRunStore.ts performs topological sorting via identifyBranches, then processes nodes while managing loops and branches.
• Wait nodes create user intervention points where continueRun executes selected branches and pushes results to the 3D viewer via pushBranchSceneMesh.
• State management tracks execution through Zustand stores, supporting cancellation, pausing via pauseWhile, and resuming of both manual loops and batch iterations.

Frequently Asked Questions

How does Modly handle cyclic dependencies in workflows?

The Modly workflow system enforces a directed acyclic graph (DAG) structure through its topological sort implementation in identifyBranches. While the data model itself doesn't prevent cycles at the type level, the execution engine assumes acyclicity and performs a DFS topological sort (topoSort) before running. If cycles existed, the topological sort would fail to produce a valid execution order, effectively preventing cyclic workflows from running.

What happens when a node fails during execution?

When executeExtensionNode encounters an error, the runner catches the exception and marks the node and its descendants as error state. For branches under a Wait node, failure prevents unblocking child waits (parentWait), and the UI reflects the error state. Users can then retry the specific branch via the Wait button interface or cancel the entire run using the cancel() method, which clears caches and resets the state machine to idle.

Can workflows be imported and exported?

Yes, the useWorkflowsStore in src/shared/stores/workflowsStore.ts provides import and export functionality via the Electron IPC bridge (window.electron.workflows.*). When loading workflows, the store runs migrateWorkflow(entry) to handle legacy formats, deduplicates by id, and restores tab and folder state from localStorage. This allows users to share workflow JSON files between different Modly installations.

What is the difference between whileNode and forEachNode in Modly?

Both implement looping constructs but differ in control flow. The whileNode creates a manual loop where the runner pauses after each iteration, allowing users to adjust parameters before clicking Continue or Retry via continueWhile() and retryWhile(). The forEachNode iterates over a file list automatically, processing each file sequentially via listIteratorFiles, though it can still be paused at file boundaries using pauseWhile(). While nodes use _pauseRequested flags for step-through debugging, while forEach nodes batch process inputs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}