How the Workflow Node System Works in Modly: A Deep Dive

Modly's workflow node system uses a behavior-driven graph architecture where nodes declare execution characteristics (passthrough, branch starter, scene output) and the engine resolves dependencies, handles asynchronous waits, and completes when all scene-output nodes receive data.

The workflow node system is the core execution engine powering Modly's modular 3D asset pipelines. Built in TypeScript and located in src/areas/workflows/, it enables complex, branching processing graphs where each node represents a discrete step—from mesh optimization to final export. This article explains the internal mechanics based on the actual source code in the lightningpixel/modly repository.

Core Architecture: Node Behaviors

The foundation of the workflow node system lies in node behaviors, defined in src/areas/workflows/nodeBehaviors.ts. Rather than hard-coding execution logic per node type, Modly assigns abstract behaviors that the engine interprets at runtime.

Key Behavior Types

• Passthrough – Nodes marked with this behavior forward upstream data unchanged. The engine treats them as transparent during dependency resolution, allowing downstream nodes to execute immediately.
• Branch Starter / Consumer – These enable conditional and parallel execution paths. A branch starter (isBranchStarter) creates a new logical execution branch, while a branch consumer (isBranchConsumer) merges or processes branch results.
• Scene Output – Nodes flagged as isSceneOutput represent workflow endpoints. The engine terminates only when all scene-output nodes have received their required data.

Critical Utility Functions

The same file exports several graph-walking utilities that drive execution:

Function	Purpose
resolveDataSource(node, context)	Determines where a node's inputs originate—upstream node output, external asset reference, or default value
nearestUpstreamWaits(node, context)	Traverses upward to find the closest preceding wait node, ensuring synchronization before async operations continue

State Management: Workflow Run Store

All runtime state lives in src/areas/workflows/workflowRunStore.ts. This central store maintains:

• The node execution queue—which nodes are currently runnable
• Data bindings per node—resolved inputs mapped to their sources
• Branch tracking metadata—which logical branch each active node belongs to

When you initiate a workflow, startWorkflowRun(workflowId) populates this store and begins the execution cycle.

Pre-flight Validation

Before any nodes execute, the engine runs a pre-flight pass implemented in src/areas/workflows/preflight.ts. This phase:

1. Validates the graph structure for cycles and disconnected inputs
2. Verifies branch and loop constructs are well-formed
3. Uses the same behavior utilities (isPassthrough, resolveDataSource, nearestUpstreamWaits) to predict execution order
4. Surfaces errors early, before expensive processing begins

Execution Flow: Step by Step

The workflow node system processes graphs through six distinct phases:

1. Graph Construction

When a workflow loads, each node instantiates with its type, declared inputs, and assigned behavior. The behavior declaration determines how the engine will treat that node throughout execution.

2. Dependency Resolution

For every node, resolveDataSource wires inputs to upstream producers. This establishes the data-flow graph independent of execution order.

3. Queue Population

Nodes with no unsatisfied dependencies—or whose dependencies are passthrough—enter the runnable queue immediately. This eager scheduling maximizes parallelism.

4. Node Processing

The engine dequeues nodes, runs their processors, stores outputs, and notifies downstream dependencies. Processors are standard async functions; for example, src/areas/workflows/nodes/mesh-optimizer/processor.ts implements mesh decimation and LOD generation.

5. Branch and Wait Handling

When encountering a branch starter, the engine spawns a new logical branch with isolated state. For synchronization points, nearestUpstreamWaits locates the relevant wait node and pauses execution until completion signals arrive—critical for GPU-bound operations that outlive a single frame.

6. Completion Detection

The run terminates only when all isSceneOutput nodes have received data and all pending waits have resolved. This ensures no premature exit leaves background processing orphaned.

Practical Code Examples

Defining a Custom Passthrough Node

import { NodeDefinition, NodeBehavior } from '@/areas/workflows/nodeBehaviors';

export const MyPassthroughNode: NodeDefinition = {
  id: 'my-passthrough',
  type: 'passthrough',
  behavior: NodeBehavior.Passthrough,
  inputs: [{ name: 'source', type: 'mesh' }],
  processor: async ({ source }) => {
    // No transformation – just forward the incoming mesh
    return source;
  },
};

Initiating a Workflow Run

import { startWorkflowRun } from '@/areas/workflows/workflowRunStore';

const workflowId = 'example-workflow';
startWorkflowRun(workflowId).then((result) => {
  console.log('Workflow completed:', result);
});

Handling Upstream Waits in a Processor

import { nearestUpstreamWaits } from '@/areas/workflows/nodeBehaviors';

export async function meshOptimizerProcessor(node, ctx) {
  // Ensure any required GPU sync is finished
  await nearestUpstreamWaits(node, ctx);
  // Perform optimization…
}

Key Source Files

File	Role
src/areas/workflows/nodeBehaviors.ts	Behavior definitions and graph-walking utilities
src/areas/workflows/workflowRunStore.ts	Central runtime state and queue management
src/areas/workflows/preflight.ts	Graph validation and pre-execution analysis
src/areas/workflows/nodes/mesh-optimizer/processor.ts	Example processor with upstream wait handling
src/areas/workflows/nodes/mesh-exporter/processor.ts	Example scene-output processor

Summary

• Behavior-driven design separates node semantics from implementation, enabling extensibility without engine modifications
• resolveDataSource and nearestUpstreamWaits in nodeBehaviors.ts provide the graph traversal primitives for dependency resolution and synchronization
• The workflow run store centralizes all mutable state, making execution predictable and debuggable
• Pre-flight validation catches structural errors before expensive processing begins
• Scene-output completion guarantees that workflows finish only when all final outputs are ready

Frequently Asked Questions

What is a node behavior in Modly?

A node behavior is an abstract classification defined in nodeBehaviors.ts that tells the workflow engine how to handle a node during execution. Behaviors include Passthrough, Branch Starter, Branch Consumer, and Scene Output. Each behavior triggers specific engine logic—for example, Scene Output nodes signal workflow termination when all have received data.

How does Modly handle asynchronous operations in workflows?

Modly uses wait nodes and the nearestUpstreamWaits utility to manage asynchrony. When a processor calls await nearestUpstreamWaits(node, ctx), the engine pauses that execution path until the upstream wait node signals completion. This pattern appears in GPU-heavy processors like the mesh optimizer, where operations may span multiple frames.

Can I create custom node types without modifying the core engine?

Yes. New node types require only a processor function and an optional behavior declaration. The engine automatically integrates custom nodes through the same graph resolution and queuing mechanisms. Place your processor in src/areas/workflows/nodes/[your-node]/processor.ts and export a NodeDefinition with your chosen behavior.

Where does workflow execution state live during a run?

All mutable state resides in the workflow run store (workflowRunStore.ts). This includes the node queue, resolved data bindings per node, and branch membership metadata. The store's centralized design enables inspection, debugging, and potential future features like workflow pause/resume or distributed execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}