What Is the Difference Between Model Extensions and Process Extensions in Modly?

Model extensions run machine-learning models in-process via the model API, while process extensions spawn separate subprocesses through the Process Runner to execute arbitrary scripts.

Modly treats extensions as first-class plugins that extend the workflow engine's capabilities. Understanding the difference between model extensions and process extensions in Modly is essential for developers building custom nodes for the lightningpixel/modly repository. These two types share a common node description format but differ fundamentally in how they execute and where their logic resides.

Core Architectural Differences

The distinction begins with the type definitions in src/shared/types/electron.d.ts, where each extension type implements a distinct interface.

Model Extensions

Model extensions provide machine-learning capabilities such as diffusion models, text-to-image generators, or 3D mesh generators. They implement the ModelExtension interface with type: 'model' and expose their functionality through an array of ExtensionNode definitions. Unlike process extensions, they do not specify an entry script because the model logic runs inside the main Electron process via the model API (window.electron.model.*).

According to the source code at lines 29-45, the ModelExtension interface includes fields for id, name, version, trusted, builtin, localPath, and nodes, but critically lacks an entry property.

Process Extensions

Process extensions wrap executable processes such as Python scripts or binaries that perform custom operations like mesh optimization or data export. They implement the ProcessExtension interface with type: 'process' and require an entry string field that specifies the path to the script that Modly will launch. The Process Runner (electron/main/process-runner.ts) manages these as separate subprocesses, isolating them from the main application process.

As defined at lines 63-80 of electron.d.ts, the ProcessExtension interface inherits the same base fields as model extensions but adds the mandatory entry property pointing to the executable script.

Technical Implementation and Storage

Modly maintains separate stores for each extension type to ensure proper isolation and loading mechanisms.

Store Separation and Loading

The src/shared/stores/extensionsStore.ts file (lines 20-28) implements distinct collections for each type. When the backend returns the extension list via IPC, the store filters them by type:

// Load all extensions from the backend
const list = (await window.electron.extensions.list()) as AnyExtension[];

// Separate them by type
modelExtensions:   list.filter((e): e is ModelExtension   => e.type === 'model'),
processExtensions: list.filter((e): e is ProcessExtension => e.type === 'process')

This separation ensures that model extensions populate the modelExtensions store for the model API, while process extensions populate the processExtensions store for the process runner.

Manifest and Validation

Both extension types support common manifest fields including id, name, version, trusted, builtin, localPath, and optional corrupted or manifestError flags. The IPC handlers in electron/main/ipc-handlers.ts expose the channel that lists, installs, and validates these extensions, ensuring the entry path exists for process extensions before registration.

Runtime Execution Behavior

The execution model represents the most significant operational difference between these extension types.

In-Process Model Inference

When a workflow node references a model extension, Modly creates a model inference request that runs entirely within the main Electron process. This provides low-latency access to ML capabilities but requires the model code to be compatible with the main process environment. The node data specifies the extensionId matching a ModelExtension.id, along with input types and parameters:

// A node that invokes a Stable Diffusion model
{
  id: 'node-1',
  type: 'model',
  data: {
    extensionId: 'sd-v1-4',
    inputType: 'image',
    params: { steps: 30, cfg: 7.0 }
  }
}

Out-of-Process Script Execution

When a workflow node references a process extension, Modly spawns a separate subprocess via the Process Runner. This isolation prevents unstable scripts from crashing the main application and allows extensions to use different Python environments or system binaries. The node references the extension by ID, and Modly executes the script defined in the entry field:

// A node that runs the mesh-optimizer script
{
  id: 'node-2',
  type: 'process',
  data: {
    extensionId: 'mesh-optimizer',
    inputType: 'mesh',
    params: { targetFaceCount: 5000 }
  }
}

The src/areas/workflows/nodes/mesh-optimizer/processor.ts file (line 32) demonstrates this pattern by registering a process extension that handles mesh optimization workflows.

Key Source Files

Understanding these files clarifies how Modly distinguishes between in-process models and out-of-process scripts:

• src/shared/types/electron.d.ts — Defines the ModelExtension (lines 29-45) and ProcessExtension (lines 63-80) interfaces and the shared ExtensionNode structure.
• src/shared/stores/extensionsStore.ts — Implements the store interface (lines 20-28) that separates and manages both extension collections.
• electron/main/process-runner.ts — Spawns and manages subprocesses for ProcessExtension entries.
• electron/main/ipc-handlers.ts — Exposes IPC channels for extension listing and validation.
• src/areas/workflows/mockExtensions.ts — Converts raw extensions into the WorkflowExtension type used by the workflow engine (lines 34-61).

Summary

• Model extensions run machine-learning models in-process using the model API without requiring an entry script.
• Process extensions execute arbitrary scripts in isolated subprocesses managed by the Process Runner and require an entry path.
• Both types share the ExtensionNode description format for inputs, outputs, and parameters but use different stores and execution mechanisms.
• The type system in electron.d.ts enforces these distinctions through the type field and the presence or absence of the entry property.

Frequently Asked Questions

Can a single extension act as both a model and process extension?

No. The type system enforces mutual exclusivity through the type field, which must be either 'model' or 'process'. An extension definition cannot specify both types simultaneously, and the extensionsStore.ts logic strictly separates them into different collections based on this field.

How does Modly handle security for process extensions?

Process extensions include a trusted boolean field in their manifest that Modly evaluates before execution. The IPC handlers in electron/main/ipc-handlers.ts validate extensions during installation, and the Process Runner executes the entry script in a subprocess to isolate potential crashes or malicious code from the main Electron process.

What happens if a process extension script fails?

Since process extensions run in separate subprocesses via electron/main/process-runner.ts, failures in the subprocess do not crash the main Modly application. The Process Runner manages the lifecycle of these scripts, and the workflow engine can detect non-zero exit codes or timeouts through the IPC communication channel, marking the node as failed without affecting other workflow components.

Where does Modly validate extension manifests?

Manifest validation occurs in electron/main/ipc-handlers.ts when extensions are listed or installed. The validation checks for required fields including id, name, and version, and specifically verifies that process extensions include a valid entry path. Invalid manifests populate the optional manifestError field, and corrupted extensions are flagged with the corrupted boolean to prevent their use in workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}