Extension Points in Modly: Model and Process Extension Architecture Explained

Modly exposes two distinct extension points—ModelExtension for AI model integration and ProcessExtension for workflow node routines—enabling third-party functionality through runtime discovery in the Extensions Store without modifying core source code.

The lightningpixel/modly repository implements a plugin architecture centered on specific extension points that decouple custom functionality from the main application. Developers extend Modly by creating either model or process extensions, which the system discovers and manages through a centralized store and Electron IPC layer.

The Two Extension Types

Modly defines two primary extension interfaces that determine how external code integrates with the platform.

ModelExtension

A ModelExtension describes an AI model that Modly can load and execute, such as diffusion models, text-to-image generators, or 3D-generation models. This extension point handles model metadata, inference APIs, and asset bundling. The interface definition resides in src/shared/types/electron.d.ts, where it specifies the contract that model providers must implement to expose their capabilities to the Modly runtime.

ProcessExtension

A ProcessExtension describes a callable routine that executes within workflow nodes. Unlike models, these extensions run as separate worker processes capable of reading and writing files, performing post-processing, or chaining external tools. According to the source in electron/main/ipc-handlers.ts, processes are invoked via the extensions:runProcess IPC command, making them ideal for custom mesh optimizers, texture generators, or arbitrary scripts exposed as workflow nodes.

Runtime Discovery and Management

Extensions are discovered and managed at runtime through the Extensions Store, eliminating the need for static linking or core code modifications.

The Extensions Store

The central management layer lives in src/shared/stores/extensionsStore.ts as a Zustand store. This store:

• Tracks installed extensions via window.electron.extensions.list().
• Maintains separate arrays for modelExtensions and processExtensions.
• Handles installation progress through the InstallProgress interface, supporting states: downloading, extracting, validating, setting_up, done, and error.
• Merges built-in extensions with user-installed extensions from the user data directory specified in settings.extensionsDir.

Electron IPC Communication

The frontend communicates with the backend through a typed IPC layer:

• Preload script: electron/preload/electron-api.ts exposes the API surface to the renderer process.
• Main handlers: electron/main/ipc-handlers.ts implements the filesystem operations, Python bridge interactions, and HTTP calls.

The available IPC commands for extension management include:

• extensions:list
• extensions:installFromGitHub
• extensions:installFromLocal
• extensions:uninstall
• extensions:reload
• extensions:runProcess

Extension Manifest and Lifecycle

Every extension must include a manifest JSON file that declares its metadata. The manifest specifies the type (either model or process), along with id, name, description, and version properties. The Extensions Store validates this manifest during the validating phase of the InstallProgress workflow.

Practical Implementation Examples

Listing Installed Extensions

Access the store to retrieve loaded extensions from the UI:

import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function showExtensions() {
  const { loadExtensions, modelExtensions, processExtensions } = useExtensionsStore.getState()
  await loadExtensions()
  console.log('Models:', modelExtensions.map(e => e.name))
  console.log('Processes:', processExtensions.map(e => e.name))
}

Source: src/shared/stores/extensionsStore.ts

Installing a Model Extension from GitHub

Install remote extensions programmatically:

import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function installFromGitHub(url: string) {
  const { installFromGitHub, clearInstallState } = useExtensionsStore.getState()
  clearInstallState()
  const result = await installFromGitHub(url)
  if (result.success) {
    console.log('Extension installed!')
  } else {
    console.error('Failed:', result.error)
  }
}

Source: src/shared/stores/extensionsStore.ts – the installFromGitHub method forwards to window.electron.extensions.installFromGitHub.

Running a Process Extension from Workflow Nodes

Invoke a process extension within workflow logic:

// Inside a workflow node implementation
async function runMyProcess(input: { filePath?: string; text?: string }) {
  const { runProcess } = window.electron.extensions
  const extId = 'my-process-extension'   // must match the extension's id
  const params = { someOption: true }

  const result = await runProcess(extId, input, params)
  if (result.success) {
    console.log('Process output:', result.result)
  } else {
    console.error('Process error:', result.error)
  }
}

Source: electron/preload/electron-api.ts (API) and electron/main/ipc-handlers.ts (handler).

Uninstalling Extensions

Remove extensions through the store:

import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function removeExtension(extId: string) {
  const { uninstall } = useExtensionsStore.getState()
  const result = await uninstall(extId)
  if (result.success) {
    console.log(`Extension ${extId} removed.`)
  } else {
    console.error('Uninstall failed:', result.error)
  }
}

Source: src/shared/stores/extensionsStore.ts – calls window.electron.extensions.uninstall.

Summary

• Modly provides two extension points: ModelExtension for AI model integration and ProcessExtension for workflow routines.
• Runtime discovery occurs through the Extensions Store (src/shared/stores/extensionsStore.ts), which merges built-in and user-installed extensions.
• IPC architecture in electron/main/ipc-handlers.ts and electron/preload/electron-api.ts enables secure communication between the UI and extension lifecycle methods.
• Installation tracking uses the InstallProgress interface to provide real-time feedback during download, extraction, and validation phases.
• Example usage of extension interfaces appears in src/areas/workflows/mockExtensions.ts, demonstrating practical integration patterns.

Frequently Asked Questions

What is the difference between ModelExtension and ProcessExtension in Modly?

ModelExtension defines AI models that Modly loads and runs directly, handling inference logic and model assets, while ProcessExtension defines callable routines that execute in separate worker processes for file manipulation, post-processing, or external tool integration. The former integrates with Modly's model inference pipeline, whereas the latter connects to workflow nodes via extensions:runProcess.

How does Modly discover extensions at runtime?

The Extensions Store (src/shared/stores/extensionsStore.ts) discovers extensions by scanning the user data directory configured in settings.extensionsDir and merging results with built-in extensions. It exposes this state through loadExtensions() and separates models from processes into distinct reactive arrays for UI consumption.

Can I install Modly extensions from sources other than GitHub?

Yes. The window.electron.extensions API and corresponding store methods support both GitHub repositories via installFromGitHub and local folders via installFromLocal, as implemented in electron/main/ipc-handlers.ts. Both pathways validate the extension manifest and report progress through the same InstallProgress interface.

Where are extension IPC handlers implemented in the Modly codebase?

The IPC handlers live in electron/main/ipc-handlers.ts, which contains the backend logic for extensions:list, extensions:installFromGitHub, extensions:runProcess, and other commands. The frontend types and API surface are defined in electron/preload/electron-api.ts and src/shared/types/electron.d.ts, respectively.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}