# Extension Points in Modly: Model and Process Extension Architecture Explained

> Discover Modly's extension points: ModelExtension for AI integration and ProcessExtension for workflow nodes. Extend Modly functionality at runtime via the Extensions Store.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: architecture
- Published: 2026-08-19

---

**Modly exposes two distinct extension points—`ModelExtension` for AI model integration and `ProcessExtension` for workflow node routines—enabling third-party functionality through runtime discovery in the Extensions Store without modifying core source code.**

The `lightningpixel/modly` repository implements a plugin architecture centered on specific extension points that decouple custom functionality from the main application. Developers extend Modly by creating either model or process extensions, which the system discovers and manages through a centralized store and Electron IPC layer.

## The Two Extension Types

Modly defines **two primary extension interfaces** that determine how external code integrates with the platform.

### ModelExtension

A `ModelExtension` describes an AI model that Modly can load and execute, such as diffusion models, text-to-image generators, or 3D-generation models. This extension point handles model metadata, inference APIs, and asset bundling. The interface definition resides in [`src/shared/types/electron.d.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/types/electron.d.ts), where it specifies the contract that model providers must implement to expose their capabilities to the Modly runtime.

### ProcessExtension

A `ProcessExtension` describes a callable routine that executes within workflow nodes. Unlike models, these extensions run as separate worker processes capable of reading and writing files, performing post-processing, or chaining external tools. According to the source in [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts), processes are invoked via the `extensions:runProcess` IPC command, making them ideal for custom mesh optimizers, texture generators, or arbitrary scripts exposed as workflow nodes.

## Runtime Discovery and Management

Extensions are discovered and managed at runtime through the **Extensions Store**, eliminating the need for static linking or core code modifications.

### The Extensions Store

The central management layer lives in [`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts) as a Zustand store. This store:

- Tracks installed extensions via `window.electron.extensions.list()`.
- Maintains separate arrays for `modelExtensions` and `processExtensions`.
- Handles installation progress through the `InstallProgress` interface, supporting states: `downloading`, `extracting`, `validating`, `setting_up`, `done`, and `error`.
- Merges built-in extensions with user-installed extensions from the user data directory specified in `settings.extensionsDir`.

### Electron IPC Communication

The frontend communicates with the backend through a typed IPC layer:

- **Preload script**: [`electron/preload/electron-api.ts`](https://github.com/lightningpixel/modly/blob/main/electron/preload/electron-api.ts) exposes the API surface to the renderer process.
- **Main handlers**: [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts) implements the filesystem operations, Python bridge interactions, and HTTP calls.

The available IPC commands for extension management include:
- `extensions:list`
- `extensions:installFromGitHub`
- `extensions:installFromLocal`
- `extensions:uninstall`
- `extensions:reload`
- `extensions:runProcess`

## Extension Manifest and Lifecycle

Every extension must include a **manifest JSON file** that declares its metadata. The manifest specifies the `type` (either `model` or `process`), along with `id`, `name`, `description`, and `version` properties. The Extensions Store validates this manifest during the `validating` phase of the `InstallProgress` workflow.

## Practical Implementation Examples

### Listing Installed Extensions

Access the store to retrieve loaded extensions from the UI:

```typescript
import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function showExtensions() {
  const { loadExtensions, modelExtensions, processExtensions } = useExtensionsStore.getState()
  await loadExtensions()
  console.log('Models:', modelExtensions.map(e => e.name))
  console.log('Processes:', processExtensions.map(e => e.name))
}

```

*Source:* [`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts)

### Installing a Model Extension from GitHub

Install remote extensions programmatically:

```typescript
import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function installFromGitHub(url: string) {
  const { installFromGitHub, clearInstallState } = useExtensionsStore.getState()
  clearInstallState()
  const result = await installFromGitHub(url)
  if (result.success) {
    console.log('Extension installed!')
  } else {
    console.error('Failed:', result.error)
  }
}

```

*Source:* [`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts) – the `installFromGitHub` method forwards to `window.electron.extensions.installFromGitHub`.

### Running a Process Extension from Workflow Nodes

Invoke a process extension within workflow logic:

```typescript
// Inside a workflow node implementation
async function runMyProcess(input: { filePath?: string; text?: string }) {
  const { runProcess } = window.electron.extensions
  const extId = 'my-process-extension'   // must match the extension's id
  const params = { someOption: true }

  const result = await runProcess(extId, input, params)
  if (result.success) {
    console.log('Process output:', result.result)
  } else {
    console.error('Process error:', result.error)
  }
}

```

*Source:* [`electron/preload/electron-api.ts`](https://github.com/lightningpixel/modly/blob/main/electron/preload/electron-api.ts) (API) and [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts) (handler).

### Uninstalling Extensions

Remove extensions through the store:

```typescript
import { useExtensionsStore } from '@shared/stores/extensionsStore'

async function removeExtension(extId: string) {
  const { uninstall } = useExtensionsStore.getState()
  const result = await uninstall(extId)
  if (result.success) {
    console.log(`Extension ${extId} removed.`)
  } else {
    console.error('Uninstall failed:', result.error)
  }
}

```

*Source:* [`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts) – calls `window.electron.extensions.uninstall`.

## Summary

- **Modly provides two extension points**: `ModelExtension` for AI model integration and `ProcessExtension` for workflow routines.
- **Runtime discovery** occurs through the Extensions Store ([`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts)), which merges built-in and user-installed extensions.
- **IPC architecture** in [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts) and [`electron/preload/electron-api.ts`](https://github.com/lightningpixel/modly/blob/main/electron/preload/electron-api.ts) enables secure communication between the UI and extension lifecycle methods.
- **Installation tracking** uses the `InstallProgress` interface to provide real-time feedback during download, extraction, and validation phases.
- **Example usage** of extension interfaces appears in [`src/areas/workflows/mockExtensions.ts`](https://github.com/lightningpixel/modly/blob/main/src/areas/workflows/mockExtensions.ts), demonstrating practical integration patterns.

## Frequently Asked Questions

### What is the difference between ModelExtension and ProcessExtension in Modly?

**`ModelExtension`** defines AI models that Modly loads and runs directly, handling inference logic and model assets, while **`ProcessExtension`** defines callable routines that execute in separate worker processes for file manipulation, post-processing, or external tool integration. The former integrates with Modly's model inference pipeline, whereas the latter connects to workflow nodes via `extensions:runProcess`.

### How does Modly discover extensions at runtime?

The **Extensions Store** ([`src/shared/stores/extensionsStore.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/stores/extensionsStore.ts)) discovers extensions by scanning the user data directory configured in `settings.extensionsDir` and merging results with built-in extensions. It exposes this state through `loadExtensions()` and separates models from processes into distinct reactive arrays for UI consumption.

### Can I install Modly extensions from sources other than GitHub?

Yes. The `window.electron.extensions` API and corresponding store methods support both **GitHub repositories** via `installFromGitHub` and **local folders** via `installFromLocal`, as implemented in [`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts). Both pathways validate the extension manifest and report progress through the same `InstallProgress` interface.

### Where are extension IPC handlers implemented in the Modly codebase?

The IPC handlers live in **[`electron/main/ipc-handlers.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/ipc-handlers.ts)**, which contains the backend logic for `extensions:list`, `extensions:installFromGitHub`, `extensions:runProcess`, and other commands. The frontend types and API surface are defined in **[`electron/preload/electron-api.ts`](https://github.com/lightningpixel/modly/blob/main/electron/preload/electron-api.ts)** and **[`src/shared/types/electron.d.ts`](https://github.com/lightningpixel/modly/blob/main/src/shared/types/electron.d.ts)**, respectively.