# How Process Extensions Communicate Progress and Logs to the Modly Main App

> Discover how Modly process extensions send logs and progress updates to the main app using context object functions and IPC or stdout JSON.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: internals
- Published: 2026-08-15

---

**Modly process extensions communicate back to the main app through a context object exposing `log()` and `progress()` functions, which serialize messages to the `ProcessRunner` via worker-thread IPC or stdout JSON lines.**

Modly's extension system isolates user code for security and stability. JavaScript extensions run in **Node.js worker threads**, Python extensions in **subprocesses**. Both use the same message schema to stream progress and logs to the main Electron process. This article examines the implementation in [`electron/main/process-runner.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/process-runner.ts) and shows how extension authors and main-process developers coordinate this communication.

## The Extension Context: log() and progress()

Every process extension receives a **context object** containing two callable functions. Extension authors use these to emit status updates without worrying about underlying transport mechanisms.

| Function | Signature | Emitted message |
|----------|-----------|---------------|
| `log(message)` | `log(string)` | `{ type: 'log', message: <string> }` |
| `progress(percent, label)` | `progress(number, string)` | `{ type: 'progress', percent: <number>, label: <string> }` |

These functions abstract away whether the extension runs in a worker thread or subprocess. The main app receives identically-structured messages regardless of runtime.

## JavaScript Extensions: Worker Thread IPC

### Context Creation and Message Posting

JavaScript extensions execute inside a dedicated worker thread. The worker constructs the context object at lines 30-38 of [`electron/main/process-runner.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/process-runner.ts):

```typescript
// Inside the worker thread (extracted from process-runner.ts)
const context = {
  log: (msg: string) => {
    parentPort.postMessage({ type: 'log', message: msg });
  },
  progress: (pct: number, label: string) => {
    parentPort.postMessage({ type: 'progress', percent: pct, label });
  }
};

```

Calling `parentPort.postMessage` serializes the message and transfers it to the parent thread through Node.js's worker_threads MessageChannel.

### Parent Thread Message Handling

The `ProcessRunner.run` method (lines 31-44 in the same file) attaches an `onmessage` handler to receive these updates:

```typescript
// ProcessRunner.run implementation pattern
worker.on('message', (msg) => {
  if (msg.type === 'log') {
    onLog?.(msg.message);
  } else if (msg.type === 'progress') {
    onProgress?.(msg.percent, msg.label);
  }
});

```

The optional callbacks `onLog` and `onProgress` are supplied by the caller—typically a store or UI controller.

## Python Extensions: Stdout JSON Lines

Python extensions follow a **line-delimited JSON protocol** written to stdout. The same message schema applies, but transport occurs through process I/O rather than IPC.

### Emitting Messages from Python

A Python extension prints JSON objects to stdout, flushing after each line:

```python
import json
import sys

def run():
    for i in range(0, 101, 25):
        # Progress update

        print(json.dumps({
            "type": "progress",
            "percent": i,
            "label": f"Stage {i//25 + 1}"
        }))
        sys.stdout.flush()
    
    # Log message

    print(json.dumps({
        "type": "log",
        "message": "Processing complete"
    }))
    sys.stdout.flush()
    
    # Completion signal with result

    print(json.dumps({
        "type": "done",
        "result": {"text": "Done"}
    }))
    sys.stdout.flush()

```

The `type: 'done'` message is Python-specific—it signals completion and carries the return value, since Python lacks the equivalent of a worker's `return` statement.

### Parsing in PythonProcessRunner

Around lines 98-108 of [`electron/main/process-runner.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/process-runner.ts), the `PythonProcessRunner` reads stdout line-by-line:

```typescript
// Simplified from the stdout handler
pythonProcess.stdout.on('data', (data: Buffer) => {
  const lines = data.toString().split('\n').filter(Boolean);
  for (const line of lines) {
    try {
      const msg = JSON.parse(line);
      if (msg.type === 'log') onLog?.(msg.message);
      if (msg.type === 'progress') onProgress?.(msg.percent, msg.label);
      if (msg.type === 'done') resolve(msg.result);
    } catch (err) {
      // Non-JSON lines treated as raw logs
      onLog?.(line);
    }
  }
});

```

**Critical detail**: The parser falls back to treating unparsable lines as raw log output, allowing `print()` debugging without breaking the protocol.

## Complete JavaScript Extension Example

```typescript
// extensions/image-resize/processor.js
const sharp = require('sharp');

module.exports = async (input, params, { log, progress }) => {
  log(`Loading image from ${input.path}`);
  
  const steps = ['read', 'resize', 'sharpen', 'encode', 'write'];
  const pipeline = sharp(input.path)
    .resize(params.width, params.height)
    .sharpen();
  
  for (let i = 0; i < steps.length; i++) {
    progress(
      Math.round((i / steps.length) * 100),
      `Step ${i + 1}/${steps.length}: ${steps[i]}`
    );
    // Actual processing happens here...
    await pipeline.toFile(`${input.path}.resized.jpg`);
  }
  
  progress(100, 'Complete');
  log('Image processing finished successfully');
  
  return { outputPath: `${input.path}.resized.jpg` };
};

```

The `module.exports` function receives three arguments: `input` (the node input), `params` (user-configured parameters), and the context object destructured to `{ log, progress }`.

## Registering Callbacks from the Main Process

The `ProcessRunner` class accepts callback functions that bridge to UI state. Here's a complete main-process integration:

```typescript
import { ProcessRunner } from '@/electron/main/process-runner';
import { useAppStore } from '@/shared/stores/appStore';
import { useWorkflowRunStore } from '@/areas/workflows/workflowRunStore';

async function executeExtension(extDir: string, nodeConfig: any) {
  const runner = new ProcessRunner(
    extDir,
    'processor.js',           // or 'processor.py'
    workspaceDir,
    tempDir
  );

  const appStore = useAppStore.getState();
  
  try {
    const result = await runner.run(
      nodeConfig.input,
      nodeConfig.params,
      // onProgress: updates both local node state and global app progress
      (percent, label) => {
        useWorkflowRunStore.getState().updateNodeProgress(
          nodeConfig.nodeId,
          percent,
          label
        );
        appStore.updateCurrentJob({ progress: percent, step: label });
      },
      // onLog: streams to developer console and UI log panel
      (message) => {
        console.log(`[${nodeConfig.nodeId}]`, message);
        appStore.appendToJobLog(message);
      }
    );
    
    return result;
  } finally {
    runner.cleanup();
  }
}

```

**Key integration points**:
- `workflowRunStore` tracks per-node progress within a multi-node workflow
- `appStore` maintains the global "current job" state for UI display
- Both stores update reactive UI components through Zustand subscriptions

## Message Flow Architecture

```

┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  JS Extension   │     │   Python Ext.    │     │   Main Process  │
│  (worker thread)│     │   (subprocess)   │     │  ProcessRunner  │
└────────┬────────┘     └────────┬─────────┘     └────────┬────────┘
         │                       │                        │
    log("msg")              print(json.dumps({        on('message')
    progress(50, "half")        "type": "log"...   ───►│  stdout.on('data')
         │                       │                        │
         ▼                       ▼                        ▼
   parentPort.             process.stdout           parse & route to
   postMessage()           (line-delimited)         onLog / onProgress
         │                       │                        │
         └───────────────────────┴────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │  AppStore /     │
                    │  WorkflowRunStore│
                    │  (UI state)     │
                    └─────────────────┘

```

This unified design lets Modly treat both runtimes identically at the UI layer while respecting their underlying constraints.

## Error Handling and Edge Cases

### Buffering and Partial Lines

The Python stdout handler accumulates partial lines when `data` events contain incomplete JSON. The implementation in [`process-runner.ts`](https://github.com/lightningpixel/modly/blob/main/process-runner.ts) uses an internal buffer to handle split UTF-8 sequences and truncated JSON objects.

### Malformed Messages

As noted above, JSON parse failures fall back to raw log output. This prevents a single malformed line from crashing the extension runner while preserving visibility into what went wrong.

### Worker Termination

If a JavaScript extension throws or calls `process.exit`, the worker's `'error'` and `'exit'` events trigger cleanup. The `ProcessRunner` wrapper rejects the run promise with diagnostic information.

## Summary

- **Two runtimes, one protocol**: JavaScript worker threads and Python subprocesses both use `log()` and `progress()` context functions with identical message schemas
- **Transport differs**: JS uses `parentPort.postMessage()`; Python uses `stdout` JSON lines
- **Central dispatch**: [`electron/main/process-runner.ts`](https://github.com/lightningpixel/modly/blob/main/electron/main/process-runner.ts) contains all message routing logic for both runtimes
- **Callback-based UI updates**: The `run()` method accepts `onLog` and `onProgress` callbacks that feed into `AppStore` and `WorkflowRunStore`
- **Graceful degradation**: Python extensions can use plain `print()`; the parser treats non-JSON lines as logs

## Frequently Asked Questions

### What happens if a Python extension forgets to flush stdout?

The `PythonProcessRunner` may receive incomplete or delayed messages. Always call `sys.stdout.flush()` after each `print()` to ensure timely delivery. The example code in the Modly documentation includes this pattern as a requirement.

### Can extensions send custom message types beyond log and progress?

The base `ProcessRunner` only recognizes `log`, `progress`, and `done` (Python). For custom protocols, extensions can overload the `input` object with request-response patterns or write to files in the `tempDir` passed to the constructor.

### How does Modly handle concurrent extension runs?

Each `ProcessRunner` instance creates an isolated worker or subprocess. Multiple runners can operate simultaneously, with their callbacks routing to different node IDs in `WorkflowRunStore`. No global locks coordinate between extensions—state management is the caller's responsibility.

### Is there a size limit for log messages?

Worker threads serialize through the structured clone algorithm, which handles strings up to memory limits. Python's stdout pipe has typical shell buffer constraints (~64KB on many systems). For large payloads, extensions should write to the filesystem and emit a path reference.