# How Modly Installs Extensions from GitHub and Handles Backup/Restore

> Learn how Modly installs extensions from GitHub by cloning and renaming repositories. Discover its efficient backup and restore process using dot-prefixed folders.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Modly installs extensions from GitHub by cloning repositories into a temporary dot-prefixed directory, marking them with a `.modly-incomplete` file during setup, and renaming them upon successful completion, while using dot-prefixed folders to create lightweight filesystem backups that can be restored by renaming.**

The **lightningpixel/modly** repository implements a robust extension management system that treats each model or processing extension as a self-contained GitHub repository. Understanding how extensions are installed from GitHub and the underlying backup/restore process is essential for developers managing custom AI model integrations.

## The GitHub Extension Installation Workflow

Modly manages extensions through the `EXTENSIONS_DIR` environment variable, which defaults to `~/.modly/extensions`. When a user initiates an installation from GitHub, the system follows a specific staging protocol to ensure atomic operations and prevent partial installations.

### Staging in Dot-Prefixed Directories

The installation process begins by cloning the remote repository into a **temporary directory** prefixed with a dot (`.`). For example, an extension named `my-extension` clones into `.my-extension` within `EXTENSIONS_DIR`. This convention serves two purposes: it hides the staging folder from the registry scanner, and it provides a namespace for backup operations.

According to the source code in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), the registry explicitly ignores any directory starting with a dot during the discovery phase:

```python

# api/services/generator_registry.py – _discover_extensions method

if ext_dir.name.startswith("."):
    continue  # Skip hidden/staging directories

```

### The .modly-incomplete Marker

While the extension is being cloned and configured, Modly creates a marker file named `.modly-incomplete` inside the temporary directory. This file acts as a semaphore to prevent the registry from loading an extension that has not finished its setup sequence.

The registry checks for this marker before attempting to load any extension:

```python
if (ext_dir / ".modly-incomplete").exists():
    print(f"[Registry] Skipping '{ext_dir.name}': install has not completed")
    continue

```

Only after the [`setup.py`](https://github.com/lightningpixel/modly/blob/main/setup.py) script executes successfully (if present) does the installer rename the directory to its final name (removing the dot prefix) and delete the `.modly-incomplete` marker.

### Setup and Virtual Environment Creation

Extensions requiring isolation run in **subprocess mode**, which necessitates creating an isolated virtual environment. TheFastAPI endpoint `/setup/{ext_id}` in [`api/routers/extensions.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/extensions.py) handles this:

```python

# api/routers/extensions.py

@router.post("/setup/{ext_id}")
async def setup_extension(ext_id: str):
    """
    Creates the isolated venv for an extension by running its setup.py.
    Called automatically after installing an extension from GitHub.
    """
    ext_dir = EXTENSIONS_DIR / ext_id
    setup_py = ext_dir / "setup.py"
    # Spawns subprocess using Modly’s embedded Python

```

Extensions without a [`setup.py`](https://github.com/lightningpixel/modly/blob/main/setup.py) are treated as legacy extensions and load directly via [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py).

## Registry Discovery and Validation

The **GeneratorRegistry** scans the extensions directory during startup to build the available model catalog. Located in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), the `_discover_extensions` method enforces strict validation rules:

1. Skip dot-prefixed directories (staging/backups)
2. Skip directories containing `.modly-incomplete`
3. Require [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) for valid extensions

Valid extensions populate the registry and become available through Modly's API. The registry distinguishes between **direct mode** (loading [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) directly) and **subprocess mode** (managing an isolated process via [`api/services/extension_process.py`](https://github.com/lightningpixel/modly/blob/main/api/services/extension_process.py)).

## The Backup and Restore Mechanism

Modly implements backup and restore through filesystem operations rather than database snapshots or API calls, leveraging the same dot-prefix convention used for staging.

### Creating Backups During Updates

When updating an existing extension, Modly first copies the current extension folder to a dot-prefixed backup directory (e.g., `.my-extension-backup`). The new version clones into the original directory name. If the new installation fails or produces errors, the system can revert by renaming the backup folder back to the active extension name.

This approach ensures that a working version remains available on disk until the new version completes successfully.

### Restoring Previous Versions

Restoration requires no specialized API endpoints. Users or automated scripts can restore a previous version by manipulating directory names:

```python
import os
import shutil

EXT_DIR = os.path.expanduser("~/.modly/extensions")
ext_id = "my-extension"

backup_dir = os.path.join(EXT_DIR, f".{ext_id}-backup")
original_dir = os.path.join(EXT_DIR, ext_id)

# Remove faulty installation

if os.path.isdir(original_dir):
    shutil.rmtree(original_dir)

# Restore backup

os.rename(backup_dir, original_dir)

```

After restoration, calling the reload endpoint refreshes the registry without restarting the application:

```python
import requests
requests.post("http://localhost:8000/extensions/reload")

```

## Implementation Example: Complete Installation Flow

The client-side installation logic, typically handled in tools like [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py), orchestrates the clone, setup, and activation sequence:

```python
import os
import subprocess
import requests

EXT_DIR = os.getenv("EXTENSIONS_DIR", os.path.expanduser("~/.modly/extensions"))
ext_id = "my-cool-extension"
git_url = "https://github.com/user/my-cool-extension"

# Step 1: Clone into temporary dot-folder

tmp_dir = os.path.join(EXT_DIR, f".{ext_id}")
subprocess.run(["git", "clone", git_url, tmp_dir], check=True)

# Step 2: Mark as incomplete

incomplete_marker = os.path.join(tmp_dir, ".modly-incomplete")
open(incomplete_marker, "w").close()

# Step 3: Run setup via API

requests.post(f"http://localhost:8000/extensions/setup/{ext_id}")

# Step 4: Activate by renaming and removing marker

final_dir = os.path.join(EXT_DIR, ext_id)
os.rename(tmp_dir, final_dir)
os.remove(os.path.join(final_dir, ".modly-incomplete"))

```

## Summary

- **Staging**: Extensions clone into dot-prefixed directories (e.g., `.my-extension`) to remain hidden from the registry until ready.
- **Safety**: The `.modly-incomplete` marker prevents partial installations from loading, with checks implemented in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py).
- **Setup**: The `/setup/{ext_id}` endpoint in [`api/routers/extensions.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/extensions.py) creates isolated virtual environments for subprocess-mode extensions.
- **Backup**: Dot-prefixed folders serve as atomic backups during updates, allowing instant rollback by renaming directories.
- **Restore**: Recovery is filesystem-based—simply rename the backup folder and call `/extensions/reload` to refresh the registry.

## Frequently Asked Questions

### What happens if an extension installation is interrupted?

If the installation process terminates before completion, the `.modly-incomplete` marker file remains in the dot-prefixed directory. During the next registry scan in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), the system detects this marker and skips the directory, preventing Modly from attempting to load a corrupted or incomplete extension. The user can safely delete the temporary folder and retry the installation.

### Can I manually restore an extension without using the API?

Yes. The backup mechanism relies entirely on filesystem operations. If you have a dot-prefixed backup folder (such as `.my-extension-backup`), you can restore it by deleting the current extension folder and renaming the backup to remove the dot prefix. After renaming, send a POST request to `/extensions/reload` or restart Modly to refresh the registry.

### Where does Modly store extensions by default?

By default, Modly stores extensions in `~/.modly/extensions` as defined by the `EXTENSIONS_DIR` environment variable. Within this directory, active extensions appear as normal folders (e.g., `my-extension`), while staging directories and backups use dot prefixes (e.g., `.my-extension` or `.my-extension-backup`) to remain invisible to the registry scanner.

### Does every extension need a setup.py file?

No. Extensions only require a [`setup.py`](https://github.com/lightningpixel/modly/blob/main/setup.py) if they intend to run in **subprocess mode** with an isolated virtual environment. Extensions without [`setup.py`](https://github.com/lightningpixel/modly/blob/main/setup.py) are treated as legacy extensions and load directly via their [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) file in the main process. However, all valid extensions must contain both a [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and a [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) to be recognized by the GeneratorRegistry.