# How the Artifact Registry Service Manages and Downloads Model Files in Modly

> Discover how Modly's artifact registry service manages and downloads model files. Explore centralized storage, automatic Hugging Face downloads, and FastAPI endpoint exposure. Learn more!

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: how-to-guide
- Published: 2026-08-15

---

**The artifact registry service in Modly uses a centralized `GeneratorRegistry` to discover model extensions, instantiate generators in direct or subprocess modes, automatically download Hugging Face repositories on demand, and expose model status through FastAPI endpoints—all with files stored under `~/.modly/models`.**

The **artifact registry** is the backbone of Modly's model management system. Implemented in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py), it orchestrates the entire lifecycle of model adapters from discovery through download to runtime exposure. Understanding its architecture helps developers extend Modly with custom generators and troubleshoot deployment issues.

## Core Architecture of the GeneratorRegistry

The `GeneratorRegistry` class serves as Modly's artifact registry service. It maintains the active model state, discovers available extensions, and delegates file operations to generator instances.

### Model Storage Location

All model files live under a consistent directory structure. The registry initializes the **models directory** at startup:

```python

# From api/services/generator_registry.py lines 24-30

MODELS_DIR = Path(os.environ.get("MODLY_MODELS_DIR", "~/.modly/models")).expanduser()
WORKSPACE_DIR = Path(os.environ.get("MODLY_WORKSPACE_DIR", "~/.modly/workspace")).expanduser()

MODELS_DIR.mkdir(parents=True, exist_ok=True)
WORKSPACE_DIR.mkdir(parents=True, exist_ok=True)

```

This defaults to `~/.modly/models` but respects the `MODLY_MODELS_DIR` environment variable for custom deployments.

## Model Discovery and Registration

### Extension Scanning Process

The registry discovers models by scanning `EXTENSIONS_DIR` for valid extension folders. Each extension must contain:

- [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) — model metadata including Hugging Face repository and download settings
- [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) — the generator class implementation

The `_discover_extensions()` method builds a lookup map:

```python

# api/services/generator_registry.py lines 45-53

def _discover_extensions(self) -> Dict[str, Tuple[Type, dict, Path]]:
    """Discover all available generator extensions."""
    extensions = {}
    for ext_dir in EXTENSIONS_DIR.iterdir():
        if not ext_dir.is_dir():
            continue
        manifest_path = ext_dir / "manifest.json"
        generator_path = ext_dir / "generator.py"
        # ... validation and registration logic

        extensions[full_id] = (GeneratorClass, manifest, ext_dir)
    return extensions

```

This produces a mapping of **full IDs** to `(GeneratorClass, manifest, ext_dir)` tuples that powers all subsequent operations.

### Manifest.json Structure

Each extension's [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) declares critical metadata used by the artifact registry service:

| Field | Purpose |
|-------|---------|
| `id` | Unique model identifier |
| `hf_repo` | Hugging Face repository path for download |
| `download_check` | File to verify successful download |
| `hf_skip_prefixes` | Files to exclude from download |
| `hf_include_prefixes` | Only download matching files (if specified) |

## Generator Instantiation Modes

The registry supports two execution modes determined during `initialize()`:

### Direct Mode

For simple extensions without special environment requirements:

```python

# api/services/generator_registry.py lines 71-77

if not needs_subprocess:
    # Direct import and instantiation

    generator_class = discovered[model_id][0]
    instance = generator_class(
        model_dir=MODELS_DIR / model_id,
        workspace_dir=WORKSPACE_DIR
    )
    self._generators[model_id] = instance

```

The generator class imports directly from [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py) and receives pre-configured paths.

### Subprocess Mode

For extensions requiring isolation (`.venv` present or build steps needed):

```python

# api/services/generator_registry.py lines 78-87

else:
    # Wrap in subprocess for isolation

    from services.extension_process import ExtensionProcess
    process = ExtensionProcess(
        ext_dir=discovered[model_id][2],
        model_dir=MODELS_DIR / model_id,
        workspace_dir=WORKSPACE_DIR
    )
    self._generators[model_id] = process

```

The `ExtensionProcess` handles its own download logic; the registry only calls `load()` after initialization.

## Automatic Download and Model Loading

The artifact registry service's download logic triggers transparently when models are accessed.

### On-Demand Download Flow

When `get_active()` or `get_generator()` is called:

```python

# api/services/generator_registry.py lines 41-55

def get_active(self) -> BaseGenerator:
    """Get the currently active generator, downloading if necessary."""
    if self._active_model_id is None:
        raise RuntimeError("No active model set. Call switch_model() first.")
    
    generator = self._generators.get(self._active_model_id)
    if generator is None:
        raise RuntimeError(f"Model {self._active_model_id} not initialized")
    
    # Trigger automatic download for direct-mode generators

    if hasattr(generator, '_auto_download') and not generator.is_downloaded():
        generator._auto_download()
    
    if not generator.is_loaded():
        generator.load()
    
    return generator

```

### Direct-Mode Download Implementation

The `_auto_download()` method in [`services/generators/base.py`](https://github.com/lightningpixel/modly/blob/main/services/generators/base.py) implements Hugging Face repository downloads:

- Respects `hf_skip_prefixes` and `hf_include_prefixes` from manifest
- Verifies completion via `download_check` file
- Caches files under the model-specific subdirectory of `MODELS_DIR`

### Subprocess-Mode Delegation

For isolated extensions, download responsibility shifts:

```python

# Subprocess extensions handle download internally

# Registry only verifies load() succeeds after subprocess initialization

process.load()  # Download already completed by ExtensionProcess

```

## Status Reporting and Monitoring

The artifact registry service exposes comprehensive model status through two methods.

### Active Model Status

```python

# api/services/generator_registry.py lines 86-94

def active_status(self) -> dict:
    """Get status of the currently active model."""
    if self._active_model_id is None:
        return {"active": None}
    
    gen = self._generators.get(self._active_model_id)
    return {
        "id": self._active_model_id,
        "downloaded": gen.is_downloaded() if gen else False,
        "loaded": gen.is_loaded() if gen else False,
        "active": True
    }

```

### All Models Status

```python

# api/services/generator_registry.py lines 96-112

def all_status(self) -> List[dict]:
    """Get status of all discovered models."""
    statuses = []
    for model_id in self._discovered.keys():
        gen = self._generators.get(model_id)
        statuses.append({
            "id": model_id,
            "downloaded": gen.is_downloaded() if gen else False,
            "loaded": gen.is_loaded() if gen else False,
            "active": model_id == self._active_model_id
        })
    return statuses

```

## Practical Usage Examples

### Basic Model Access with Automatic Download

```python
from api.services.generator_registry import generator_registry

# Triggers download if model files missing

active_gen = generator_registry.get_active()
print(f"Model directory: {active_gen.model_dir}")

# Output: Model directory: /home/user/.modly/models/stable-fast-3d

```

### Programmatic Model Switching

```python

# Switch to different model—previous generator unloaded automatically

generator_registry.switch_model('my-custom-model')

# New model downloads/loads on next access

new_gen = generator_registry.get_active()

```

### Registry Inspection

```python
import json

# Check all available models and their states

print(json.dumps(generator_registry.all_status(), indent=2))

```

Sample output:

```json
[
  {
    "id": "stable-fast-3d",
    "downloaded": true,
    "loaded": true,
    "active": true
  },
  {
    "id": "my-custom-model",
    "downloaded": false,
    "loaded": false,
    "active": false
  }
]

```

## API Exposure

The artifact registry service feeds FastAPI endpoints defined in [`api/routers/model.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/model.py):

| Endpoint | Registry Method | Purpose |
|----------|-----------------|---------|
| `GET /model/status` | `active_status()` | Current model state |
| `GET /model/all` | `all_status()` | All models overview |
| `GET /model/params` | `get_active().get_params()` | Active model parameters |

## Summary

- **Discovery**: `GeneratorRegistry._discover_extensions()` scans `EXTENSIONS_DIR` for valid extensions with [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) and [`generator.py`](https://github.com/lightningpixel/modly/blob/main/generator.py)
- **Storage**: All models stored under `~/.modly/models` (configurable via `MODLY_MODELS_DIR`)
- **Instantiation**: Direct mode for simple extensions, subprocess mode for isolated environments
- **Download**: Automatic via `BaseGenerator._auto_download()` for direct mode; delegated to `ExtensionProcess` for subprocess mode
- **Loading**: Lazy on first access through `get_active()` or `get_generator()`
- **Status**: Full visibility via `active_status()` and `all_status()` methods

## Frequently Asked Questions

### Where does Modly store downloaded model files?

Modly stores all model files under `~/.modly/models` by default, as implemented in [`api/services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/api/services/generator_registry.py) lines 24-30. This path is configurable via the `MODLY_MODELS_DIR` environment variable before starting the service.

### How does the artifact registry service know when to download a model?

The registry checks `gen.is_downloaded()` when `get_active()` is called. If the model files are missing and the generator supports `_auto_download()`, it triggers automatically before returning the generator instance. For subprocess-mode extensions, download happens internally within the `ExtensionProcess`.

### What controls which files get downloaded from Hugging Face?

The [`manifest.json`](https://github.com/lightningpixel/modly/blob/main/manifest.json) in each extension directory specifies download behavior through `hf_skip_prefixes` (exclude), `hf_include_prefixes` (whitelist), and `download_check` (verification file). These are consumed by `BaseGenerator._auto_download()` in [`services/generators/base.py`](https://github.com/lightningpixel/modly/blob/main/services/generators/base.py).

### Can I use a model before it finishes downloading?

No. The registry blocks on download completion in direct mode, or on subprocess initialization in isolated mode. The `get_active()` method only returns after `is_loaded()` confirms the model is ready for inference.