# How Modly Schedules and Executes Workflow Runs in FastAPI

> Learn how Modly schedules and executes workflow runs in FastAPI using background tasks for asynchronous processing and immediate run_id returns. Optimize your workflows now.

- Repository: [lightningpixel/modly](https://github.com/lightningpixel/modly)
- Tags: internals
- Published: 2026-08-20

---

**Modly schedules workflow runs by queuing heavy generation tasks as FastAPI background tasks, returning an immediate `run_id` while processing continues asynchronously.**

The **Modly** open-source 3D generation platform implements a fully asynchronous workflow-run pipeline. When you submit an image for processing, the API delegates the computationally expensive mesh generation to a background worker, allowing the HTTP response to return instantly with a job identifier you can poll for results.

---

## Scheduling a Workflow Run via POST /workflow-runs/from-image

Workflow runs are initiated through the `POST /workflow-runs/from-image` endpoint defined in [`api/routers/workflow_runs.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/workflow_runs.py). According to the Modly source code, this endpoint performs six critical operations before returning control to the client.

### Endpoint Processing Steps

1. **Validates the uploaded image** and the requested `model_id` against available generators.

2. **Registers the model** in the global `generator_registry` from [`services/generator_registry.py`](https://github.com/lightningpixel/modly/blob/main/services/generator_registry.py) if not already loaded.

3. **Generates a unique job identifier** using `uuid.uuid4()`.

4. **Creates a `JobStatus` entry** in the in-memory `_jobs` dictionary with initial status `"pending"`.

5. **Allocates a cancellation flag** in `_cancel_events` for potential abort operations.

6. **Enqueues a background task** via `background_tasks.add_task(_run_generation, ...)` — this schedules the actual generation work without blocking the HTTP response.

The background task mechanism is core to Modly's architecture. By using FastAPI's `BackgroundTasks`, the server immediately returns a JSON payload containing the `run_id` while `_run_generation` continues executing in a separate thread.

```python

# Conceptual flow from api/routers/workflow_runs.py (lines 25-62)

background_tasks.add_task(
    _run_generation,
    job_id=run_id,
    image=validated_image,
    model_id=model_id,
    params=params,
    cancel_event=_cancel_events[run_id]
)
return WorkflowRunResponse(run_id=run_id, status="pending")

```

---

## Background Task Execution in _run_generation

The heavy lifting occurs in `_run_generation`, implemented in **[`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py)**. This function runs after the HTTP connection closes, updating job state progressively.

### State Updates During Processing

The background task maintains the `_jobs` entry throughout the pipeline:

- **Progress percentage** — increments from 0 to 100 as generation stages complete.
- **Current step description** — user-visible status like "extracting features" or "mesh optimization".
- **Error capture** — any exceptions are logged and stored in the job record.
- **Output URL** — final 3D mesh location once generation succeeds.

Since `_run_generation` operates outside the request-response cycle, it can spawn subprocesses, perform GPU-intensive computations, and run for minutes without affecting API responsiveness.

---

## Querying Workflow Run Status

Clients track progress via **`GET /workflow-runs/{run_id}`**, which reads the current state from `_jobs` and returns a `WorkflowRunStatus` object (lines 65-84 of [`workflow_runs.py`](https://github.com/lightningpixel/modly/blob/main/workflow_runs.py)).

### Status Response Structure

| Field | Description |
|-------|-------------|
| `run_id` | The UUID assigned at creation |
| `status` | `"pending"`, `"running"`, `"completed"`, `"failed"`, or `"cancelled"` |
| `progress` | Integer percentage (0-100) |
| `step` | Human-readable current operation |
| `error` | Error message if failed |
| `scene_candidate` | Loadable 3D scene data when complete |

```bash

# Retrieve current status via HTTP

curl http://localhost:8000/workflow-runs/<run_id>

# Or using the Modly CLI

python tools/modly-cli/agent.py workflow-run status <run_id>

```

---

## Cancelling an In-Progress Run

The **`POST /workflow-runs/{run_id}/cancel`** endpoint (lines 86-98) implements cooperative cancellation:

1. Marks the job status as `"cancelled"` in `_jobs`.
2. Sets the `Event` object in `_cancel_events[run_id]`, signaling `_run_generation` to stop.
3. If a generator subprocess is active, terminates it forcefully.

This design allows generators to check `cancel_event.is_set()` at safe interruption points, ensuring clean shutdown even during long-running operations.

```bash

# Cancel via HTTP

curl -X POST http://localhost:8000/workflow-runs/<run_id>/cancel

# Or via CLI

python tools/modly-cli/agent.py workflow-run cancel <run_id>

```

---

## In-Memory State Management

All workflow run coordination relies on three global registries defined in [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py):

- **`_jobs`** — `dict[str, JobStatus]` mapping run IDs to full status objects.
- **`_cancel_events`** — `dict[str, Event]` providing thread-safe cancellation signals.
- **`_cancelled`** — `set[str]` tracking which runs received cancellation requests.

These structures are **in-memory only** — restarting the FastAPI server clears all run history. For production deployments, the Modly architecture would need external persistence (Redis, database) to survive restarts.

---

## Complete CLI Usage Example

The [`tools/modly-cli/agent.py`](https://github.com/lightningpixel/modly/blob/main/tools/modly-cli/agent.py) wrapper simplifies interaction with the workflow-run API:

```bash

# Start a run and block until completion (--wait flag)

python tools/modly-cli/agent.py workflow-run start \
  --image ./my-photo.png \
  --wait

# Poll status manually

python tools/modly-cli/agent.py workflow-run status <run_id>

# Abort if needed

python tools/modly-cli/agent.py workflow-run cancel <run_id>

```

---

## Summary

- **FastAPI background tasks** decouple HTTP responses from generation work, enabling immediate `run_id` returns.
- **`_run_generation`** in [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py) performs the actual mesh generation asynchronously.
- **In-memory registries** (`_jobs`, `_cancel_events`) track state and support cancellation without external dependencies.
- **`GET /workflow-runs/{run_id}`** provides real-time progress polling for UI or CLI clients.
- **Cancellation** uses `threading.Event` for cooperative shutdown plus subprocess termination as fallback.

---

## Frequently Asked Questions

### How does Modly handle long-running generation without blocking API requests?

Modly uses FastAPI's `BackgroundTasks` to schedule `_run_generation` after sending the HTTP response. The endpoint returns a `run_id` immediately, while the actual processing continues in a background thread. This pattern is implemented in [`api/routers/workflow_runs.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/workflow_runs.py) lines 25-62.

### What happens to workflow runs if the Modly server restarts?

All run state is stored in in-memory dictionaries (`_jobs`, `_cancel_events`). A server restart clears this state, causing any in-progress runs to be lost. The source code in [`api/routers/generation.py`](https://github.com/lightningpixel/modly/blob/main/api/routers/generation.py) does not implement persistence.

### Can multiple workflow runs execute simultaneously?

Yes. Each run receives its own background task and cancellation event. The FastAPI server can process multiple `POST /workflow-runs/from-image` requests concurrently, with each `_run_generation` task running independently unless constrained by GPU or CPU resources.

### How does cancellation propagate to the underlying generator?

When `POST /workflow-runs/{run_id}/cancel` is called, Modly sets the `Event` object in `_cancel_events[run_id]` and marks the job cancelled. The `_run_generation` loop checks this event periodically. For stubborn processes, the endpoint also kills the generator subprocess directly.