How Modly Schedules and Executes Workflow Runs in FastAPI

Modly schedules workflow runs by queuing heavy generation tasks as FastAPI background tasks, returning an immediate run_id while processing continues asynchronously.

The Modly open-source 3D generation platform implements a fully asynchronous workflow-run pipeline. When you submit an image for processing, the API delegates the computationally expensive mesh generation to a background worker, allowing the HTTP response to return instantly with a job identifier you can poll for results.

---

Scheduling a Workflow Run via POST /workflow-runs/from-image

Workflow runs are initiated through the POST /workflow-runs/from-image endpoint defined in api/routers/workflow_runs.py. According to the Modly source code, this endpoint performs six critical operations before returning control to the client.

Endpoint Processing Steps

1. Validates the uploaded image and the requested model_id against available generators.

2. Registers the model in the global generator_registry from services/generator_registry.py if not already loaded.

3. Generates a unique job identifier using uuid.uuid4().

4. Creates a JobStatus entry in the in-memory _jobs dictionary with initial status "pending".

5. Allocates a cancellation flag in _cancel_events for potential abort operations.

6. Enqueues a background task via background_tasks.add_task(_run_generation, ...) — this schedules the actual generation work without blocking the HTTP response.

The background task mechanism is core to Modly's architecture. By using FastAPI's BackgroundTasks, the server immediately returns a JSON payload containing the run_id while _run_generation continues executing in a separate thread.

# Conceptual flow from api/routers/workflow_runs.py (lines 25-62)

background_tasks.add_task(
    _run_generation,
    job_id=run_id,
    image=validated_image,
    model_id=model_id,
    params=params,
    cancel_event=_cancel_events[run_id]
)
return WorkflowRunResponse(run_id=run_id, status="pending")

---

Background Task Execution in _run_generation

The heavy lifting occurs in _run_generation, implemented in api/routers/generation.py. This function runs after the HTTP connection closes, updating job state progressively.

State Updates During Processing

The background task maintains the _jobs entry throughout the pipeline:

• Progress percentage — increments from 0 to 100 as generation stages complete.
• Current step description — user-visible status like "extracting features" or "mesh optimization".
• Error capture — any exceptions are logged and stored in the job record.
• Output URL — final 3D mesh location once generation succeeds.

Since _run_generation operates outside the request-response cycle, it can spawn subprocesses, perform GPU-intensive computations, and run for minutes without affecting API responsiveness.

---

Querying Workflow Run Status

Clients track progress via GET /workflow-runs/{run_id}, which reads the current state from _jobs and returns a WorkflowRunStatus object (lines 65-84 of workflow_runs.py).

Status Response Structure

Field	Description
run_id	The UUID assigned at creation
status	"pending", "running", "completed", "failed", or "cancelled"
progress	Integer percentage (0-100)
step	Human-readable current operation
error	Error message if failed
scene_candidate	Loadable 3D scene data when complete

# Retrieve current status via HTTP

curl http://localhost:8000/workflow-runs/<run_id>

# Or using the Modly CLI

python tools/modly-cli/agent.py workflow-run status <run_id>

---

Cancelling an In-Progress Run

The POST /workflow-runs/{run_id}/cancel endpoint (lines 86-98) implements cooperative cancellation:

1. Marks the job status as "cancelled" in _jobs.
2. Sets the Event object in _cancel_events[run_id], signaling _run_generation to stop.
3. If a generator subprocess is active, terminates it forcefully.

This design allows generators to check cancel_event.is_set() at safe interruption points, ensuring clean shutdown even during long-running operations.

# Cancel via HTTP

curl -X POST http://localhost:8000/workflow-runs/<run_id>/cancel

# Or via CLI

python tools/modly-cli/agent.py workflow-run cancel <run_id>

---

In-Memory State Management

All workflow run coordination relies on three global registries defined in api/routers/generation.py:

• _jobs — dict[str, JobStatus] mapping run IDs to full status objects.
• _cancel_events — dict[str, Event] providing thread-safe cancellation signals.
• _cancelled — set[str] tracking which runs received cancellation requests.

These structures are in-memory only — restarting the FastAPI server clears all run history. For production deployments, the Modly architecture would need external persistence (Redis, database) to survive restarts.

---

Complete CLI Usage Example

The tools/modly-cli/agent.py wrapper simplifies interaction with the workflow-run API:

# Start a run and block until completion (--wait flag)

python tools/modly-cli/agent.py workflow-run start \
  --image ./my-photo.png \
  --wait

# Poll status manually

python tools/modly-cli/agent.py workflow-run status <run_id>

# Abort if needed

python tools/modly-cli/agent.py workflow-run cancel <run_id>

---

Summary

• FastAPI background tasks decouple HTTP responses from generation work, enabling immediate run_id returns.
• _run_generation in api/routers/generation.py performs the actual mesh generation asynchronously.
• In-memory registries (_jobs, _cancel_events) track state and support cancellation without external dependencies.
• GET /workflow-runs/{run_id} provides real-time progress polling for UI or CLI clients.
• Cancellation uses threading.Event for cooperative shutdown plus subprocess termination as fallback.

---

Frequently Asked Questions

How does Modly handle long-running generation without blocking API requests?

Modly uses FastAPI's BackgroundTasks to schedule _run_generation after sending the HTTP response. The endpoint returns a run_id immediately, while the actual processing continues in a background thread. This pattern is implemented in api/routers/workflow_runs.py lines 25-62.

What happens to workflow runs if the Modly server restarts?

All run state is stored in in-memory dictionaries (_jobs, _cancel_events). A server restart clears this state, causing any in-progress runs to be lost. The source code in api/routers/generation.py does not implement persistence.

Can multiple workflow runs execute simultaneously?

Yes. Each run receives its own background task and cancellation event. The FastAPI server can process multiple POST /workflow-runs/from-image requests concurrently, with each _run_generation task running independently unless constrained by GPU or CPU resources.

How does cancellation propagate to the underlying generator?

When POST /workflow-runs/{run_id}/cancel is called, Modly sets the Event object in _cancel_events[run_id] and marks the job cancelled. The _run_generation loop checks this event periodically. For stubborn processes, the endpoint also kills the generator subprocess directly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}