# MiroFish Progress Callback Mechanism: Monitoring Multi-Stage Simulation Preparation

> Understand the MiroFish progress callback mechanism for monitoring multi-stage simulation preparation. Track real-time updates via HTTP API, database, and WebSocket.

- Repository: [BaiFu/mirofish](https://github.com/666ghj/mirofish)
- Tags: internals
- Published: 2026-02-23

---

**The MiroFish progress callback mechanism injects a function from the HTTP API layer through to the service layer, mapping per-stage progress (0-100) to weighted overall percentages while persisting real-time updates to the database and WebSocket clients.**

The MiroFish open-source platform orchestrates complex simulation preparation across multiple logical stages, from reading graph entities to generating agent profiles and copying scripts. To provide real-time visibility into these long-running background tasks, the codebase implements a robust **progress callback mechanism** that propagates granular status updates from deep within the service layer up to the HTTP API and frontend clients.

## How the Progress Callback Works in MiroFish

### Callback Definition at the API Layer

In [`backend/app/api/simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/simulation.py), the callback is defined as a nested function within the endpoint that initiates simulation preparation. This function accepts the current stage name, per-stage progress (0-100), a human-readable message, and optional keyword arguments for item-level tracking.

```python
def progress_callback(stage, progress, message, **kwargs):
    # stage → (start%, end%) mapping

    stage_weights = {
        "reading": (0, 20),
        "generating_profiles": (20, 70),
        "generating_config": (70, 90),
        "copying_scripts": (90, 100)
    }
    start, end = stage_weights.get(stage, (0, 100))
    overall = int(start + (end - start) * progress / 100)

```

### Stage Weighting and Overall Progress Calculation

The mechanism maps each preparation stage to a specific weight range within the overall 0-100% scale. Reading entities occupies 0-20%, generating profiles 20-70%, configuration generation 70-90%, and script copying 90-100%. The callback calculates the weighted overall progress using linear interpolation between the stage's start and end boundaries.

### Rich Progress Details for the Frontend

Beyond the single percentage value, the callback constructs a detailed dictionary containing granular metadata. This includes the current stage index, total stage count, per-stage progress, current item counters, and item descriptions. This payload enables the frontend to render detailed progress panes with stage-specific context.

```python
detail = {
    "current_stage": stage,
    "stage_progress": progress,
    "current_item": kwargs.get("current", 0),
    "total_items": kwargs.get("total", 0),
    "item_description": message,
}

```

### Task Manager Integration

Every invocation of the callback triggers an update to the `TaskManager`, which persists the progress to the database and broadcasts it to connected WebSocket clients. The update includes the calculated overall percentage, a human-readable message formatted with current/total counters, and the detailed progress payload.

```python
task_manager.update_task(
    task_id,
    progress=overall,
    message=human,
    progress_detail=detail,
)

```

## Propagating Callbacks Through the Service Layer

### Simulation Manager Implementation

The `SimulationManager.prepare_simulation` method in [`backend/app/services/simulation_manager.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_manager.py) accepts the `progress_callback` as an optional argument and invokes it at the beginning and end of each major preparation step. This ensures that even long-running operations like entity reading and profile generation report their status back to the API layer.

```python
if progress_callback:
    progress_callback("reading", 0, "正在连接Zep图谱...")

# … after reading entities …

if progress_callback:
    progress_callback("reading", 100,
        f"完成，共 {filtered.filtered_count} 个实体",
        current=filtered.filtered_count,
        total=filtered.filtered_count)

```

### Profile Generation Wrapper

During the agent profile generation stage, the manager defines a nested `profile_progress` wrapper function that translates granular profile-generation events into the standard callback signature. This wrapper calculates the percentage based on current and total profile counts and forwards the message with the `"generating_profiles"` stage identifier.

```python
def profile_progress(current, total, msg):
    if progress_callback:
        progress_callback(
            "generating_profiles",
            int(current / total * 100),
            msg,
            current=current,
            total=total,
            item_name=msg,
        )

```

## Key Files in the Callback Architecture

The progress callback mechanism spans multiple layers of the MiroFish backend:

- **[`backend/app/api/simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/simulation.py)** – Defines the high-level `progress_callback`, implements stage weighting logic, and interfaces with `TaskManager` for persistence and WebSocket broadcasting.
- **[`backend/app/services/simulation_manager.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_manager.py)** – Accepts and invokes the callback during entity reading, profile generation, configuration building, and script copying.
- **[`backend/app/services/simulation_config_generator.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_config_generator.py)** – Contains internal `report_progress` hooks called by the manager to emit step numbers aggregated by the outer callback.
- **[`backend/app/services/oasis_profile_generator.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/oasis_profile_generator.py)** – Generates agent profiles and invokes the nested `profile_progress` wrapper provided by the manager.
- **[`backend/app/api/report.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/report.py)** – Demonstrates a similar callback pattern for report generation tasks.
- **[`backend/app/api/graph.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/graph.py)** – Implements progress callbacks for graph-related background operations.

## Summary

The MiroFish progress callback mechanism provides real-time visibility into multi-stage simulation preparation through a carefully architected injection pattern:

- **API Layer Definition** – The callback is defined in [`backend/app/api/simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/simulation.py) with stage-specific weight mappings (reading 0-20%, profiles 20-70%, config 70-90%, scripts 90-100%).
- **Weighted Progress Calculation** – Per-stage 0-100 values are interpolated into overall percentages using linear mapping between stage boundaries.
- **Rich Metadata Payload** – Each callback invocation carries detailed progress data including current/total item counts, stage indices, and human-readable descriptions.
- **Service Layer Propagation** – `SimulationManager` in [`backend/app/services/simulation_manager.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_manager.py) receives the callback and invokes it at stage boundaries, using nested wrappers like `profile_progress` to adapt internal APIs.
- **Real-Time Persistence** – Updates flow through `TaskManager` to the database and WebSocket clients, enabling live progress bars in the frontend.

## Frequently Asked Questions

### How does MiroFish calculate overall progress from individual stage progress?

MiroFish assigns each preparation stage a weighted range within the 0-100% scale: reading entities (0-20%), generating profiles (20-70%), generating configuration (70-90%), and copying scripts (90-100%). The callback in [`backend/app/api/simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/simulation.py) interpolates the per-stage progress into the stage's allocated range using the formula `overall = int(start + (end - start) * progress / 100)`, producing a smooth global progress bar.

### What information does the progress callback send to the frontend?

Beyond the overall percentage, the callback constructs a detailed dictionary containing the current stage name, per-stage progress value, current item index, total items in the stage, and a human-readable message. This payload is passed to `task_manager.update_task()` along with the task ID, enabling the frontend to render rich progress panes with stage-specific context and item-level granularity.

### Where is the progress callback defined and how does it reach the service layer?

The callback is defined as a nested function within the simulation preparation endpoint in [`backend/app/api/simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/api/simulation.py). It is passed as the `progress_callback` parameter to `SimulationManager.prepare_simulation()` in [`backend/app/services/simulation_manager.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_manager.py). The manager then invokes this function at the beginning and end of each major step (reading, profile generation, configuration, copying), and creates specialized wrappers like `profile_progress` to adapt internal progress events to the standard callback signature.

### Can the progress callback mechanism handle nested or parallel operations?

Yes, the mechanism supports nested operations through wrapper functions. For example, during agent profile generation in [`backend/app/services/simulation_manager.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_manager.py), the manager defines a nested `profile_progress` function that translates granular profile-generation events into the standard callback format with the `"generating_profiles"` stage identifier. This allows parallel profile generation processes to report progress through a unified interface while maintaining the weighted overall progress calculation.