MiroFish Progress Callback Mechanism: Monitoring Multi-Stage Simulation Preparation

The MiroFish progress callback mechanism injects a function from the HTTP API layer through to the service layer, mapping per-stage progress (0-100) to weighted overall percentages while persisting real-time updates to the database and WebSocket clients.

The MiroFish open-source platform orchestrates complex simulation preparation across multiple logical stages, from reading graph entities to generating agent profiles and copying scripts. To provide real-time visibility into these long-running background tasks, the codebase implements a robust progress callback mechanism that propagates granular status updates from deep within the service layer up to the HTTP API and frontend clients.

How the Progress Callback Works in MiroFish

Callback Definition at the API Layer

In backend/app/api/simulation.py, the callback is defined as a nested function within the endpoint that initiates simulation preparation. This function accepts the current stage name, per-stage progress (0-100), a human-readable message, and optional keyword arguments for item-level tracking.

def progress_callback(stage, progress, message, **kwargs):
    # stage → (start%, end%) mapping

    stage_weights = {
        "reading": (0, 20),
        "generating_profiles": (20, 70),
        "generating_config": (70, 90),
        "copying_scripts": (90, 100)
    }
    start, end = stage_weights.get(stage, (0, 100))
    overall = int(start + (end - start) * progress / 100)

Stage Weighting and Overall Progress Calculation

The mechanism maps each preparation stage to a specific weight range within the overall 0-100% scale. Reading entities occupies 0-20%, generating profiles 20-70%, configuration generation 70-90%, and script copying 90-100%. The callback calculates the weighted overall progress using linear interpolation between the stage's start and end boundaries.

Rich Progress Details for the Frontend

Beyond the single percentage value, the callback constructs a detailed dictionary containing granular metadata. This includes the current stage index, total stage count, per-stage progress, current item counters, and item descriptions. This payload enables the frontend to render detailed progress panes with stage-specific context.

detail = {
    "current_stage": stage,
    "stage_progress": progress,
    "current_item": kwargs.get("current", 0),
    "total_items": kwargs.get("total", 0),
    "item_description": message,
}

Task Manager Integration

Every invocation of the callback triggers an update to the TaskManager, which persists the progress to the database and broadcasts it to connected WebSocket clients. The update includes the calculated overall percentage, a human-readable message formatted with current/total counters, and the detailed progress payload.

task_manager.update_task(
    task_id,
    progress=overall,
    message=human,
    progress_detail=detail,
)

Propagating Callbacks Through the Service Layer

Simulation Manager Implementation

The SimulationManager.prepare_simulation method in backend/app/services/simulation_manager.py accepts the progress_callback as an optional argument and invokes it at the beginning and end of each major preparation step. This ensures that even long-running operations like entity reading and profile generation report their status back to the API layer.

if progress_callback:
    progress_callback("reading", 0, "正在连接Zep图谱...")

# … after reading entities …

if progress_callback:
    progress_callback("reading", 100,
        f"完成，共 {filtered.filtered_count} 个实体",
        current=filtered.filtered_count,
        total=filtered.filtered_count)

Profile Generation Wrapper

During the agent profile generation stage, the manager defines a nested profile_progress wrapper function that translates granular profile-generation events into the standard callback signature. This wrapper calculates the percentage based on current and total profile counts and forwards the message with the "generating_profiles" stage identifier.

def profile_progress(current, total, msg):
    if progress_callback:
        progress_callback(
            "generating_profiles",
            int(current / total * 100),
            msg,
            current=current,
            total=total,
            item_name=msg,
        )

Key Files in the Callback Architecture

The progress callback mechanism spans multiple layers of the MiroFish backend:

• backend/app/api/simulation.py – Defines the high-level progress_callback, implements stage weighting logic, and interfaces with TaskManager for persistence and WebSocket broadcasting.
• backend/app/services/simulation_manager.py – Accepts and invokes the callback during entity reading, profile generation, configuration building, and script copying.
• backend/app/services/simulation_config_generator.py – Contains internal report_progress hooks called by the manager to emit step numbers aggregated by the outer callback.
• backend/app/services/oasis_profile_generator.py – Generates agent profiles and invokes the nested profile_progress wrapper provided by the manager.
• backend/app/api/report.py – Demonstrates a similar callback pattern for report generation tasks.
• backend/app/api/graph.py – Implements progress callbacks for graph-related background operations.

Summary

The MiroFish progress callback mechanism provides real-time visibility into multi-stage simulation preparation through a carefully architected injection pattern:

• API Layer Definition – The callback is defined in backend/app/api/simulation.py with stage-specific weight mappings (reading 0-20%, profiles 20-70%, config 70-90%, scripts 90-100%).
• Weighted Progress Calculation – Per-stage 0-100 values are interpolated into overall percentages using linear mapping between stage boundaries.
• Rich Metadata Payload – Each callback invocation carries detailed progress data including current/total item counts, stage indices, and human-readable descriptions.
• Service Layer Propagation – SimulationManager in backend/app/services/simulation_manager.py receives the callback and invokes it at stage boundaries, using nested wrappers like profile_progress to adapt internal APIs.
• Real-Time Persistence – Updates flow through TaskManager to the database and WebSocket clients, enabling live progress bars in the frontend.

Frequently Asked Questions

How does MiroFish calculate overall progress from individual stage progress?

MiroFish assigns each preparation stage a weighted range within the 0-100% scale: reading entities (0-20%), generating profiles (20-70%), generating configuration (70-90%), and copying scripts (90-100%). The callback in backend/app/api/simulation.py interpolates the per-stage progress into the stage's allocated range using the formula overall = int(start + (end - start) * progress / 100), producing a smooth global progress bar.

What information does the progress callback send to the frontend?

Beyond the overall percentage, the callback constructs a detailed dictionary containing the current stage name, per-stage progress value, current item index, total items in the stage, and a human-readable message. This payload is passed to task_manager.update_task() along with the task ID, enabling the frontend to render rich progress panes with stage-specific context and item-level granularity.

Where is the progress callback defined and how does it reach the service layer?

The callback is defined as a nested function within the simulation preparation endpoint in backend/app/api/simulation.py. It is passed as the progress_callback parameter to SimulationManager.prepare_simulation() in backend/app/services/simulation_manager.py. The manager then invokes this function at the beginning and end of each major step (reading, profile generation, configuration, copying), and creates specialized wrappers like profile_progress to adapt internal progress events to the standard callback signature.

Can the progress callback mechanism handle nested or parallel operations?

Yes, the mechanism supports nested operations through wrapper functions. For example, during agent profile generation in backend/app/services/simulation_manager.py, the manager defines a nested profile_progress function that translates granular profile-generation events into the standard callback format with the "generating_profiles" stage identifier. This allows parallel profile generation processes to report progress through a unified interface while maintaining the weighted overall progress calculation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}