MiroFish File Structure for Storing Simulation Data and Configurations

MiroFish stores every simulation in a dedicated subdirectory under backend/uploads/simulations, containing JSON configuration files, agent profiles, runtime state, and optional execution logs.

Managing simulation data requires a predictable directory hierarchy. In the 666ghj/mirofish repository, the file structure for storing simulation data and configurations follows a strict convention that separates persistent meta-state from runtime execution data. Each simulation receives its own isolated folder, making it easy to archive, inspect, or delete individual runs without affecting the broader system.

Root Storage Location

All simulation data lives inside backend/uploads/simulations, defined centrally in backend/app/config.py. The configuration constant OASIS_SIMULATION_DATA_DIR establishes this path:

OASIS_SIMULATION_DATA_DIR = os.path.join(
    os.path.dirname(__file__), '../uploads/simulations'
)

When a user creates a new simulation via the API, the system generates a subdirectory named after the simulation ID (e.g., sim_20240223_01). This self-contained unit houses every file related to that specific run, created automatically during the preparation phase.

Configuration and Profile Files

Each simulation directory contains several JSON and CSV files that define the simulation parameters and agent populations:

• simulation_config.json: The complete configuration object generated by the LLM, including time settings, agent activities, planned events, and platform-specific parameters. Created in backend/app/api/simulation.py during the preparation phase.

• reddit_profiles.json: Array of Reddit agent profiles containing names, personas, and activity rules for each simulated user.

• twitter_profiles.csv: Tabular data for Twitter/X agents, stored as CSV for easy parsing by the underlying OASIS engine.

• state.json: Persistent meta-state tracking the simulation's lifecycle status (preparing, ready, running, completed), timestamps, and entity counts. The API consults this file to determine if a simulation is ready for execution.

Runtime Execution Files

During active execution, the runner populates additional files to track progress and capture outputs:

• run_state.json: Live runtime state managed by SimulationRunner in backend/app/services/simulation_runner.py. Tracks the current round, runner status, and execution progress.

• actions.jsonl: Optional newline-delimited JSON log recording every agent action during the simulation. Each line contains a single JSON object representing one action.

• simulation.log: Raw stdout and stderr capture from the underlying OASIS engine process, useful for debugging execution failures.

The simulation scripts themselves (such as run_twitter_simulation.py and run_reddit_simulation.py) remain in backend/scripts and are not copied into individual simulation directories. The runner references these scripts directly when launching execution.

Accessing Simulation Data Programmatically

To retrieve a simulation's directory path, combine the base configuration with the simulation ID:

from backend.app.config import Config
import os

def get_simulation_dir(simulation_id: str) -> str:
    """Return the absolute path for a given simulation."""
    return os.path.join(Config.OASIS_SIMULATION_DATA_DIR, simulation_id)

# Example usage

sim_dir = get_simulation_dir("sim_20240223_01")
print(sim_dir)

# Output: /path/to/mirofish/backend/uploads/simulations/sim_20240223_01

To load the configuration file for analysis or verification:

import json
from pathlib import Path
from backend.app.config import Config

def load_simulation_config(simulation_id: str) -> dict:
    cfg_path = Path(Config.OASIS_SIMULATION_DATA_DIR) / simulation_id / "simulation_config.json"
    with cfg_path.open(encoding="utf-8") as f:
        return json.load(f)

config = load_simulation_config("sim_20240223_01")
print(config["time_config"]["total_simulation_hours"])

To check simulation readiness programmatically:

from backend.app.api.simulation import _check_simulation_prepared

ready, info = _check_simulation_prepared("sim_20240223_01")
if ready:
    print("Simulation ready! Status:", info["status"])
else:
    print("Not ready:", info["reason"])

Summary

• Root Directory: All simulations live under backend/uploads/simulations, defined by OASIS_SIMULATION_DATA_DIR in backend/app/config.py.
• Per-Simulation Folders: Each simulation ID gets its own subdirectory containing all related files.
• Configuration Layer: simulation_config.json, reddit_profiles.json, and twitter_profiles.csv define the simulation parameters and agents.
• State Management: state.json tracks persistent meta-state, while run_state.json handles active execution status.
• Execution Logs: actions.jsonl and simulation.log capture detailed runtime output for debugging.
• Script Separation: Runner scripts remain in backend/scripts and are referenced, not copied, keeping simulation directories clean.

Frequently Asked Questions

Where does MiroFish store simulation data?

MiroFish stores all simulation data under backend/uploads/simulations, with each simulation receiving a dedicated subdirectory named after its unique simulation ID. This root path is defined in backend/app/config.py as the OASIS_SIMULATION_DATA_DIR constant.

What files are generated when creating a new simulation?

Each simulation directory contains state.json for lifecycle tracking, simulation_config.json for LLM-generated parameters, reddit_profiles.json and twitter_profiles.csv for agent definitions, and optionally run_state.json, actions.jsonl, and simulation.log during execution.

How does the system check if a simulation is ready to run?

The API inspects state.json via the _check_simulation_prepared function in backend/app/api/simulation.py. This verification confirms that all configuration files exist and that the persistent meta-state indicates a "ready" status before allowing execution to begin.

Are the simulation scripts copied into each simulation folder?

No. The OASIS runner scripts (such as run_twitter_simulation.py) remain in backend/scripts and are referenced directly by the SimulationRunner service. Only data, configuration, and log files reside in the individual simulation directories under backend/uploads/simulations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}