# Parallel Twitter and Reddit Simulation Architecture in mirofish: Async Orchestration and IPC Design

> Explore the mirofish architecture for parallel Twitter and Reddit simulations. Discover async orchestration and IPC design for concurrent simulations.

- Repository: [BaiFu/mirofish](https://github.com/666ghj/mirofish)
- Tags: architecture
- Published: 2026-02-23

---

**The mirofish backend runs Twitter and Reddit simulations concurrently using an asyncio-driven orchestrator that launches platform-specific coroutines with `asyncio.gather`, wraps environments in `PlatformSimulation` containers, and enables post-simulation interaction via a file-based IPC mechanism.**

The mirofish open-source project implements a sophisticated backend system for managing parallel simulations of Twitter and Reddit through a unified async architecture. By leveraging Python's `asyncio` library and a modular orchestrator design, the system can execute both social media platform simulations simultaneously while maintaining separate state, logging, and inter-process communication channels for each environment.

## Async Orchestrator and Entry Point

The simulation lifecycle begins in [`backend/scripts/run_parallel_simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/scripts/run_parallel_simulation.py), where the `main()` function (lines 24-78) handles CLI argument parsing and configuration loading. The orchestrator accepts a unified [`simulation_config.json`](https://github.com/666ghj/mirofish/blob/main/simulation_config.json) that defines parameters for both platforms, then initializes a centralized `SimulationLogManager` to coordinate logging across the parallel environments.

When invoked without platform restrictions, the orchestrator determines that both Twitter and Reddit simulations should execute. Rather than running sequentially, the system uses `asyncio.gather` to achieve true concurrency.

## Concurrent Platform Execution

The core parallel execution occurs at lines 84-89 of the orchestrator script, where the awaitable coroutines are launched simultaneously:

```python
await asyncio.gather(
    run_twitter_simulation(config, log_manager, shutdown_event),
    run_reddit_simulation(config, log_manager, shutdown_event)
)

```

Each platform simulation follows an identical structural pattern implemented in [`backend/app/services/simulation_runner.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_runner.py):

- **`run_twitter_simulation`** (lines 1011-1086): Constructs an OASIS environment via `oasis.make`, initializes a platform-specific SQLite database, generates an agent graph from Twitter profile configurations, and executes a round-based simulation loop with action logging and database extraction.

- **`run_reddit_simulation`** (lines 1394-1470): Mirrors the Twitter implementation but operates on Reddit-specific profiles and database schemas, maintaining complete isolation between the two simulation states.

Both functions return a `PlatformSimulation` container object that encapsulates the live environment and agent graph for downstream processing.

## PlatformSimulation Wrapper and State Management

The `PlatformSimulation` class, defined at lines 217-225 in the orchestrator script, serves as the primary abstraction for managing simulation state:

```python
class PlatformSimulation:
    def __init__(self, env, agent_graph, platform_name: str):
        self.env = env
        self.agent_graph = agent_graph
        self.platform_name = platform_name
        self.db_path = None

```

This wrapper preserves references to the OASIS environment (`env`) and the agent relationship graph (`agent_graph`) after the simulation loop completes. When the orchestrator runs both platforms, it instantiates two separate `PlatformSimulation` objects, each with its own database connection and memory space, preventing cross-platform contamination while allowing unified management.

## Inter-Process Communication and Command Mode

After simulation completion, the orchestrator can remain alive in command mode (unless `--no-wait` is specified) to accept external instructions. At lines 304-311, the system instantiates a `ParallelIPCHandler` with references to both platform environments and graphs:

```python
ipc_handler = ParallelIPCHandler(
    twitter_sim=twitter_result,
    reddit_sim=reddit_result,
    command_dir=command_dir,
    response_dir=response_dir
)

```

The `ParallelIPCHandler` class (defined in [`backend/app/services/simulation_ipc.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_ipc.py) and referenced from line 217 onward in the orchestrator) implements a file-based IPC protocol:

1. **Status Publication**: Creates [`env_status.json`](https://github.com/666ghj/mirofish/blob/main/env_status.json) and two directories (`ipc_commands`, `ipc_responses`) under the simulation root.
2. **Command Polling**: The `process_commands` method repeatedly scans the command folder for JSON instruction files.
3. **Dispatch Logic**: Parses commands such as `interview`, `batch_interview`, and `close_env`, routing them to the appropriate platform's `ManualAction` interface.
4. **Response Writing**: Generates response files in `ipc_responses/` and removes processed commands from the queue.

Example command structure for interviewing a specific agent:

```json
{
  "command_id": "001",
  "command_type": "interview",
  "args": {
    "agent_id": 3,
    "prompt": "What do you think about the new policy?",
    "platform": "twitter"
  }
}

```

## Graceful Shutdown Handling

The architecture implements cooperative cancellation through a global `asyncio.Event` named `_shutdown_event`. The `setup_signal_handlers()` function (lines 53-78) registers handlers for `SIGTERM` and `SIGINT` that set this event, signaling all concurrent loops to terminate gracefully.

The shutdown flag propagates through three critical paths:
- The main orchestration loop checks the event between platform executions.
- The IPC polling loop in `ParallelIPCHandler` monitors the flag during command processing.
- The per-round simulation loops within `run_twitter_simulation` and `run_reddit_simulation` verify the event state between rounds, ensuring OASIS environments close properly without data corruption.

## Practical Usage Examples

Execute both simulations in parallel with IPC mode enabled (default):

```bash
python backend/scripts/run_parallel_simulation.py \
    --config path/to/simulation_config.json

```

Run only the Twitter simulation and exit immediately after completion:

```bash
python backend/scripts/run_parallel_simulation.py \
    --config path/to/simulation_config.json \
    --twitter-only \
    --no-wait

```

Send an interview command to a running simulation:

```bash

# Create command file while orchestrator is waiting

echo '{"command_id": "001", "command_type": "interview", "args": {"agent_id": 3, "prompt": "Opinion on policy changes?", "platform": "twitter"}}' \
  > sim_dir/ipc_commands/001_interview.json

```

The handler processes the file, executes the interview against the Twitter OASIS environment, writes the response to [`sim_dir/ipc_responses/001.json`](https://github.com/666ghj/mirofish/blob/main/sim_dir/ipc_responses/001.json), and removes the command file.

## Summary

- **Async Concurrency**: The orchestrator uses `asyncio.gather` in [`backend/scripts/run_parallel_simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/scripts/run_parallel_simulation.py) (lines 84-89) to run Twitter and Reddit simulations simultaneously rather than sequentially.
- **State Isolation**: Each platform runs in its own `PlatformSimulation` wrapper (lines 217-225) with dedicated SQLite databases and agent graphs generated via [`backend/app/services/graph_builder.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/graph_builder.py).
- **File-Based IPC**: The `ParallelIPCHandler` enables post-simulation interaction through JSON command files in `ipc_commands/` and response files in `ipc_responses/`, supporting operations like agent interviews and environment closures.
- **Graceful Termination**: Signal handlers (lines 53-78) set a global `_shutdown_event` that coordinates clean shutdown across the main loop, IPC polling, and per-round simulation logic.
- **Modular Services**: Core logic resides in [`backend/app/services/simulation_runner.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_runner.py) for platform execution and [`backend/app/services/simulation_ipc.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_ipc.py) for command handling, with centralized logging via [`backend/app/utils/logger.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/utils/logger.py).

## Frequently Asked Questions

### How does mirofish prevent the Twitter and Reddit simulations from interfering with each other?

Each simulation runs in complete isolation through separate `PlatformSimulation` instances that encapsulate distinct OASIS environments, SQLite database connections, and agent graphs. Because `run_twitter_simulation` and `run_reddit_simulation` are independent coroutines launched via `asyncio.gather`, they maintain separate memory spaces and database handles, ensuring that actions in one platform never affect the state of the other.

### What types of commands can be sent to running simulations via the IPC mechanism?

The `ParallelIPCHandler` supports three primary command types: `interview` (query a specific agent with a custom prompt), `batch_interview` (interview multiple agents simultaneously), and `close_env` (gracefully terminate a specific platform environment). These commands are submitted as JSON files in the `ipc_commands/` directory, processed by polling loops, and responses are written to corresponding files in `ipc_responses/`.

### Can the simulations be run independently without the parallel orchestrator?

Yes. While the parallel orchestrator in [`backend/scripts/run_parallel_simulation.py`](https://github.com/666ghj/mirofish/blob/main/backend/scripts/run_parallel_simulation.py) provides unified management, the underlying `run_twitter_simulation` and `run_reddit_simulation` functions in [`backend/app/services/simulation_runner.py`](https://github.com/666ghj/mirofish/blob/main/backend/app/services/simulation_runner.py) are self-contained and can be imported and executed individually. The orchestrator simply adds concurrency management, unified configuration loading, and IPC capabilities on top of these core simulation runners.

### How does the system handle unexpected shutdowns or interruptions?

The architecture implements cooperative multitasking with a global `_shutdown_event` asyncio Event object. Signal handlers for `SIGTERM` and `SIGINT` (lines 53-78) set this event, which is checked at regular intervals by the main orchestration loop, the IPC polling mechanism, and the per-round simulation loops. When triggered, each component begins graceful termination, ensuring OASIS environments close properly and database transactions complete before the process exits.