How Shadowbroker Manages Inter-Service Communication: FastAPI Gateway and Dynamic Mesh Transport

Shadowbroker coordinates inter-service communication through a centralized FastAPI application running under uvicorn, where internal components exchange JSON messages over HTTP while discovering transport configuration and health status via a shared file-based registry that supports dynamic reloading for Reticulum mesh networks.

The Shadowbroker backend (available at BigBodyCobain/Shadowbroker) implements a lightweight, decoupled architecture designed to simplify how distributed components interact. Rather than relying on complex message brokers or proprietary protocols, the system exposes a single ASGI entry point that standardizes all internal communication through HTTP requests, while maintaining the flexibility to route traffic over decentralized Reticulum mesh networks when operating in isolated environments.

FastAPI as the Central Communication Hub

At the core of the architecture is a FastAPI application defined in the top-level main.py module and launched via backend/wormhole_server.py. This process creates the singular HTTP entry point that every other service in the ecosystem targets.

When the server initializes, it executes uvicorn.run("main:app", host=HOST, port=PORT, ...) to expose the ASGI application. All inter-service calls—from the desktop-shell UI to background mesh-node scripts—are standardized as HTTP POST or GET requests to specific routes defined in main.py. This design ensures that services remain loosely coupled, requiring only the base URL (http://HOST:PORT) and the JSON contract of each endpoint to communicate.

The FastAPI routes handle payload validation and then forward requests to internal helpers or the Reticulum mesh layer, returning JSON responses to callers. This abstraction allows downstream consumers to interact with complex mesh networking logic without implementing Reticulum-specific code themselves.

Transport Abstraction and Reticulum Mesh Integration

The backend supports dual transport modes controlled by the MESH_ONLY environment variable. When MESH_ONLY=true, the system operates purely on a Reticulum mesh network (referred to as the "wormhole" mesh), bypassing standard internet routes.

Before the server starts, backend/wormhole_server.py detects the transport configuration:

• Standard mode: Uses direct HTTP over TCP/IP
• Mesh-only mode: Routes all traffic through the Reticulum library for decentralized communication

The selected transport (TRANSPORT_ACTIVE) and any SOCKS5 proxy configuration (PROXY_ACTIVE) for hidden transports (Tor/I2P) are persisted to a status file. This allows services to discover the active transport strategy at runtime by reading wormhole_status.json rather than maintaining hardcoded configuration.

Service Discovery via the Wormhole Status File

Inter-service coordination relies on a file-based discovery mechanism rather than service registries like Consul or etcd. The write_wormhole_status function in backend/wormhole_server.py generates a JSON status payload every time the server starts or transport settings change.

This status file contains:

• Current transport mode (transport_active)
• Proxy configuration (proxy_active)
• Process ID (pid)
• Health metrics and restart flags

Downstream micro-services—including the desktop-shell UI and mesh-node scripts—read this file to discover how to route messages. For example, a companion service might check /tmp/wormhole_status.json to determine whether to send requests directly to the FastAPI HTTP endpoint or to package them for the Reticulum mesh.

Dynamic Configuration Reloading

To support zero-downtime transport switching, the backend implements a background watcher thread (_watch_transport_settings) that monitors wormhole.json for configuration changes.

When the watcher detects a transport update:

1. It writes a new status entry with reason="transport_change"
2. Updates environment variables with the new transport and proxy settings
3. Triggers a process restart using os.execv(sys.executable, [sys.executable, __file__])

This mechanism ensures that all services instantly pick up new routing configurations without manual intervention. The restart is atomic and fast because the server state is reconstructed from the updated status file on launch.

Practical Implementation Examples

Starting the Backend Gateway

The following code from backend/wormhole_server.py initializes the FastAPI application and starts the transport watcher:

# backend/wormhole_server.py

if __name__ == "__main__":
    threading.Thread(target=_watch_transport_settings, daemon=True).start()
    uvicorn.run(
        "main:app",               # FastAPI app entry point

        host=HOST,
        port=PORT,
        reload=RELOAD,
        log_level="info",
    )

Updating Transport and Notifying Services

When transport settings change, the system writes status and restarts to apply the new configuration:

# backend/wormhole_server.py (inside _watch_transport_settings)

if new_transport.lower() != TRANSPORT.lower() or new_proxy != SOCKS_PROXY:
    write_wormhole_status(
        reason="transport_change",
        transport=new_transport,
        proxy=new_proxy,
        transport_active="",
        proxy_active="",
        restart=True,
    )
    os.execv(sys.executable, [sys.executable, __file__])   # Restart with new env

Reading Status from Consumer Services

Any component can discover the current communication parameters by reading the status file:

# Example consumer (desktop-shell, mesh-node scripts, etc.)

import json
import pathlib

status_path = pathlib.Path("/tmp/wormhole_status.json")
status = json.loads(status_path.read_text())
print(status["transport_active"], status["proxy_active"])

Calling Backend Endpoints from Services

Services communicate by making standard HTTP requests to the FastAPI routes:

import requests

BASE = f"http://{HOST}:{PORT}"
resp = requests.post(f"{BASE}/api/v1/geocode", json={"query": "KJFK"})
data = resp.json()
print(data)   # JSON result forwarded by the FastAPI route

Summary

• Centralized Gateway: All inter-service communication flows through a single FastAPI application exposed via uvicorn in backend/wormhole_server.py, standardizing interactions as HTTP JSON requests.
• File-Based Discovery: Services discover transport configuration and health status by reading wormhole_status.json, written by the write_wormhole_status function whenever the server starts or configuration changes.
• Dynamic Mesh Support: The backend supports both standard HTTP and Reticulum mesh transports, switched via the MESH_ONLY environment variable and reloaded at runtime through the _watch_transport_settings thread.
• Decoupled Architecture: Consumer services (desktop-shell, mesh-node scripts) require only the base URL and JSON contract, remaining agnostic to underlying transport implementation details.

Frequently Asked Questions

How do services discover the Shadowbroker backend address?

Services discover the backend by reading the JSON status file generated at startup. The write_wormhole_status function in backend/wormhole_server.py persists the active transport, proxy settings, and PID to a shared location (typically /tmp/wormhole_status.json), which client components poll to determine the current endpoint configuration and health state.

What happens when the transport mode changes at runtime?

When configuration changes are detected in wormhole.json, the _watch_transport_settings background thread triggers an atomic restart using os.execv. This reloads the process with updated environment variables (TRANSPORT_ACTIVE, PROXY_ACTIVE) and broadcasts the change via the status file, ensuring all connected services switch to the new transport mode without manual reconfiguration.

Can services communicate without internet access?

Yes. When MESH_ONLY is set to true, the backend operates exclusively over the Reticulum mesh network ("wormhole"), allowing services to communicate via decentralized packet routing. The FastAPI HTTP interface remains available locally, but underlying traffic flows through the mesh rather than traditional internet routes, enabling offline operation.

Where are the transport settings configured and loaded?

Transport settings are defined in wormhole.json and loaded by services/wormhole_settings.py. The backend/wormhole_server.py module reads these values at startup and monitors the file for changes, while services/wormhole_status.py handles persisting the runtime state for other components to consume.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}