How OpenLogi Manages Crash Recovery and Process Supervision: A Deep Dive into Its Self-Healing Architecture

OpenLogi implements a two-layer crash recovery system combining macOS LaunchAgent supervision with an in-process Rust supervisor using the succession crate to automatically respawn failed components and evict stale instances.

OpenLogi's crash recovery and process supervision strategy is designed for resilience without user intervention. The system operates through complementary OS-level and application-level mechanisms that ensure the agent and its helper processes stay running even after unexpected termination. This article examines the actual implementation in the AprilNEA/OpenLogi repository.

OS-Level Supervision: macOS LaunchAgent Integration

OpenLogi leverages macOS's native launchd system to supervise the agent binary at the operating system level. This provides the first line of defense against crashes.

How launchd Supervision Works

The agent is registered as a LaunchAgent, which launchd monitors continuously. When the agent process crashes or exits abnormally, launchd automatically restarts it. The relevant orchestration code lives in crates/openlogi-desktop/src/services/ipc/launch.rs, where the system invokes launchctl kickstart for production deployments:

let output = std::process::Command::new("/bin/launchctl")
    .args(&["kickstart", "-w", &service_path])
    .output()
    .map_err(|e| {
        warn!(error = %e, "could not run launchctl");
        LaunchError::LaunchCtlFailed
    })?;
if !output.status.success() {
    warn!("launchctl kickstart failed");
    return Err(LaunchError::LaunchCtlFailed);
}

This approach guarantees that a dead agent is promptly respawned without user interaction. The -w flag ensures the service remains enabled for future login sessions.

In-Process Supervision: The Succession Crate Architecture

Within the agent itself, OpenLogi runs a lightweight supervisor for its helper processes—specifically the Actions Ring overlay. This second layer uses the succession crate, which implements a role-based single-instance model with automatic respawning.

Core Concepts: Role, Run, and Supervision

The succession crate operates on three key abstractions:

• Role – Identifies a supervised component (e.g., "overlay" for the Actions Ring)
• Run – A unique token minted for each agent instance
• Supervisor – Monitors the child process and enforces the role contract

In crates/openlogi-agent/src/overlay.rs, the overlay supervisor is initialized as follows:

let mine = Run::mint();                               // unique run token
let mut supervisor = Supervisor::new(
    Role::new(directory, "overlay"), mine);           // bind role to this run
let result = std::thread::Builder::new()
    .name("openlogi-overlay-supervisor".into())
    .spawn(move || {
        let mut spawn = || Command::new(&binary)
            .env(RUN_ENV, mine.get().to_string())
            .spawn();
        loop {
            if let Err(error) = supervisor.tick(&mut spawn, &mut |event| {
                report(&event, &helper, &mut pressed_anonymous);
            }) {
                warn!(%error, "could not read the Actions Ring overlay role");
            }
        }
    });

The RUN_ENV environment variable passes the run token to the child, enabling it to recognize its originating agent.

Automatic Respawning with Back-Off

The supervisor runs an infinite loop calling Supervisor::tick(), which:

1. Checks if the child process is alive
2. Respawns it with exponential back-off if terminated
3. Logs events through the provided callback

This ensures the Actions Ring overlay remains available unless the user explicitly quits the agent.

Eviction Logic: Preventing Orphaned Processes

A critical challenge in process supervision is handling stale instances—overlays from previous agent runs that didn't clean up properly. OpenLogi solves this through deterministic eviction.

Detecting and Evicting Stale Overlays

When the agent starts, it checks whether the overlay role is already occupied. If a prior instance holds the role, it evicts that process before claiming it. The eviction logic in crates/openlogi-agent/src/overlay.rs implements a structured policy:

let Ok(Occupancy::HeldBy(record)) = Role::new(directory, "overlay").occupancy() else {
    return;
};
let outcome = eviction::evict(
    &record,
    &Policy {
        escalate_after: Some(Duration::from_millis(150)),
        deadline: Duration::from_millis(750),
        ..Policy::default()
    },
);
info!(?outcome, "asked the overlay to leave before exiting");

The Policy parameters control eviction behavior:

• escalate_after: 150ms – When to escalate from polite request to forceful termination
• deadline: 750ms – Maximum time to wait before considering eviction failed

This prevents orphaned overlays from blocking a new agent run, ensuring clean handoff between agent instances.

Layered Recovery Summary

Layer	Mechanism	Recovery Trigger	Key Source File
OS-level	macOS LaunchAgent/launchd	Agent crash or exit	crates/openlogi-desktop/src/services/ipc/launch.rs
In-process	succession::Supervisor	Overlay termination	crates/openlogi-agent/src/overlay.rs
Eviction	succession::eviction::evict	Stale overlay detected	crates/openlogi-overlay/src/session.rs

Dependencies and Implementation Files

The supervision system relies on specific crate dependencies and source locations:

• succession = "0.2" – Declared in crates/openlogi-agent/Cargo.toml with features ["supervision","eviction"]
• crates/openlogi-agent/src/overlay.rs – Overlay supervisor setup and respawn loop
• crates/openlogi-overlay/src/session.rs – Role definition, claim/election, and eviction policies
• crates/openlogi-desktop/src/services/ipc/launch.rs – macOS agent launch via launchctl kickstart

Summary

OpenLogi's crash recovery and process supervision architecture demonstrates production-grade resilience through:

• OS-level guarantees via macOS LaunchAgent integration, ensuring the agent itself restarts on failure
• Fine-grained in-process supervision using the succession crate, with role-based single-instance enforcement and automatic respawning
• Deterministic eviction policies that eliminate stale processes and enable clean agent restarts
• Exponential back-off retry logic that prevents resource exhaustion during rapid failure cycles

This layered approach keeps the Actions Ring and agent processes available without manual intervention, even under adverse conditions.

Frequently Asked Questions

What happens if the OpenLogi agent crashes on macOS?

The macOS launchd system automatically restarts it. OpenLogi registers the agent as a LaunchAgent, and the launchctl kickstart command in crates/openlogi-desktop/src/services/ipc/launch.rs ensures it runs under launchd supervision. No user action is required for recovery.

How does OpenLogi prevent multiple overlay instances from running simultaneously?

The succession crate enforces a single-instance role model. Each agent run mints a unique Run token passed via RUN_ENV. The supervisor in crates/openlogi-agent/src/overlay.rs binds the overlay to this run, and the Role abstraction in crates/openlogi-overlay/src/session.rs prevents concurrent claims. Existing instances are evicted using succession::eviction::evict with a 750ms deadline policy.

What is the succession crate and why did OpenLogi choose it?

succession is a Rust library implementing process supervision primitives including role-based single-instance enforcement, run token generation, and coordinated eviction. OpenLogi uses version 0.2 with supervision and eviction features to handle overlay lifecycle management without custom reimplementation. It provides exponential back-off respawning and clean handoff semantics out of the box.

How does eviction work when restarting the OpenLogi agent?

When starting, the agent checks overlay occupancy via Role::new(directory, "overlay").occupancy(). If held by a previous run's record, it calls eviction::evict() with a policy escalating after 150ms and hard-deadlining at 750ms. This cooperatively or forcefully terminates the stale overlay before the new supervisor claims the role.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}