# How OpenLogi Manages Crash Recovery and Process Supervision: A Deep Dive into Its Self-Healing Architecture

> Discover how OpenLogi leverages macOS LaunchAgent and Rust's succession crate for robust crash recovery and process supervision. Learn about its self-healing architecture.

- Repository: [Xuan Zhang/OpenLogi](https://github.com/AprilNEA/OpenLogi)
- Tags: deep-dive
- Published: 2026-09-01

---

**OpenLogi implements a two-layer crash recovery system combining macOS LaunchAgent supervision with an in-process Rust supervisor using the `succession` crate to automatically respawn failed components and evict stale instances.**

OpenLogi's **crash recovery and process supervision** strategy is designed for resilience without user intervention. The system operates through complementary OS-level and application-level mechanisms that ensure the agent and its helper processes stay running even after unexpected termination. This article examines the actual implementation in the AprilNEA/OpenLogi repository.

## OS-Level Supervision: macOS LaunchAgent Integration

OpenLogi leverages macOS's native `launchd` system to supervise the agent binary at the operating system level. This provides the first line of defense against crashes.

### How launchd Supervision Works

The agent is registered as a **LaunchAgent**, which `launchd` monitors continuously. When the agent process crashes or exits abnormally, `launchd` automatically restarts it. The relevant orchestration code lives in [`crates/openlogi-desktop/src/services/ipc/launch.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-desktop/src/services/ipc/launch.rs), where the system invokes `launchctl kickstart` for production deployments:

```rust
let output = std::process::Command::new("/bin/launchctl")
    .args(&["kickstart", "-w", &service_path])
    .output()
    .map_err(|e| {
        warn!(error = %e, "could not run launchctl");
        LaunchError::LaunchCtlFailed
    })?;
if !output.status.success() {
    warn!("launchctl kickstart failed");
    return Err(LaunchError::LaunchCtlFailed);
}

```

This approach guarantees that a dead agent is **promptly respawned without user interaction**. The `-w` flag ensures the service remains enabled for future login sessions.

## In-Process Supervision: The Succession Crate Architecture

Within the agent itself, OpenLogi runs a lightweight supervisor for its helper processes—specifically the **Actions Ring overlay**. This second layer uses the `succession` crate, which implements a **role-based single-instance model** with automatic respawning.

### Core Concepts: Role, Run, and Supervision

The `succession` crate operates on three key abstractions:

- **Role** – Identifies a supervised component (e.g., `"overlay"` for the Actions Ring)
- **Run** – A unique token minted for each agent instance
- **Supervisor** – Monitors the child process and enforces the role contract

In [`crates/openlogi-agent/src/overlay.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/src/overlay.rs), the overlay supervisor is initialized as follows:

```rust
let mine = Run::mint();                               // unique run token
let mut supervisor = Supervisor::new(
    Role::new(directory, "overlay"), mine);           // bind role to this run
let result = std::thread::Builder::new()
    .name("openlogi-overlay-supervisor".into())
    .spawn(move || {
        let mut spawn = || Command::new(&binary)
            .env(RUN_ENV, mine.get().to_string())
            .spawn();
        loop {
            if let Err(error) = supervisor.tick(&mut spawn, &mut |event| {
                report(&event, &helper, &mut pressed_anonymous);
            }) {
                warn!(%error, "could not read the Actions Ring overlay role");
            }
        }
    });

```

The `RUN_ENV` environment variable passes the run token to the child, enabling it to recognize its originating agent.

### Automatic Respawning with Back-Off

The supervisor runs an infinite loop calling `Supervisor::tick()`, which:

1. Checks if the child process is alive
2. Respawns it with **exponential back-off** if terminated
3. Logs events through the provided callback

This ensures the Actions Ring overlay remains available unless the user explicitly quits the agent.

## Eviction Logic: Preventing Orphaned Processes

A critical challenge in process supervision is handling **stale instances**—overlays from previous agent runs that didn't clean up properly. OpenLogi solves this through deterministic eviction.

### Detecting and Evicting Stale Overlays

When the agent starts, it checks whether the overlay role is already occupied. If a prior instance holds the role, it evicts that process before claiming it. The eviction logic in [`crates/openlogi-agent/src/overlay.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/src/overlay.rs) implements a structured policy:

```rust
let Ok(Occupancy::HeldBy(record)) = Role::new(directory, "overlay").occupancy() else {
    return;
};
let outcome = eviction::evict(
    &record,
    &Policy {
        escalate_after: Some(Duration::from_millis(150)),
        deadline: Duration::from_millis(750),
        ..Policy::default()
    },
);
info!(?outcome, "asked the overlay to leave before exiting");

```

The **Policy** parameters control eviction behavior:
- `escalate_after: 150ms` – When to escalate from polite request to forceful termination
- `deadline: 750ms` – Maximum time to wait before considering eviction failed

This prevents orphaned overlays from blocking a new agent run, ensuring **clean handoff between agent instances**.

## Layered Recovery Summary

| Layer | Mechanism | Recovery Trigger | Key Source File |
|-------|-----------|------------------|---------------|
| **OS-level** | macOS LaunchAgent/`launchd` | Agent crash or exit | [`crates/openlogi-desktop/src/services/ipc/launch.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-desktop/src/services/ipc/launch.rs) |
| **In-process** | `succession::Supervisor` | Overlay termination | [`crates/openlogi-agent/src/overlay.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/src/overlay.rs) |
| **Eviction** | `succession::eviction::evict` | Stale overlay detected | [`crates/openlogi-overlay/src/session.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-overlay/src/session.rs) |

## Dependencies and Implementation Files

The supervision system relies on specific crate dependencies and source locations:

- **`succession = "0.2"`** – Declared in [`crates/openlogi-agent/Cargo.toml`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/Cargo.toml) with features `["supervision","eviction"]`
- **[`crates/openlogi-agent/src/overlay.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/src/overlay.rs)** – Overlay supervisor setup and respawn loop
- **[`crates/openlogi-overlay/src/session.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-overlay/src/session.rs)** – Role definition, claim/election, and eviction policies
- **[`crates/openlogi-desktop/src/services/ipc/launch.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-desktop/src/services/ipc/launch.rs)** – macOS agent launch via `launchctl kickstart`

## Summary

OpenLogi's **crash recovery and process supervision** architecture demonstrates production-grade resilience through:

- **OS-level guarantees** via macOS LaunchAgent integration, ensuring the agent itself restarts on failure
- **Fine-grained in-process supervision** using the `succession` crate, with role-based single-instance enforcement and automatic respawning
- **Deterministic eviction policies** that eliminate stale processes and enable clean agent restarts
- **Exponential back-off retry logic** that prevents resource exhaustion during rapid failure cycles

This layered approach keeps the Actions Ring and agent processes available without manual intervention, even under adverse conditions.

## Frequently Asked Questions

### What happens if the OpenLogi agent crashes on macOS?

The macOS `launchd` system automatically restarts it. OpenLogi registers the agent as a LaunchAgent, and the `launchctl kickstart` command in [`crates/openlogi-desktop/src/services/ipc/launch.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-desktop/src/services/ipc/launch.rs) ensures it runs under `launchd` supervision. No user action is required for recovery.

### How does OpenLogi prevent multiple overlay instances from running simultaneously?

The `succession` crate enforces a **single-instance role model**. Each agent run mints a unique `Run` token passed via `RUN_ENV`. The supervisor in [`crates/openlogi-agent/src/overlay.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-agent/src/overlay.rs) binds the overlay to this run, and the `Role` abstraction in [`crates/openlogi-overlay/src/session.rs`](https://github.com/AprilNEA/OpenLogi/blob/main/crates/openlogi-overlay/src/session.rs) prevents concurrent claims. Existing instances are evicted using `succession::eviction::evict` with a 750ms deadline policy.

### What is the `succession` crate and why did OpenLogi choose it?

`succession` is a Rust library implementing **process supervision primitives** including role-based single-instance enforcement, run token generation, and coordinated eviction. OpenLogi uses version 0.2 with `supervision` and `eviction` features to handle overlay lifecycle management without custom reimplementation. It provides exponential back-off respawning and clean handoff semantics out of the box.

### How does eviction work when restarting the OpenLogi agent?

When starting, the agent checks overlay occupancy via `Role::new(directory, "overlay").occupancy()`. If held by a previous run's record, it calls `eviction::evict()` with a policy escalating after 150ms and hard-deadlining at 750ms. This cooperatively or forcefully terminates the stale overlay before the new supervisor claims the role.