# How Herdr Live Handoff Transfers Running Panes Between Server Instances

> Learn how Herdr live handoff transfers running panes between server instances by duplicating PTY file descriptors and using Unix SCM_RIGHTS sockets for seamless session replacement.

- Repository: [Can Celik/herdr](https://github.com/ogulcancelik/herdr)
- Tags: how-to-guide
- Published: 2026-05-31

---

**Herdr transfers running panes by duplicating PTY file descriptors and passing them to a newly spawned server instance via Unix SCM_RIGHTS sockets, allowing the replacement process to assume control of existing terminal sessions without interrupting executing commands.**

The **herdr live handoff** mechanism enables zero-downtime server upgrades by migrating active terminal sessions from one process to another. When triggered through the JSON API, the current server freezes pane output, serializes workspace state into a `HandoffManifest`, and transfers the underlying PTY file descriptors to a replacement binary. This ensures that **running panes** continue executing under the new instance while client sessions reconnect seamlessly.

## The 12-Step Handoff Protocol

The live handoff implemented in [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs) follows a strict coordination sequence to prevent data loss. Each step maps to specific functions in the Herdr codebase:

1. **Request Initiation** – A client sends `{"method":"server.live_handoff","params":{…}}` to the API socket, routed to `HeadlessServer::perform_live_handoff` at lines 71–73 of [`src/api/server.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/api/server.rs).

2. **Socket Preparation** – The server generates a unique Unix socket (`herdr‑handoff‑<pid>.sock`) with `0o600` permissions and a random validation token. This logic resides in [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs) at lines 51–59.

3. **PTY Stream Freezing** – Each `TerminalRuntime` calls `pause_handoff_reader` with a 2-second timeout to halt PTY data consumption, preventing output corruption during the transfer (see [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs) lines 15–24).

4. **State Snapshot** – The workspace layout, focus states, and pane configurations are captured via `crate::persist::capture` (lines 26–37 of [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs)).

5. **Runtime Metadata Collection** – For every pane, the server constructs a `HandoffRuntimeState` containing `child_pid`, dimensions (`rows`, `cols`), keyboard protocol flags, and optional scrollback history. This structure is defined in [`src/handoff_runtime.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/handoff_runtime.rs) at lines 5–22.

6. **Manifest Construction** – `handoff::manifest_for` assembles the `HandoffManifest` struct (lines 33–43 of [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs)), which includes version metadata, the snapshot, and the per-pane runtime vector.

7. **Import Server Spawn** – The old server executes a new Herdr binary with `--handoff-import <socket> <token>` arguments via `spawn_handoff_import` (lines 56–84 of [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs)). The child process runs headless and awaits the manifest.

8. **File Descriptor Duplication** – Each `TerminalRuntime` invokes `duplicate_handoff_fd()` (referenced in the loop at lines 80–88 of `perform_live_handoff`), performing a Unix `dup` on the PTY fd and storing it in a `Vec<RawFd>`.

9. **Connection Validation** – The old server accepts the import server's connection on the temporary socket, verifies the token, and transmits the JSON manifest, expecting a `"validated"` response (lines 31–49 of [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs)).

10. **FD Transfer via SCM_RIGHTS** – `send_fds_and_wait_restored` transmits the duplicated PTY fds using `SCM_RIGHTS` control messages and blocks until the import server replies `"restored"` (lines 63–71 of [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs)).

11. **Commit and Cleanup** – Upon receiving `"ready"` and `"restored"` signals, the old server deletes its public API and client sockets, invokes `preserve_for_handoff` on each runtime to maintain PTY liveness, and sends `"committed"` to the new server (lines 66–78 of [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs)).

12. **Graceful Shutdown** – The original process sets `self.shutting_down = true`, marks `should_quit = true`, and exits, leaving the new server to continue serving clients with the inherited pane processes (lines 74–80 of [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs)).

## Core Implementation Details

### The perform_live_handoff Orchestrator

The primary entry point in [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs) orchestrates the entire migration:

```rust
fn perform_live_handoff(
    &mut self,
    params: crate::api::schema::ServerLiveHandoffParams,
) -> io::Result<()> {
    self.handoff_in_progress = true;
    self.disconnect_all_clients_for_handoff();
    
    // Freeze PTY streams to prevent data races
    for terminal_id in pane_by_terminal.keys() {
        self.app.terminal_runtimes[terminal_id]
            .pause_handoff_reader(Duration::from_secs(2))?;
    }
    
    // Capture workspace and build manifest
    let snapshot = crate::persist::capture(...);
    let mut handoff_entries = Vec::new();
    for (terminal_id, runtime) in self.app.terminal_runtimes.iter() {
        handoff_entries.push((terminal_id.clone(), runtime.handoff_runtime_state(pane_id)));
    }
    let manifest = crate::server::handoff::manifest_for(...);
    
    // Spawn replacement and transfer state
    let mut import_child = crate::server::handoff::spawn_handoff_import(...)?;
    let mut fds = Vec::new();
    for (terminal_id, _) in &handoff_entries {
        fds.push(self.app.terminal_runtimes[terminal_id].duplicate_handoff_fd()?);
    }
    
    let mut stream = crate::server::handoff::accept_and_validate_on(...)?;
    crate::server::handoff::send_fds_and_wait_restored(&mut stream, &fds)?;
    // Commit and shutdown...
}

```

This method ensures atomicity: if any step fails before the commit phase, the server invokes `rollback_handoff_before_commit` to re-enable PTY readers and restore public sockets.

### Serializing Pane State with HandoffManifest

The manifest structure transferred between processes contains version compatibility checks and session metadata:

```rust
pub(crate) struct HandoffManifest {
    pub version: u32,
    pub source_version: String,
    pub source_protocol: u32,
    pub expected_version: Option<String>,
    pub expected_protocol: Option<u32>,
    pub snapshot: crate::persist::SessionSnapshot,
    pub panes: Vec<crate::handoff_runtime::HandoffRuntimeState>,
}

```

Each `HandoffRuntimeState` (defined in [`src/handoff_runtime.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/handoff_runtime.rs)) preserves critical terminal attributes:

```rust
pub(crate) struct HandoffRuntimeState {
    pub pane_id: u32,
    pub child_pid: u32,
    pub rows: u16,
    pub cols: u16,
    pub cell_width_px: u32,
    pub cell_height_px: u32,
    pub keyboard_protocol_flags: u16,
    pub keyboard_protocol_ansi: Option<String>,
    pub input_state: Option<crate::pane::InputState>,
    pub initial_history_ansi: Option<String>,
}

```

### Transferring PTY File Descriptors via SCM_RIGHTS

The actual **transfer running panes** mechanism relies on Unix domain socket ancillary data. The `send_fds` function (lines 69–100 of [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs)) packs duplicated file descriptors into a control message using `SCM_RIGHTS`, while the import server uses `recv_fds` (lines 104–149) to receive them. This kernel-assisted handoff ensures that the new process possesses valid file descriptors pointing to the same underlying pseudo-terminals as the original server.

## Practical Usage

### CLI Trigger

Execute a live handoff from the command line:

```bash
herdr server live-handoff

```

This forwards the JSON RPC request to the running server, which then executes the 12-step protocol. Once complete, clients can reconnect via `herdr attach` to resume sessions exactly where they left off.

### Direct JSON API

For scripted automation, send a line-terminated JSON request to the API socket (`herdr.sock`):

```json
{
  "id": "handoff-1",
  "method": "server.live_handoff",
  "params": {
    "import_exe": "/usr/local/bin/herdr",
    "expected_version": "0.9.3",
    "expected_protocol": 9
  }
}

```

The server validates the optional `expected_version` and `expected_protocol` fields against the spawned binary before proceeding, preventing version mismatches during the handoff.

## Error Handling and Rollback Safety

If any phase fails before the commit signal, `rollback_handoff_before_commit` (lines 82–90 of [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs)) executes automatically:

- **Re-enables PTY readers** via the `TerminalRuntime` pause mechanism
- **Clears** the `handoff_in_progress` flag
- **Restores** the original public sockets so existing clients maintain connectivity

This design guarantees that **running panes** never terminate unexpectedly; they either migrate successfully to the new server or continue under the old server if the transfer aborts.

## Summary

- **Herdr live handoff** migrates active terminal sessions between server processes using a 12-step Unix socket protocol.
- **PTY file descriptors** are duplicated and transferred via `SCM_RIGHTS`, allowing the new server to control existing processes without interruption.
- **`HandoffManifest`** and **`HandoffRuntimeState`** serialize layout, dimensions, and process metadata from [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs) and [`src/handoff_runtime.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/handoff_runtime.rs).
- **`perform_live_handoff`** in [`src/server/headless.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/headless.rs) orchestrates the transfer with automatic rollback on failure.
- Clients reconnect after handoff completion to find panes in identical states, including scrollback history and keyboard protocols.

## Frequently Asked Questions

### What happens to running processes during a herdr live handoff?

Running processes continue executing uninterrupted because the **herdr live handoff** mechanism transfers only the controlling PTY file descriptors to the new server process. The underlying child processes (identified by `child_pid` in `HandoffRuntimeState`) remain the same operating system entities; only the server process managing them changes. The original server calls `preserve_for_handoff` to ensure PTYs remain valid in the new process before shutting down.

### How does Herdr ensure file descriptors are transferred securely?

Herdr employs a **token-validated Unix socket** with `0o600` permissions created specifically for the handoff transaction. Before transferring file descriptors via `SCM_RIGHTS`, the old server verifies that the connecting import server presents the correct random token generated during socket preparation. This prevents unauthorized processes from intercepting the PTY fds during the `accept_and_validate_on` phase in [`src/server/handoff.rs`](https://github.com/ogulcancelik/herdr/blob/main/src/server/handoff.rs).

### What causes a handoff to rollback instead of commit?

A handoff rolls back if **any step fails** before the commit phase, including: socket creation errors, PTY duplication failures, manifest validation mismatches, or timeout during the `send_fds_and_wait_restored` call. When triggered, `rollback_handoff_before_commit` re-enables all PTY readers and restores the original server's public sockets, ensuring clients experience only a brief pause rather than a session termination.

### Can clients reconnect during a live handoff?

Clients must disconnect during the handoff because `disconnect_all_clients_for_handoff` drops existing connections to prevent state synchronization conflicts. However, once the new server signals `"committed"` and the old process removes its sockets, clients can immediately reconnect via `herdr attach` or new TCP/Unix socket connections to the new server instance, resuming their **running panes** without data loss.