# How the SwarmForge Handoff Protocol Works: A Complete Technical Guide

> Master the SwarmForge handoff protocol with this technical guide. Understand how agents exchange work items via a daemon-backed file transport system for durable, auditable queues.

- Repository: [Robert C. Martin/swarm-forge](https://github.com/unclebob/swarm-forge)
- Tags: deep-dive
- Published: 2026-09-02

---

**The SwarmForge handoff protocol uses a daemon‑backed file transport system where agents exchange work‑items through durable, auditable directory queues without directly manipulating tmux sockets.**

The handoff protocol is the core coordination mechanism in [unclebob/swarm-forge](https://github.com/unclebob/swarm-forge), enabling AI agents to collaborate asynchronously. This article explains the protocol's architecture, lifecycle, and implementation based on the source code.

## Core Architecture: Directory‑Based Queue System

The protocol eliminates external dependencies by using the filesystem as its single source of truth. Each agent maintains a `.swarmforge/handoffs/` hierarchy with predictable state transitions expressed through file locations.

### Directory Layout and State Machine

| Location | Purpose | State Meaning |
|----------|---------|---------------|
| `outbox/` | Pending outbound deliveries | Draft validated, awaiting daemon pickup |
| `outbox/sent/` | Successfully delivered | Originating agent's copy archived |
| `outbox/failed/` | Delivery errors | Permanent failure, requires manual review |
| `inbox/new/` | Unread incoming work | Available for processing |
| `inbox/in_process/` | Currently active | Agent has claimed this item |
| `inbox/completed/` | Finished work | Immutable audit trail |

This design provides **crash‑safe durability**: any interruption can be recovered by examining file locations, with no database or log required.

## Handoff File Format and Naming Convention

Every handoff follows strict conventions to ensure deterministic ordering and machine parsability.

### Filename Structure

Files use a sortable, collision‑resistant format:

```

<priority>_<timestamp>_<seq>_from_<sender>_to_<recipients>.handoff

```

Example: `00_20260615T140531Z_000042_from_architect_to_coder.handoff`

The components enable **priority‑first, timestamp‑second** ordering while encoding provenance directly in the name.

### Header Block Specification

Each `.handoff` file begins with a structured header (YAML‑style) followed by an opaque body:

```

id: uuid-generated-by-swarm_handoff.sh
from: architect
to: cleaner
recipient: cleaner  # added by daemon during delivery

priority: 00
type: git_handoff
created_at: 2026-06-15T14:05:31Z
enqueued_at: 2026-06-15T14:05:33Z  # added by daemon

---
<agent-specific payload>

```

Reserved keys are strictly enforced; the daemon validates presence of `id`, `from`, `to`, `priority`, `type`, and `created_at` before delivery.

### Permitted Message Types

The protocol restricts `type` to two values:

- **`git_handoff`** — Request to merge a specific commit into recipient's worktree
- **`note`** — Free‑form message with no automated processing

This constraint prevents protocol fragmentation and ensures every handoff has unambiguous semantics.

## The Handoff Daemon: `handoffd.bb`

The Babashka‑implemented daemon (`swarmforge/scripts/handoffd.bb`) provides the transport layer without requiring agents to manage sockets or network connections.

### Daemon Responsibilities

1. **Scan** each agent's `outbox/` for complete `.handoff` files (files not open for writing)
2. **Validate** header integrity and required fields
3. **Deliver** copies to every recipient's `inbox/new/` with injected `recipient` and `enqueued_at` headers
4. **Notify** via generic tmux message: `display-message "You have new handoff mail from <sender>"`
5. **Archive** the original to `sent/` on success, or `failed/` on unrecoverable error

The tmux wake‑up is intentionally generic; the daemon does not track whether recipients are online. This decouples transport from availability.

## Role Receive Modes: Task vs. Batch

Agents declare processing semantics in [`swarmforge.conf`](https://github.com/unclebob/swarm-forge/blob/main/swarmforge.conf), persisted to `.swarmforge/roles.tsv`. The [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh) dispatcher reads this configuration to route execution correctly.

| Mode | Behavior | Use Case |
|------|----------|----------|
| **`task`** | Process one handoff at a time, strict FIFO within priority | Sequential roles (reviewers, integrators) |
| **`batch`** | Atomically claim all available `inbox/new/` items | Parallelizable work (test runners, document generators) |

This mode is evaluated at runtime by [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh), allowing role reconfiguration without protocol changes.

## Helper Scripts: The Agent Interface

Agents interact with the protocol through three shell scripts that enforce the audit gate and state transitions.

### [`swarm_handoff.sh`](https://github.com/unclebob/swarm-forge/blob/main/swarm_handoff.sh) — Outbound Validation

This script implements a **two‑call audit gate** for `git_handoff` types:

```sh

# First call: create audit record, require manual review

swarm_handoff.sh ./tmp/handoff.txt

# Output: AUDIT_REQUIRED

# Second call (after review): atomically stage to outbox/

swarm_handoff.sh ./tmp/handoff.txt

# Output: <path to staged .handoff file>

```

The script validates headers, generates `id` and `created_at`, writes to `outbox/tmp/`, then renames to `outbox/` for atomic visibility.

### [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh) — Inbound Dispatch

Selects and claims the next work item:

```sh
ready_for_next.sh

```

Output format (parseable by agent scripts):

```

TASK: .swarmforge/handoffs/inbox/in_process/00_20260615T140531Z_000042_from_architect_to_coder.handoff
FROM: architect
TYPE: git_handoff
PRIORITY: 00
TASK_NAME: task-1-cave-setup
PAYLOAD:
  Re-read your role and constitution.
  merge_and_process.sh architect a1b2c3d9

```

The script moves the selected file from `inbox/new/` to `inbox/in_process/` before printing, preventing double‑claiming.

### [`done_with_current.sh`](https://github.com/unclebob/swarm-forge/blob/main/done_with_current.sh) — Completion and Continuation

Finalizes processing and signals availability:

```sh
done_with_current.sh

# Output: COMPLETED: <filename>

#         MAIL_WAITING  (or NO_TASK)

```

This records `completed_at`, archives to `inbox/completed/`, and indicates whether the agent should immediately call [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh) again.

## Complete Workflow Example

```sh

# 1. Create outbound git handoff draft

cat > ./tmp/handoff.txt <<'EOF'
type: git_handoff
to: cleaner
priority: 50
task: task-1-cave-setup
commit: a1b2c3d9e8
EOF

# 2. First validation call (audit gate)

swarm_handoff.sh ./tmp/handoff.txt

# → AUDIT_REQUIRED

# 3. Second validation call (after review) queues for daemon

swarm_handoff.sh ./tmp/handoff.txt

# 4. Recipient checks for work

ready_for_next.sh

# 5. Execute payload (example: merge_and_process.sh cleaner a1b2c3d9e8)

# 6. Mark complete and check for more

done_with_current.sh

```

## Crash Recovery and Idempotency

The protocol guarantees **at‑least‑once delivery** with **exactly‑once processing** per agent through filesystem state:

- **Sender crash**: `outbox/` files are picked up by daemon on restart; `sent/` confirms delivery
- **Daemon crash**: Incomplete deliveries are re‑attempted; duplicate detection uses `id` header
- **Recipient crash**: `in_process/` files are resumed by [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh) (moves back to `new/` if stale timeout exceeded, or continues processing)

No central coordinator is required for recovery.

## Summary

- **Filesystem as queue**: All state expressed through `.swarmforge/handoffs/` directory locations
- **Daemon transport**: `handoffd.bb` delivers files and triggers tmux notifications without socket manipulation
- **Strict typing**: Only `git_handoff` and `note` message types permitted
- **Audit gate**: [`swarm_handoff.sh`](https://github.com/unclebob/swarm-forge/blob/main/swarm_handoff.sh) enforces two‑call validation for git operations
- **Role modes**: `task` and `batch` receive modes adapt processing semantics per role
- **Atomic operations**: All state transitions use rename‑based atomicity for crash safety

## Frequently Asked Questions

### What makes the SwarmForge handoff protocol durable?

The protocol uses filesystem atomic operations (create‑temp‑then‑rename) and encodes all queue state in directory locations. No external database or message broker is required; recovery requires only examining files in `outbox/`, `inbox/new/`, and `inbox/in_process/`.

### How does the protocol prevent duplicate work?

Each handoff receives a UUID in its `id` header during creation. The daemon uses this for deduplication when delivering to multiple recipients. Recipients atomically move files from `inbox/new/` to `inbox/in_process/` before processing, ensuring only one agent handles each item per worktree.

### What happens if the handoff daemon crashes mid‑delivery?

The daemon validates file completeness before processing (checking that `.handoff` files are not open for writing). A partially copied file in a recipient's `inbox/new/` will lack required daemon‑injected headers (`recipient`, `enqueued_at`), causing validation failure on pickup. The original remains in `outbox/` for re‑delivery.

### Can agents communicate across machines with this protocol?

The current implementation assumes shared filesystem access or same‑host execution. The protocol design is transport‑agnostic—`handoffd.bb` could be extended to use `rsync`, `sftp`, or object storage while preserving the same file format and state semantics.