How the SwarmForge Handoff Protocol Works: A Complete Technical Guide

The SwarmForge handoff protocol uses a daemon‑backed file transport system where agents exchange work‑items through durable, auditable directory queues without directly manipulating tmux sockets.

The handoff protocol is the core coordination mechanism in unclebob/swarm-forge, enabling AI agents to collaborate asynchronously. This article explains the protocol's architecture, lifecycle, and implementation based on the source code.

Core Architecture: Directory‑Based Queue System

The protocol eliminates external dependencies by using the filesystem as its single source of truth. Each agent maintains a .swarmforge/handoffs/ hierarchy with predictable state transitions expressed through file locations.

Directory Layout and State Machine

Location	Purpose	State Meaning
outbox/	Pending outbound deliveries	Draft validated, awaiting daemon pickup
outbox/sent/	Successfully delivered	Originating agent's copy archived
outbox/failed/	Delivery errors	Permanent failure, requires manual review
inbox/new/	Unread incoming work	Available for processing
inbox/in_process/	Currently active	Agent has claimed this item
inbox/completed/	Finished work	Immutable audit trail

This design provides crash‑safe durability: any interruption can be recovered by examining file locations, with no database or log required.

Handoff File Format and Naming Convention

Every handoff follows strict conventions to ensure deterministic ordering and machine parsability.

Filename Structure

Files use a sortable, collision‑resistant format:

<priority>_<timestamp>_<seq>_from_<sender>_to_<recipients>.handoff

Example: 00_20260615T140531Z_000042_from_architect_to_coder.handoff

The components enable priority‑first, timestamp‑second ordering while encoding provenance directly in the name.

Header Block Specification

Each .handoff file begins with a structured header (YAML‑style) followed by an opaque body:

id: uuid-generated-by-swarm_handoff.sh
from: architect
to: cleaner
recipient: cleaner  # added by daemon during delivery

priority: 00
type: git_handoff
created_at: 2026-06-15T14:05:31Z
enqueued_at: 2026-06-15T14:05:33Z  # added by daemon

---
<agent-specific payload>

Reserved keys are strictly enforced; the daemon validates presence of id, from, to, priority, type, and created_at before delivery.

Permitted Message Types

The protocol restricts type to two values:

• git_handoff — Request to merge a specific commit into recipient's worktree
• note — Free‑form message with no automated processing

This constraint prevents protocol fragmentation and ensures every handoff has unambiguous semantics.

The Handoff Daemon: handoffd.bb

The Babashka‑implemented daemon (swarmforge/scripts/handoffd.bb) provides the transport layer without requiring agents to manage sockets or network connections.

Daemon Responsibilities

1. Scan each agent's outbox/ for complete .handoff files (files not open for writing)
2. Validate header integrity and required fields
3. Deliver copies to every recipient's inbox/new/ with injected recipient and enqueued_at headers
4. Notify via generic tmux message: display-message "You have new handoff mail from <sender>"
5. Archive the original to sent/ on success, or failed/ on unrecoverable error

The tmux wake‑up is intentionally generic; the daemon does not track whether recipients are online. This decouples transport from availability.

Role Receive Modes: Task vs. Batch

Agents declare processing semantics in swarmforge.conf, persisted to .swarmforge/roles.tsv. The ready_for_next.sh dispatcher reads this configuration to route execution correctly.

Mode	Behavior	Use Case
task	Process one handoff at a time, strict FIFO within priority	Sequential roles (reviewers, integrators)
batch	Atomically claim all available inbox/new/ items	Parallelizable work (test runners, document generators)

This mode is evaluated at runtime by ready_for_next.sh, allowing role reconfiguration without protocol changes.

Helper Scripts: The Agent Interface

Agents interact with the protocol through three shell scripts that enforce the audit gate and state transitions.

swarm_handoff.sh — Outbound Validation

This script implements a two‑call audit gate for git_handoff types:

# First call: create audit record, require manual review

swarm_handoff.sh ./tmp/handoff.txt

# Output: AUDIT_REQUIRED

# Second call (after review): atomically stage to outbox/

swarm_handoff.sh ./tmp/handoff.txt

# Output: <path to staged .handoff file>

The script validates headers, generates id and created_at, writes to outbox/tmp/, then renames to outbox/ for atomic visibility.

ready_for_next.sh — Inbound Dispatch

Selects and claims the next work item:

ready_for_next.sh

Output format (parseable by agent scripts):

TASK: .swarmforge/handoffs/inbox/in_process/00_20260615T140531Z_000042_from_architect_to_coder.handoff
FROM: architect
TYPE: git_handoff
PRIORITY: 00
TASK_NAME: task-1-cave-setup
PAYLOAD:
  Re-read your role and constitution.
  merge_and_process.sh architect a1b2c3d9

The script moves the selected file from inbox/new/ to inbox/in_process/ before printing, preventing double‑claiming.

done_with_current.sh — Completion and Continuation

Finalizes processing and signals availability:

done_with_current.sh

# Output: COMPLETED: <filename>

#         MAIL_WAITING  (or NO_TASK)

This records completed_at, archives to inbox/completed/, and indicates whether the agent should immediately call ready_for_next.sh again.

Complete Workflow Example

# 1. Create outbound git handoff draft

cat > ./tmp/handoff.txt <<'EOF'
type: git_handoff
to: cleaner
priority: 50
task: task-1-cave-setup
commit: a1b2c3d9e8
EOF

# 2. First validation call (audit gate)

swarm_handoff.sh ./tmp/handoff.txt

# → AUDIT_REQUIRED

# 3. Second validation call (after review) queues for daemon

swarm_handoff.sh ./tmp/handoff.txt

# 4. Recipient checks for work

ready_for_next.sh

# 5. Execute payload (example: merge_and_process.sh cleaner a1b2c3d9e8)

# 6. Mark complete and check for more

done_with_current.sh

Crash Recovery and Idempotency

The protocol guarantees at‑least‑once delivery with exactly‑once processing per agent through filesystem state:

• Sender crash: outbox/ files are picked up by daemon on restart; sent/ confirms delivery
• Daemon crash: Incomplete deliveries are re‑attempted; duplicate detection uses id header
• Recipient crash: in_process/ files are resumed by ready_for_next.sh (moves back to new/ if stale timeout exceeded, or continues processing)

No central coordinator is required for recovery.

Summary

• Filesystem as queue: All state expressed through .swarmforge/handoffs/ directory locations
• Daemon transport: handoffd.bb delivers files and triggers tmux notifications without socket manipulation
• Strict typing: Only git_handoff and note message types permitted
• Audit gate: swarm_handoff.sh enforces two‑call validation for git operations
• Role modes: task and batch receive modes adapt processing semantics per role
• Atomic operations: All state transitions use rename‑based atomicity for crash safety

Frequently Asked Questions

What makes the SwarmForge handoff protocol durable?

The protocol uses filesystem atomic operations (create‑temp‑then‑rename) and encodes all queue state in directory locations. No external database or message broker is required; recovery requires only examining files in outbox/, inbox/new/, and inbox/in_process/.

How does the protocol prevent duplicate work?

Each handoff receives a UUID in its id header during creation. The daemon uses this for deduplication when delivering to multiple recipients. Recipients atomically move files from inbox/new/ to inbox/in_process/ before processing, ensuring only one agent handles each item per worktree.

What happens if the handoff daemon crashes mid‑delivery?

The daemon validates file completeness before processing (checking that .handoff files are not open for writing). A partially copied file in a recipient's inbox/new/ will lack required daemon‑injected headers (recipient, enqueued_at), causing validation failure on pickup. The original remains in outbox/ for re‑delivery.

Can agents communicate across machines with this protocol?

The current implementation assumes shared filesystem access or same‑host execution. The protocol design is transport‑agnostic—handoffd.bb could be extended to use rsync, sftp, or object storage while preserving the same file format and state semantics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}