SwarmForge Handoff File Lifecycle and Audit Trail Headers: Complete Technical Guide

The SwarmForge handoff system uses immutable .handoff files with structured headers to track work through every state transition, eliminating traditional logbooks while maintaining full auditability.

The unclebob/swarm-forge repository implements a durable, file-based protocol for moving tasks between agent roles. Understanding the handoff file lifecycle and the audit trail headers that drive it is essential for debugging routing issues, extending the protocol, or operating swarm workflows at scale.

Handoff Directory Structure and Flow

SwarmForge organizes handoffs in a strict directory tree under each work-tree's .swarmforge/handoffs/:

.swarmforge/handoffs/
 ├─ outbox/
 │   ├─ tmp/       ← Incomplete writes (ignored by daemon)
 │   └─ *.handoff  ← Validated, ready for delivery
 ├─ sent/          ← Successfully delivered originals
 ├─ failed/        ← Invalid or undeliverable files
 └─ inbox/
     ├─ new/       ← Pending pickup by recipient
     ├─ in_process/← Currently being worked
     └─ completed/ ← Finished tasks

The daemon—implemented in handoffd.bb—orchestrates movement between these states. According to swarmforge/handoff-protocol.md lines 40-50, handoffd validates each completed .handoff, copies it to every recipient's inbox/new/, injects delivery metadata, wakes the recipient via tmux, then archives the original to sent/.

Handoff File Format Specification

Every handoff file follows a three-part structure defined in swarmforge/handoff-protocol.md lines 91-99:

1. Header block — key: value pairs, one per line
2. Blank line — Separator
3. Body — Opaque payload (system-generated, never edited)

This format enables both human inspection and deterministic parsing by downstream tools.

Complete Audit Trail Header Reference

Headers carry the full provenance of a handoff. Ownership is explicitly assigned per the protocol specification:

Header	Purpose	Written By
id	<timestamp>_<sequence>_from_<sender> — globally unique audit key	swarm_handoff.sh (lines 96-103 of protocol)
from	Sender role identifier	swarm_handoff.sh
to	Recipient list (space or comma separated)	swarm_handoff.sh
recipient	Specific target of this copy (populated during routing)	handoffd
priority	Two-digit sort key (00–99)	swarm_handoff.sh
type	git_handoff or note	swarm_handoff.sh
role	Duplicate of from for convenience	swarm_handoff.sh
task	Stable, human-readable task name	swarm_handoff.sh
commit	Canonical 10-character git SHA	swarm_handoff.sh
created_at	Timestamp when draft was accepted	swarm_handoff.sh
enqueued_at	Timestamp when copy entered recipient inbox	handoffd
dequeued_at	Timestamp when work began	ready_for_next_task.sh or ready_for_next_batch.sh
completed_at	Timestamp when work finished	done_with_current_task.sh or done_with_current_batch.sh

The protocol explicitly maps header ownership to prevent update conflicts—no two components write the same header.

Header Lifecycle: Creation Through Completion

Creation Phase

swarm_handoff.sh validates the draft, generates id and created_at, populates core headers, then performs an atomic write to outbox/ (protocol lines 36-45). Files in outbox/tmp/ are ignored until the rename completes.

Delivery Phase

handoffd copies the file to each recipient's inbox/new/, appending recipient and enqueued_at headers per protocol lines 81-88. The original then moves to sent/.

Acceptance Phase

When a recipient claims work, ready_for_next_task.sh (or *_batch.sh) relocates the file to inbox/in_process/ and writes dequeued_at (protocol lines 95-100).

Completion Phase

done_with_current_* scripts add completed_at and archive to inbox/completed/ (protocol lines 120-128).

This yields the terminal state sequence: draft → validated → queued → delivered → accepted → completed.

Programmatic Header Manipulation with handoff_lib.bb

The swarmforge/scripts/handoff_lib.bb library provides atomic header operations for scripts and debugging:

;; Read a header value
(header-field file "dequeued_at")

;; Atomically update a header (write-then-rename)
(set-header! file "dequeued_at" (timestamp))

Implementation spans lines 91-124, using temporary files to ensure crash safety. These utilities power all state-transition scripts.

Practical Workflow Examples

Drafting and Submitting a Handoff

# 1. Create a draft handoff

cat > /tmp/handoff.txt <<'EOF'
type: git_handoff
to: cleaner
priority: 50
task: build-frontend
commit: a1b2c3d9e8
EOF

# 2. Validate and queue (generates id, created_at, etc.)

swarm_handoff.sh /tmp/handoff.txt

# 3. Inspect generated headers

head -n 12 .swarmforge/handoffs/outbox/50_20260701T123456Z_000001_from_coder_to_cleaner.handoff

Inspecting Headers During Debugging

# Read creation timestamp

bb -e '(require '\''handoff-lib) (println (header-field "path/to/file.handoff" "created_at"))'

Manual Header Updates (Testing Only)

# Force a dequeue timestamp (normally automated)

bb -e '(require '\''handoff-lib) (set-header! "path/to/file.handoff" "dequeued_at" (timestamp))'

Key Implementation Files

File	Purpose	Critical Lines
swarmforge/handoff-protocol.md	Canonical lifecycle and header specification	21-33 (header list), 35-44 (ownership)
swarmforge/scripts/handoff_lib.bb	Header read/write utilities	91-124 (header-field, set-header!)
swarmforge/scripts/swarm_handoff.bb	Outbound validation and initial handoff creation	Protocol "Creation" section
swarmforge/scripts/ready_for_next_task.bb / *_batch.bb	Accept work, write dequeued_at	Protocol "ready_for_next_task.sh"
swarmforge/scripts/done_with_current_task.bb / *_batch.bb	Finalize work, write completed_at	Protocol "done_with_current_task.sh"
swarmforge/scripts/handoffd.bb	Route files, write recipient/enqueued_at	Protocol "Handoff Daemon" section

Summary

• Immutable files replace mutable logbooks—each handoff carries its own audit trail
• Atomic operations (write-then-rename) prevent partial writes at every state transition
• Explicit header ownership eliminates write conflicts between daemon, scripts, and operators
• Deterministic pipeline: draft → validated → queued → delivered → accepted → completed
• Clojure/Babashka utilities in handoff_lib.bb provide safe, scriptable header access

Frequently Asked Questions

How does SwarmForge ensure handoff files are never corrupted mid-write?

SwarmForge uses atomic file operations at every stage. Scripts write to temporary paths (typically under outbox/tmp/) then rename into place. The set-header! function in handoff_lib.bb (lines 91-124) implements this pattern explicitly: it writes a new temporary file, then performs an atomic rename to overwrite the target. The daemon ignores any file not ending in .handoff.

Can I manually edit headers without breaking the audit trail?

You can safely read headers at any time. For writes, use only the provided utilities—set-header! from handoff_lib.bb—which handle atomicity. Direct in-place edits risk corrupting the file or creating parseable but inconsistent state. The protocol assigns each header to exactly one writer; respect these ownership rules to maintain audit validity.

What happens if the handoffd daemon crashes during delivery?

The daemon's design is crash-recoverable. It only moves an original to sent/ after successfully copying to all recipient inboxes. A crash between copy operations leaves the original in outbox/; restart picks up from there. Duplicate enqueued_at headers in a recipient's inbox indicate redelivery and can be detected by matching id fields.

How do I trace a handoff's full history across multiple machines?

The id header provides global uniqueness via <timestamp>_<sequence>_from_<sender>. Combined with created_at, enqueued_at, dequeued_at, and completed_at timestamps, you can reconstruct the complete timeline. Each machine's .swarmforge/handoffs/ tree preserves its local view; correlating by id merges distributed traces.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}