Swarm-Forge Handoffs Directory Layout and Agent Inbox State Management Explained

The .swarmforge/handoffs/ directory uses a state-machine folder structure with eight subdirectories—new, inbox/in_process, inbox/completed, outbox, outbox/tmp, sent, failed, and pending_approval—that agents manipulate atomically via handoff_lib.bb commands to track work item lifecycles.

The Swarm-Forge runtime, created by Robert C. Martin (unclebob), implements a file-based orchestration system where autonomous agents coordinate through hand-off files. Understanding the .swarmforge/handoffs/ directory layout and how agents interact with inbox states is essential for building reliable agent workflows and debugging production issues.

Directory Layout of .swarmforge/handoffs/

Root Structure

The Swarm-Forge bootstrap process automatically creates a hidden .swarmforge folder in the project root. Within it, the handoffs/ directory serves as the central coordination hub. As implemented in test/swarmforge/script_test.clj (line 1059), the structure is initialized with:

(fs/create-dirs (fs/path root ".swarmforge/handoffs/inbox/new"))

State Subdirectories

Each subfolder represents a discrete state in the hand-off lifecycle:

Subdirectory	Purpose
handoffs/new/	Fresh hand-off files awaiting agent pickup
handoffs/inbox/in_process/	Files currently being processed by an agent
handoffs/inbox/completed/	Successfully processed hand-offs
handoffs/outbox/	Agent-generated results for downstream consumption
handoffs/outbox/tmp/	Staging area for partial or intermediate results
handoffs/sent/	Hand-offs handed off to the next executor
handoffs/failed/	Items with unrecoverable errors
handoffs/pending_approval/	Items blocked for human or role-based approval
handoffs/audit_pending/	Items queued for audit before final acceptance

This layout enables observable state transitions—any operator can inspect the filesystem to determine exactly where a work item stands in the pipeline.

Agent Inbox State Interaction Flow

Step 1: Discovery

Agents scan handoffs/new/ for files ending in .handoff. The handoffd.bb daemon (located at swarmforge/scripts/handoffd.bb) automates this polling loop, launching agent processes when new items appear.

Step 2: Claiming (Atomic Move)

To prevent race conditions, agents claim work by atomically moving the file to inbox/in_process/. The handoff_lib.bb utility stamps a dequeued_at header:

handoff_lib.bb set-header .swarmforge/handoffs/inbox/new/item.handoff dequeued_at "2026-06-16T00:00:00Z"

This pattern appears in test/swarmforge/script_test.clj (line 79), validating that the move-and-stamp operation is transactionally safe.

Step 3: Processing

The agent reads the hand-off file, extracts parameters via header-field, performs its domain task, and writes outputs. Intermediate results go to outbox/tmp/; final results to outbox/.

Step 4: Completion

Successful processing triggers two operations:

1. Set completed_at header
2. Move file to inbox/completed/

Optionally, the agent posts a result hand-off to outbox/ for the next pipeline stage.

Step 5: Error Handling

On unrecoverable exceptions, the agent:

1. Sets failed_at header with timestamp
2. Moves file to handoffs/failed/

This preserves forensic state for debugging without blocking the main workflow.

Core Operations in handoff_lib.bb

The swarmforge/scripts/handoff_lib.bb library abstracts all filesystem interactions. Understanding these commands is critical for custom agent development:

Command	Signature	Effect
set-header	<path> <key> <value>	Writes key:value to hand-off file (creates if missing)
header-field	<path> <key>	Retrieves value for given header key
move	<src> <dst>	Atomically relocates hand-off between state folders
list	<folder>	Returns filenames of .handoff files in directory

The test suite in test/swarmforge/handoff_test.clj (line 913) exercises these operations:

(let [queued (handoff-names (fs/path root ".swarmforge/handoffs/outbox"))]
  (slurp (str (fs/path root ".swarmforge/handoffs/outbox" (first queued)))))

Practical Agent Implementation Pattern

Below is a production-ready agent loop demonstrating proper state management:

;; Minimal agent loop following Swarm-Forge conventions
(let [inbox (fs/path "." ".swarmforge/handoffs/inbox/new")]
  (while true
    (doseq [file (fs/list-dir inbox)]
      (let [path (str (fs/path inbox file))]
        ;; Claim: atomic move to in_process
        (run (script "handoff_lib.bb") "move" path
             ".swarmforge/handoffs/inbox/in_process")
        (run (script "handoff_lib.bb") "set-header" path "dequeued_at"
             (java.time.Instant/now))

        ;; ----- Agent-specific work -----
        (process-handoff path)

        ;; Complete: timestamp and archive
        (run (script "handoff_lib.bb") "set-header" path "completed_at"
             (java.time.Instant/now))
        (run (script "handoff_lib.bb") "move" path
             ".swarmforge/handoffs/inbox/completed"))))

The run helper invokes Bash-BB scripts; this pattern ensures all agents use consistent state transition logic.

Key Source Files Reference

File	Role
swarmforge/scripts/handoff_lib.bb	Core library for header manipulation and atomic moves
swarmforge/scripts/handoffd.bb	Daemon process that watches new/ and launches agents
swarmforge/handoff-protocol.md	Normative specification of hand-off lifecycle
test/swarmforge/handoff_test.clj	Unit tests validating handoff_lib.bb operations
test/swarmforge/script_test.clj	Integration tests for directory bootstrap and full workflows

Summary

• The .swarmforge/handoffs/ directory implements a visible, file-based state machine with eight distinct folders tracking hand-off lifecycles from creation through completion or failure.
• Agents interact with inbox states exclusively through handoff_lib.bb commands, ensuring atomic moves and consistent header metadata.
• All state transitions are auditable—timestamps via dequeued_at, completed_at, and failed_at headers enable operational observability.
• The inbox/in_process/ folder serves as a distributed lock mechanism, preventing duplicate processing without external coordination services.
• Understanding handoff_lib.bb operations (set-header, header-field, move, list) is prerequisite for implementing custom Swarm-Forge agents.

Frequently Asked Questions

How does Swarm-Forge prevent two agents from processing the same hand-off file?

Atomic filesystem moves provide the coordination mechanism. When an agent claims work, it executes handoff_lib.bb move from new/ to inbox/in_process/. On POSIX-compliant systems, mv is atomic within the same filesystem, ensuring only one agent succeeds. Failed moves indicate another agent claimed the item first, prompting the scanner to continue to the next file.

What happens if an agent crashes while processing a hand-off?

The hand-off remains in inbox/in_process/ indefinitely. According to the Swarm-Forge protocol, operators or monitoring agents must implement heartbeats or timeouts to detect stalled items. The dequeued_at timestamp enables timeout detection: if now - dequeued_at > threshold, another agent may forcibly reclaim the item or alert operators. No automatic timeout recovery is implemented in the core runtime.

Can agents create hand-off files directly in outbox/ without going through new/?

Yes, but this bypasses the intended workflow. Agents are free to write to outbox/ or outbox/tmp/ at any time—the directory permissions allow this. However, downstream consumers typically poll specific folders. Direct outbox/ writes are appropriate for agent-to-agent communication, while new/ is reserved for external entry points or scheduler-initiated work. The handoff-protocol.md specification recommends using set-header on all created files to maintain audit trails.

Where is the .swarmforge folder created, and can its location be customized?

The folder is created in the project root where the Swarm-Forge runtime is initialized. As shown in test/swarmforge/script_test.clj, the bootstrap accepts a root parameter, allowing tests to use temporary directories. For production deployments, the runtime assumes the current working directory contains the .swarmforge/ hierarchy. No environment variable override for the base path exists in the current implementation—agents rely on relative paths from the project root.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}