# How to Debug Handoff Delivery Failures or Missing Messages in SwarmForge > Debug SwarmForge handoff delivery failures. Inspect daemon logs, check failed directories, and validate handoff headers to resolve missing messages. - Repository: [Robert C. Martin/swarm-forge](https://github.com/unclebob/swarm-forge) - Tags: how-to-guide - Published: 2026-08-08 --- **To debug handoff delivery failures in SwarmForge, inspect the daemon log at `.swarmforge/daemon/handoffd.log`, check the `failed` directory for `.error` files, and validate handoff headers using `handoff_lib.bb`.** SwarmForge orchestrates multi-agent workflows by moving work between roles via handoff files managed by a dedicated daemon. When messages vanish or delivery fails in the `unclebob/swarm-forge` repository, systematic debugging requires tracing the pipeline from the **outbox** through the **inbox** to the final tmux notification. ## Understanding the Handoff Architecture SwarmForge uses a file-based messaging system where the `handoffd.bb` daemon watches every role’s **outbox** for new `.handoff` files. The daemon validates each draft, copies it into each recipient’s **inbox → new** directory, and sends a tmux wake-up message. Delivery failures can occur during validation, filesystem operations, or the notification phase. ## Three Diagnostic Layers for Debugging Handoffs Trace failures through these three layers to isolate the root cause. ### Layer 1: Daemon Logging The daemon writes a chronological log to `.swarmforge/daemon/handoffd.log`. Look for entries prefixed with `delivered`, `error`, or `failed`. In `swarmforge/scripts/handoffd.bb`, the `log!` function writes timestamped entries to `log-file` at lines 34–38. A successful delivery logs `delivered `, while failures log specific error codes. ### Layer 2: Failure Archives When the daemon cannot deliver a handoff, it moves the draft to the sender’s `failed` directory and creates a companion `.error` file containing the reason. The `fail!` function in `handoffd.bb` (lines 17–22) records the failure reason, writes the `.error` file, and archives the handoff. Common triggers include missing headers, unknown recipients, or filesystem collisions. ### Layer 3: Handoff Content Validation Verify that the handoff file contains all required headers: `type`, `to`, `priority`, and `task` or `commit`. Use `swarmforge/scripts/handoff_lib.bb` to inspect files. The `header-field` function (lines 68–74) extracts specific headers, while `print-task` (lines 107–116) pretty-prints the full task representation for manual verification. ## Step-by-Step Debug Workflow Follow these steps to diagnose delivery failures systematically. ### 1. Check the Daemon Log View the last 20 lines of the daemon log to identify recent errors or successful deliveries. ```bash cat .swarmforge/daemon/handoffd.log | tail -n 20 ``` Look for `error ` or `failed ` entries. If you see `delivered` but the recipient lacks the file, the failure occurred during the **notify!** step. ### 2. Inspect the Failed Directory List failed handoffs and read their error descriptions. ```bash ls .swarmforge/handoffs//failed cat .swarmforge/handoffs//failed/.error ``` The `.error` file explains why `fail!` aborted delivery, such as *"missing to header"* or invalid priority values. ### 3. Validate Handoff Headers Manually Use the helper library to inspect the handoff content before it enters the outbox. ```bash # Print full task representation bb swarmforge/scripts/handoff_lib.bb print-task .swarmforge/handoffs//outbox/tmp/.handoff # Extract specific header (e.g., "to") bb swarmforge/scripts/handoff_lib.bb header-field .swarmforge/handoffs//outbox/tmp/.handoff to ``` Ensure the `to` list contains only role names defined in `roles.tsv`. ### 4. Confirm Recipient Inbox State Verify whether the file reached the destination inbox. ```bash ls .swarmforge/handoffs//inbox/new ``` If the file is missing despite a `delivered` log entry, the copy succeeded but the tmux notification likely failed. ### 5. Test Tmux Notification Manually The `notify!` function (lines 94–99) sends three tmux commands to wake the recipient session. Test these manually to detect socket or session errors. ```bash tmux -S .swarmforge/tmux-socket send-keys -t -l "You have new handoff mail. If idle, run ready_for_next.sh." tmux -S .swarmforge/tmux-socket send-keys -t C-m tmux -S .swarmforge/tmux-socket send-keys -t C-j ``` Non-zero exit statuses appear in the daemon log (lines 100–105). ### 6. Verify Role Configuration The daemon loads role metadata via `load-roles` (lines 44–56) from `.swarmforge/roles.tsv`. Typos in role names, missing worktree paths, or incorrect receive modes cause silent delivery aborts. ```bash cat .swarmforge/roles.tsv ``` ### 7. Check Sequence Handling Duplicate filenames occur when agents generate handoffs within the same second. The `next-sequence` function in `handoff_lib.bb` (lines 38–57) uses a lock directory to ensure uniqueness. If the lock fails, `move-with-collision` in `handoffd.bb` (lines 7–15) resolves conflicts by renaming files, potentially causing delivery delays. Inspect the `failed` directory for "collision" errors if you suspect timing issues. ## Common Pitfalls and Fixes | Symptom | Likely Cause | Fix | |---------|--------------|-----| | "missing to header" in `.error` file | Draft omitted the `to:` line or used incorrect capitalization. | Add `to: ` with valid role names from `roles.tsv`. | | No `delivered` entry but `error` shown | Invalid `priority` (must be two digits) or malformed `type`. | Ensure `priority: 00`–`99` and `type: git_handoff` or `type: note`. | | Recipient never sees file despite `delivered` log | Incorrect tmux session name or missing socket file. | Verify `roles.tsv` session column matches active tmux sessions; check `.swarmforge/tmux-socket` exists. | | Duplicate filename collisions | Multiple agents created handoffs simultaneously and lock directory failed. | Restart the daemon by killing `handoffd.pid` and rerunning `./swarm`. | | Handoff missing from `inbox/new` | Role configured for `batch` receive mode but agent runs [`ready_for_next.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next.sh) expecting single tasks. | Use [`ready_for_next_batch.sh`](https://github.com/unclebob/swarm-forge/blob/main/ready_for_next_batch.sh) for batch mode; see [`handoff-protocol.md`](https://github.com/unclebob/swarm-forge/blob/main/handoff-protocol.md) lines 44–58 for mode definitions. | ## Quick Diagnostic Script Run this script from the project root to automate the initial investigation. ```bash #!/usr/bin/env bash set -euo pipefail echo "=== Daemon Log (last 20 lines) ===" tail -n 20 .swarmforge/daemon/handoffd.log if ls .swarmforge/handoffs/*/failed/*.handoff 2>/dev/null | grep -q .; then echo; echo "=== Failed Handoffs ===" for f in .swarmforge/handoffs/*/failed/*.handoff; do echo "- $(basename "$f")" cat "${f}.error" echo done fi OUT=$(ls .swarmforge/handoffs/*/outbox/*.handoff 2>/dev/null | head -n 1 || true) if [[ -n "$OUT" ]]; then echo "=== Sample Outbox Handoff ===" bb swarmforge/scripts/handoff_lib.bb print-task "$OUT" fi ``` Save as [`debug_handoff.sh`](https://github.com/unclebob/swarm-forge/blob/main/debug_handoff.sh), make it executable (`chmod +x debug_handoff.sh`), and execute while the swarm is active. ## Summary - **Monitor** `.swarmforge/daemon/handoffd.log` for `delivered`, `error`, or `failed` entries generated by the `log!` function. - **Inspect** the `failed` directory and companion `.error` files created by `fail!` to understand validation failures. - **Validate** handoff headers using `handoff_lib.bb` functions `header-field` and `print-task` to ensure required fields (`to`, `type`, `priority`) are present. - **Test** tmux notifications manually if logs show delivery but recipients report missing messages. - **Verify** `roles.tsv` configuration matches active tmux sessions and correct receive modes (`task` vs `batch`). ## Frequently Asked Questions ### Why does the daemon log show "delivered" but the recipient never sees the file? This indicates the file copy succeeded but the tmux notification failed. The `notify!` function (lines 94–99 in `handoffd.bb`) sends wake-up commands to the recipient's tmux session. If the session name in `roles.tsv` does not match the actual tmux session, or if the socket file `.swarmforge/tmux-socket` is missing, the recipient never receives the alert. Verify the session name and socket path, then test the tmux commands manually. ### What causes the "missing to header" error in handoff files? The daemon's validation requires a `to:` header specifying the recipient role. If the handoff draft omits this header, uses wrong capitalization, or lists non-existent roles, the `fail!` function (lines 17–22 in `handoffd.bb`) moves the file to the `failed` directory with a descriptive `.error` file. Always include `to: ` where the role exists in `roles.tsv`. ### How do I fix duplicate filename collisions in the handoff system? When multiple agents create handoffs within the same second, the `next-sequence` function in `handoff_lib.bb` (lines 38–57) uses a lock directory to generate unique filenames. If the lock fails due to filesystem issues or stale locks, filenames collide. The `move-with-collision` handler (lines 7–15 in `handoffd.bb`) resolves this by renaming files, which may delay delivery. Restart the daemon by removing `handoffd.pid` and relaunching `./swarm` to clear stale locks. ### Where does the daemon store error logs for failed deliveries? Failed deliveries are not logged to a separate error log; instead, the daemon moves the undeliverable handoff file to `.swarmforge/handoffs//failed/` and creates a text file with the same name plus the `.error` extension. This file contains the specific reason for failure written by the `fail!` function. Check this directory when the main `handoffd.log` shows `failed` entries or when recipients report missing messages without corresponding `delivered` entries.