How Headroom Learns from Failed Sessions and Writes Corrections

Headroom learns from failed sessions by capturing tool errors via the TrafficLearner component, pairing them with subsequent successful retries using heuristic matching, and converting these pairs into persistent error-recovery rules written to agent-specific knowledge files like CLAUDE.md.

The Headroom proxy (chopratejas/headroom) implements a continuous learning loop that transforms failed tool executions into actionable corrections. Through the TrafficLearner class in headroom/memory/traffic_learner.py, the system observes every tool call passing through the proxy, identifies when a failed command is followed by a successful variant, and automatically documents these patterns for future use. This process enables coding agents to progressively improve their error handling based on real-world usage patterns.

The TrafficLearner Architecture

Headroom’s learning mechanism centers on the TrafficLearner class, which maintains a bounded history of recent tool calls and implements heuristic matching to detect when a successful call represents a correction of a previous failure. The architecture follows a six-stage pipeline: Capture, Detection, Pattern Building, Evidence Accumulation, Persistence, and File Flushing.

Each stage is implemented through specific async methods that process tool results as they flow through the proxy. The learner supports configurable thresholds and works with the LocalBackend SQLite storage system to maintain durable records of learned patterns.

Capturing Tool Results and Detecting Errors

The learning process begins when on_tool_result receives metadata about any tool execution. Located at lines 86-99 in headroom/memory/traffic_learner.py, this method stores calls in _tool_history (limited to the last max_history entries) and triggers error-recovery detection whenever a call succeeds.

When a successful call arrives, _extract_error_recovery (lines 80-89) scans backwards through up to five recent entries to find a matching failure. For Bash commands, the system uses _commands_related_as_retry to verify that both calls share the same binary and exhibit either an edit-distance of ≤40% or at least one substantive token overlap (ignoring common noise tokens like grep or cat).

Building Error Recovery Patterns

Once a failure-recovery pair is identified, _build_command_recovery (lines 58-76) constructs a human-readable ExtractedPattern. This pattern describes the specific correction using a concise "when X fails, try Y instead" format.

For example, if pytest test_my_pkg fails with "command_not_found" and is later followed by python -m pytest test_my_pkg, the system generates a rule such as:

Command `pytest test_my_pkg` fails (command_not_found). Use `python -m pytest test_my_pkg` instead.

Accumulating Evidence and Persistence

To prevent noise from polluting the knowledge base, Headroom requires evidence accumulation before persisting patterns. Each distinct rule is keyed by a content_hash and tracked in an in-memory counter via _accumulate (lines 180-188). Only after reaching the min_evidence threshold (default 5 occurrences) is the pattern queued to _save_queue.

A background async worker (_save_worker, lines 222-236) then pulls patterns from this queue and writes them to the configured LocalBackend. The backend stores entries in a SQLite memories table with metadata including source: "traffic_learner", category: "error_recovery", and the accumulated evidence_count.

Writing Corrections to Agent Files

The final stage occurs when flush_to_file (lines 41-53) gathers all accumulated patterns for persistence to agent-specific markdown files. This method performs several critical operations:

• Collects both persisted database rows and in-memory patterns, merging duplicate hashes
• Filters out patterns below the min_evidence threshold
• Removes contradictory pairs (such as A→B and B→A) via _drop_contradictions
• Determines project context based on absolute paths within the pattern
• Invokes the appropriate learn plugin (resolved via headroom/learn/registry.py) to write the rule to CLAUDE.md or MEMORY.md

The plugin system, defined in headroom/learn/plugin.py and discovered through headroom/learn/registry.py, handles agent-specific formatting and file placement.

Practical Implementation Example

Below is a complete example demonstrating how to initialize the learner and process tool results:

import asyncio
from headroom.memory.traffic_learner import TrafficLearner
from headroom.memory.backends.local import LocalBackend

async def main():
    # Initialize the SQLite backend

    backend = LocalBackend(db_path="memories.db")
    
    # Create learner instance for user "alice" using Claude agent

    learner = TrafficLearner(
        backend=backend, 
        user_id="alice", 
        agent_type="claude"
    )
    
    # Start background workers

    await learner.start()
    
    # Simulate a failed Bash call

    await learner.on_tool_result(
        tool_name="Bash",
        tool_input={"command": "pytest test_suite"},
        tool_output="bash: pytest: command not found",
        is_error=True,
        agent_type="claude",
    )
    
    # Simulate the corrected successful call

    await learner.on_tool_result(
        tool_name="Bash",
        tool_input={"command": "python -m pytest test_suite"},
        tool_output=".. 2 passed",
        is_error=False,
        agent_type="claude",
    )
    
    # Allow time for evidence accumulation

    await asyncio.sleep(5)
    
    # Flush patterns to CLAUDE.md

    await learner.flush_to_file()
    
    # Graceful shutdown

    await learner.stop()

if __name__ == "__main__":
    asyncio.run(main())

After execution, CLAUDE.md will contain the learned correction:

Command `pytest test_suite` fails (command_not_found). Use `python -m pytest test_suite` instead.

Summary

Headroom converts failed tool sessions into persistent corrections through a sophisticated multi-stage pipeline:

• Capture: on_tool_result records all tool executions in a bounded history buffer
• Detection: _extract_error_recovery identifies when successful calls follow recent failures of the same tool
• Pattern Building: _build_command_recovery creates human-readable rules describing the fix
• Evidence Accumulation: The _accumulate method requires 5+ sightings (min_evidence) before queuing for persistence
• Persistence: _save_worker writes verified patterns to SQLite via LocalBackend
• File Output: flush_to_file merges patterns, removes contradictions, and invokes the learn plugin to write to agent-specific markdown files

Frequently Asked Questions

How does Headroom determine if a successful call is a correction for a previous failure?

Headroom uses the _extract_error_recovery method to scan the last five tool history entries when a successful call occurs. For Bash commands, it applies the _commands_related_as_retry heuristic which checks for identical binaries and either an edit-distance of ≤40% or substantive token overlap, ensuring the successful call represents a legitimate retry rather than an unrelated command.

What prevents Headroom from learning noisy or one-off error patterns?

The system implements an evidence threshold via the _accumulate method in headroom/memory/traffic_learner.py. Each unique pattern (keyed by content_hash) must be observed at least 5 times (configurable via min_evidence) before being queued for persistence. This filtering occurs in memory before the _save_worker ever touches the database, preventing ephemeral errors from entering the permanent knowledge base.

Where are the learned corrections stored and how are they formatted?

Verified patterns are stored in a SQLite database via headroom/memory/backends/local.py with metadata including source, category, and evidence count. When flush_to_file runs, the system invokes agent-specific plugins (found in headroom/learn/registry.py) to write formatted rules into markdown files such as CLAUDE.md or MEMORY.md, using a human-readable "when X fails, use Y instead" syntax.

Can Headroom handle contradictory error recovery patterns?

Yes. Before writing to agent files, flush_to_file calls _drop_contradictions to identify and remove opposing rules (such as A→B and B→A). This ensures the knowledge base remains consistent and prevents the agent from receiving conflicting instructions about the same error condition.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}