How Headroom Enables Cross-Agent Context Sharing in Multi-Agent Workflows

Headroom's SharedContext class provides a thread-safe, in-memory cache that automatically compresses agent outputs and makes them available to any other agent in the same Python process, supporting efficient cross-agent context sharing without network overhead.

The chopratejas/headroom repository solves a critical bottleneck in multi-agent AI systems: moving large contexts between agents without consuming excessive memory or token budgets. By implementing a cross-agent context sharing abstraction, Headroom allows Crew AI, LangGraph, OpenAI Agents SDK, and custom agents to store compressed outputs once and retrieve them on demand, preserving the original data while minimizing memory footprint.

Core Architecture

The memory system centers on a public API that wraps an in-memory dictionary with automatic compression, thread synchronization, and lifecycle management.

SharedContext Class

The SharedContext class in headroom/shared_context.py serves as the primary interface for cross-agent memory operations. It maintains an internal dictionary of ContextEntry objects protected by a threading.Lock, ensuring safe concurrent access when multiple agents read and write simultaneously. The class enforces configurable TTL (time-to-live) and size limits to prevent unbounded memory growth.

ContextEntry Dataclass

Each stored item becomes a ContextEntry instance that tracks both the original and compressed representations. According to the source code in headroom/shared_context.py (lines 36-55), this dataclass stores:

• Original text and compressed text
• Token counts for both versions
• Timestamps and compression transforms applied
• Per-entry statistics like savings_percent

Compression Pipeline

When SharedContext.put() is called, it delegates to headroom.compress.compress(), running the same CCR (Compress-Cache-Retrieve) stack used by the Headroom proxy. As implemented in headroom/shared_context.py (lines 98-112), the pipeline automatically selects SmartCrusher for JSON, CodeCompressor for code blocks, and Kompress for plain text, guaranteeing that every stored item receives consistent compression heuristics.

TTL and Eviction

The system prevents memory leaks through automatic TTL expiration and size-based eviction. Entries expire after a configurable ttl period (defaulting to 1 hour), and when the cache reaches max_entries, the oldest entry is removed. This logic in headroom/shared_context.py (lines 120-129) ensures stale context never leaks across unrelated workflows.

Thread Safety

All mutating operations—including put, clear, and eviction—acquire the same threading.Lock. Read operations like get, keys, and stats also lock briefly to snapshot entries safely. This design makes the SharedContext safe to share between concurrent agents in the same Python process without race conditions.

How Cross-Agent Context Sharing Works

The workflow follows a simple store-and-retrieve pattern that minimizes redundant computation across agent boundaries.

Storing Agent Output

When Agent A completes a task, it stores the result using the put method:

from headroom import SharedContext

ctx = SharedContext(ttl=1800, max_entries=50)
entry = ctx.put("research", large_output, agent="researcher")

The put method automatically compresses the payload and creates a ContextEntry in the internal _entries map under the specified key.

Retrieving Compressed Context

Agent B can immediately access a compressed version suitable for context windows:

summary = ctx.get("research")  # Returns compressed text

By default, get returns entry.compressed, significantly reducing token consumption for downstream processing.

Accessing Original Data

When full fidelity is required, agents can bypass compression:

full_text = ctx.get("research", full=True)  # Returns original unchanged

The original text remains stored without recomputation, allowing agents to toggle between summary and detail views as needed.

Monitoring Efficiency

The stats() method aggregates token counts across all active entries, exposing total_original_tokens and savings_percent metrics useful for debugging compression efficiency in headroom/shared_context.py (lines 87-102).

Implementation Examples

Basic Initialization and Usage

Initialize a process-wide context with custom TTL and capacity limits:

from headroom import SharedContext

# 30-minute TTL, max 50 entries

ctx = SharedContext(ttl=1800, max_entries=50)

# Store large JSON result

large_json = '{"items": [...]}'
entry = ctx.put("search_results", large_json, agent="searcher")
print(entry.savings_percent)  # e.g., 85.0

# Consume compressed summary

summary = ctx.get("search_results")
print(len(summary))  # Much shorter than original

# Retrieve full data when needed

full = ctx.get("search_results", full=True)
assert full == large_json

# Inspect metadata

meta = ctx.get_entry("search_results")
print(meta.transforms)  # List of compression transforms applied

# Cleanup

print(ctx.stats())  # Aggregated token savings

ctx.clear()  # Wipe all entries

Crew AI Integration

Store research task outputs for consumption by coding agents:

# After research task completes

ctx.put("findings", researcher_task.output.raw)

# Coding task retrieves compressed context

coder_context = ctx.get("findings")

LangGraph Integration

Share context between graph nodes without serializing to disk:

def researcher_node(state):
    result = do_research()
    ctx.put("research", result)
    return {"research_summary": ctx.get("research")}

OpenAI Agents SDK Integration

Compress large messages during hand-off filters:

def compress_handoff(messages):
    for msg in messages:
        if len(msg.content) > 1000:
            ctx.put(msg.id, msg.content)
            msg.content = ctx.get(msg.id)  # Compressed version

    return messages

Key Configuration Parameters

SharedContext accepts two critical parameters for memory management:

• ttl: Time-to-live in seconds (default 3600). Entries older than this are considered expired.
• max_entries: Maximum number of entries before eviction occurs (default unbounded).

When entries exceed their TTL or the cache reaches capacity, the _evict_if_needed method (lines 130-138 in headroom/shared_context.py) automatically removes stale entries during the next put or get operation.

Summary

• Headroom's SharedContext provides a process-wide memory store enabling cross-agent context sharing through a simple Python API.
• Automatic compression via the CCR stack reduces memory footprint while preserving originals for on-demand retrieval.
• Thread-safe operations protected by threading.Lock allow concurrent access from multiple agents without race conditions.
• TTL and eviction policies prevent memory leaks by automatically removing stale entries and enforcing size limits.
• Integration flexibility supports Crew AI, LangGraph, OpenAI Agents SDK, and custom agent frameworks through standard Python imports.

Frequently Asked Questions

How does SharedContext handle concurrent access from multiple agents?

The SharedContext class uses a threading.Lock to protect all mutating operations. When agents call put, clear, or when automatic eviction triggers, the method acquires the lock before modifying the internal _entries dictionary. Read operations like get and stats also acquire the lock briefly to snapshot data, ensuring thread-safe cross-agent context sharing within the same Python process.

What compression algorithms does Headroom use for context sharing?

According to the source code in headroom/shared_context.py, the put method delegates to headroom.compress.compress(), which applies the CCR (Compress-Cache-Retrieve) stack. This includes SmartCrusher for JSON data, CodeCompressor for code blocks, and Kompress for plain text. The specific transforms applied to each entry are recorded in the ContextEntry.transforms field for debugging.

Can I retrieve the original uncompressed data after storing compressed context?

Yes. The get method accepts a full=True parameter that returns the original text rather than the compressed version. The ContextEntry dataclass stores both representations in memory, so retrieving the full version requires no recomputation. If you need metadata about the entry itself, use get_entry() to access the complete ContextEntry object.

How does automatic eviction prevent memory leaks in long-running processes?

The system implements two safeguards in headroom/shared_context.py (lines 120-138). First, entries automatically expire after the configured ttl seconds (default 1 hour), and expired entries are removed on the next access. Second, when the number of entries reaches max_entries, the oldest entry is evicted on every put operation. This bounded cache design ensures that cross-agent context sharing does not consume unbounded memory over extended workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}