How to Integrate Headroom with LangChain for Chat Model Context Compression

You can integrate Headroom with LangChain by wrapping any BaseChatModel in the HeadroomChatModel class, which intercepts LLM calls to automatically compress chat contexts while maintaining full compatibility with LangChain's synchronous, asynchronous, and streaming APIs.

The Headroom library ships with a dedicated LangChain integration that enables seamless chat model context compression without requiring architectural changes to your existing chains or agents. By implementing standard LangChain interfaces in headroom/integrations/langchain/, the library allows you to plug compression into chat models, memory stores, retrievers, and tools with minimal configuration.

Installation and Setup

To integrate Headroom with LangChain, install the package with the LangChain extra:

pip install "headroom-ai[langchain]"

This installs the core Headroom SDK along with LangChain-compatible wrappers located in headroom/integrations/langchain/.

Wrapping Chat Models with HeadroomChatModel

The primary integration point is the HeadroomChatModel wrapper defined in headroom/integrations/langchain/chat_model.py. This class subclasses LangChain's BaseChatModel and intercepts every LLM call to run the TransformPipeline compression logic.

How the Wrapper Works

When you invoke the wrapped model, the following sequence occurs:

1. Message Conversion – LangChain message objects are converted to OpenAI-style format required by Headroom's core engine
2. Pipeline Execution – The TransformPipeline applies context compression strategies (smart truncation, relevance filtering, caching)
3. Provider Auto-Detection – With auto_detect_provider=True (default), the wrapper inspects the underlying model class (e.g., ChatOpenAI, ChatAnthropic) and selects the matching Headroom provider (OpenAIProvider, AnthropicProvider) for accurate token counting
4. Response Conversion – Optimized messages convert back to LangChain format before returning

Basic Usage Example

Wrap any LangChain chat model to enable automatic compression:

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from headroom.integrations import HeadroomChatModel

# Wrap the base model

llm = HeadroomChatModel(
    ChatOpenAI(model="gpt-4o"),
    auto_detect_provider=True  # Automatically selects correct token counter

)

# Use exactly like the original model

response = llm.invoke([HumanMessage(content="Explain quantum tunnelling in detail.")])
print(response.content)

# View compression statistics

print(llm.get_savings_summary())

Each optimization pass generates an OptimizationMetrics record tracking tokens before/after and savings percentage. The wrapper aggregates these into total_tokens_saved and exposes get_savings_summary() for quick reporting.

Async and Streaming Support

The HeadroomChatModel wrapper implements _stream, _agenerate, and _astream methods, ensuring compression works across all LangChain execution patterns.

Async Implementation

async def async_chat():
    llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
    
    # Async invoke

    result = await llm.ainvoke([HumanMessage(content="What's the weather in Paris?")])
    print(result.content)
    
    # Async streaming

    async for chunk in llm.astream([HumanMessage(content="Tell me a joke.")]):
        print(chunk.content, end="", flush=True)

# asyncio.run(async_chat())

Both ainvoke and astream benefit from the same compression pipeline as synchronous calls, with metrics aggregated across the full conversation.

Memory Integration for Long Conversations

For applications with conversation history, HeadroomChatMessageHistory in headroom/integrations/langchain/memory.py wraps any ChatMessageHistory to automatically compress old turns when token thresholds are exceeded.

from langchain.memory import ConversationBufferMemory
from langchain_community.chat_message_histories import ChatMessageHistory
from headroom.integrations import HeadroomChatMessageHistory

# Create compressed message history

base_history = ChatMessageHistory()
compressed_history = HeadroomChatMessageHistory(
    base_history,
    compress_threshold_tokens=4000,  # Trigger compression when >4K tokens

    keep_recent_turns=5               # Always preserve last 5 exchanges

)

# Use with standard LangChain memory

memory = ConversationBufferMemory(chat_memory=compressed_history)

This integration ensures that long-running conversations stay within context window limits while preserving recent context in full fidelity.

Retriever Integration for Document Compression

When building RAG applications, HeadroomDocumentCompressor in headroom/integrations/langchain/retriever.py implements BaseDocumentCompressor for use within ContextualCompressionRetriever.

from langchain.retrievers import ContextualCompressionRetriever
from langchain_community.vectorstores import FAISS
from headroom.integrations import HeadroomDocumentCompressor

vectorstore = FAISS.from_documents(docs, embeddings)
base_retriever = vectorstore.as_retriever(search_kwargs={"k": 50})

# Configure compression strategy

compressor = HeadroomDocumentCompressor(
    max_documents=10,
    min_relevance=0.3,
    prefer_diverse=True
)

retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=base_retriever,
)

This filters retrieved documents by relevance and diversity before sending them to the LLM, reducing noise and token costs in retrieval-augmented generation workflows.

Tool Wrapping for Agents

For agent applications where tool outputs may be large, the wrap_tools_with_headroom utility in headroom/integrations/langchain/agents.py decorates LangChain tools to compress outputs before the next LLM turn.

from langchain_core.tools import tool
from headroom.integrations import wrap_tools_with_headroom

@tool
def search_web(query: str) -> str:
    """Return search results as potentially large JSON."""
    return large_json_payload

# Wrap tools to compress outputs >1000 characters

tools = wrap_tools_with_headroom([search_web], min_chars_to_compress=1000)

# Use with any LangChain agent

llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
agent = create_openai_tools_agent(llm, tools, prompt)

This prevents large tool outputs from consuming excessive context window space in multi-step agent workflows.

Monitoring Compression Metrics

Track token savings in real-time using StreamingMetricsTracker from headroom/integrations/langchain/streaming.py:

from headroom.integrations import StreamingMetricsTracker

tracker = StreamingMetricsTracker(model="gpt-4o")

for chunk in llm.stream([HumanMessage(content="Write a detailed analysis.")]):
    tracker.add_chunk(chunk)
    print(chunk.content, end="")

metrics = tracker.finish()
print(f"Saved {metrics.total_tokens_saved} tokens ({metrics.savings_percentage}%)")

The StreamingMetricsTracker records token usage during streaming, providing visibility into compression efficiency without blocking the response flow.

Summary

• HeadroomChatModel in headroom/integrations/langchain/chat_model.py wraps any BaseChatModel to intercept and compress contexts before LLM calls
• Provider auto-detection automatically selects the correct token counting strategy based on the wrapped model class
• Full API compatibility includes synchronous (invoke), asynchronous (ainvoke), and streaming (stream, astream) methods
• Memory integration via HeadroomChatMessageHistory automatically compresses old conversation turns while preserving recent history
• Retriever integration via HeadroomDocumentCompressor filters and compresses retrieved documents for RAG applications
• Tool wrapping via wrap_tools_with_headroom compresses large tool outputs in agent workflows
• Metrics tracking provides aggregated token savings via get_savings_summary() and StreamingMetricsTracker

Frequently Asked Questions

Does Headroom support both async and streaming APIs in LangChain?

Yes. The HeadroomChatModel wrapper implements _stream, _agenerate, and _astream methods in headroom/integrations/langchain/chat_model.py, ensuring that llm.stream(), llm.astream(), and await llm.ainvoke() all benefit from context compression while maintaining standard LangChain interfaces.

How does Headroom detect which provider I'm using?

When auto_detect_provider=True (the default), the wrapper inspects the class name of the underlying model (e.g., ChatOpenAI, ChatAnthropic) and automatically selects the matching Headroom provider (OpenAIProvider, AnthropicProvider, etc.) from headroom/integrations/langchain/providers.py. This ensures accurate token counting and enables provider-specific caching optimizations.

Can I use Headroom with LangChain's memory classes?

Yes. The HeadroomChatMessageHistory class in headroom/integrations/langchain/memory.py wraps any ChatMessageHistory and automatically compresses old conversation turns when they exceed compress_threshold_tokens, while keeping keep_recent_turns uncompressed. This integrates seamlessly with ConversationBufferMemory and other LangChain memory implementations.

Will compression affect my LLM's response quality?

The TransformPipeline applies intelligent compression strategies such as relevance filtering and smart truncation rather than naive truncation. According to the implementation in the core Headroom SDK, the system preserves semantic meaning while reducing token count, and the OptimizationMetrics provide transparency into exactly what was compressed so you can tune thresholds for your specific use case.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}