What Are the Stages in the Headroom Transform Pipeline? The Complete 11-Step Guide

Headroom’s transform pipeline executes eleven sequential stages—Setup, Pre-Start, Post-Start, Input Received, Input Cached, Input Routed, Input Compressed, Input Remembered, Pre-Send, Post-Send, and Response Received—every time the compress() function processes a request.

The chopratejas/headroom repository implements a stable request lifecycle that governs how raw messages are transformed into compressed tokens for LLM providers. Understanding the stages in the Headroom transform pipeline is essential for developers who want to instrument performance, implement custom hooks, or extend the compression workflow without modifying core internals.

The 11 Stages in the Headroom Transform Pipeline

According to the source code in sdk/typescript/src/compress.ts and documented in the repository’s README under “Pipeline internals”, the lifecycle is divided into eleven distinct phases. Each stage represents a specific transformation point where data is routed, cached, compressed, or prepared for transmission.

1. Setup – Initializes the request context and prepares the pipeline environment.

2. Pre-Start – Executes early-stage hooks before the compressor begins active processing.

3. Post-Start – Finalizes start-up logic and confirms pipeline initialization.

4. Input Received – Captures the raw messages or data payload from the caller.

5. Input Cached – Stores the input locally in the Cross-Context Retrieval (CCR) system, enabling reuse across agents.

6. Input Routed – The ContentRouter determines which compressor to invoke (e.g., SmartCrusher, CodeCompressor, or Kompress-base) based on content type.

7. Input Compressed – The selected compressor(s) transform the input, producing a reduced-token version optimized for the target LLM.

8. Input Remembered – Persists the compressed result and its metadata to CCR and cross-agent memory for future retrieval.

9. Pre-Send – Applies final adjustments via transforms like CacheAligner and content filters before transmission to the LLM provider.

10. Post-Send – Handles response-side bookkeeping and cleanup operations.

11. Response Received – Returns the LLM provider’s reply to the caller, completing the lifecycle.

How to Observe Pipeline Stages in Code

You can inspect these stages in real-time using the onPipelineEvent hook exposed in the TypeScript SDK (or on_pipeline_event in Python). This callback fires at each stage transition, receiving an event object whose stage field matches one of the eleven lifecycle steps.

The following TypeScript example demonstrates how to monitor the pipeline:

import { compress, onPipelineEvent } from "headroom-ai";

/* 1️⃣  Hook into the pipeline – print each stage as it occurs */
onPipelineEvent((event) => {
  console.log(`[Headroom] ${event.stage}: ${event.detail}`);
});

/* 2️⃣  Run a compression request */
async function runDemo() {
  const messages = [
    { role: "user", content: "Explain the quicksort algorithm in detail." },
  ];

  const result = await compress(messages, {
    model: "anthropic/claude-3-opus-20240229",
  });

  console.log("🟢 Final token count:", result.compressedTokens);
  console.log("🔧 Transforms applied:", result.transformsApplied);
}

runDemo();

Each event.stage value corresponds to the specific pipeline phase, allowing you to log timing data, inject custom logic, or profile specific transformations like Input Compressed or Input Routed.

Key Implementation Files

The transform pipeline is implemented across several critical files in the chopratejas/headroom repository:

• sdk/typescript/src/compress.ts – Core entry point for the TypeScript SDK that orchestrates the eleven-stage pipeline.

• sdk/typescript/src/shared-context.ts – Implements the cross-agent memory (CCR) used in the Input Cached and Input Remembered stages.

• sdk/typescript/src/hooks.ts – Exposes the onPipelineEvent hook for observing pipeline stages.

• headroom/providers/registry.py – Provider-agnostic registry mapping LLM providers to the compression pipeline.

• wrap.py – CLI entry point (headroom wrap …) that constructs the request lifecycle.

• README.md (section “Pipeline internals”) – Human-readable documentation describing each stage and its purpose.

Summary

• The Headroom transform pipeline consists of eleven sequential stages that execute from request initialization through response delivery.
• Key transformation points include Input Routed (ContentRouter selection), Input Compressed (token reduction), and Input Remembered (CCR persistence).
• Developers can observe stages via onPipelineEvent in TypeScript or on_pipeline_event in Python without modifying core source code.
• Critical implementation files include sdk/typescript/src/compress.ts, shared-context.ts, and headroom/providers/registry.py.

Frequently Asked Questions

What happens during the Input Routed stage?

During the Input Routed stage, the ContentRouter analyzes the incoming content and selects the appropriate compression strategy. According to the source code in chopratejas/headroom, it chooses between specialized compressors like SmartCrusher for general text, CodeCompressor for programming syntax, or Kompress-base for standard token reduction, ensuring optimal compression for the specific content type.

How does the Input Cached stage support cross-agent memory?

The Input Cached stage stores raw inputs locally using the Cross-Context Retrieval (CCR) system implemented in sdk/typescript/src/shared-context.ts. This caching enables the Input Remembered stage to later persist compressed results and metadata, allowing different agents or subsequent requests to retrieve previously processed content without re-compression, significantly reducing latency and token costs.

Can I modify the transform pipeline without forking the repository?

Yes. You can extend the pipeline using on_pipeline_event. This hook allows you to observe or modify the flow at any of the eleven stages by registering callbacks that receive stage-specific event objects. As implemented in sdk/typescript/src/hooks.ts, these extensions run alongside the core transforms without requiring changes to the source code in compress.ts or registry.py.

What is the difference between Pre-Start and Setup stages?

Setup initializes the request context and prepares the pipeline environment, while Pre-Start executes early-stage hooks immediately after setup but before the compressor begins active processing. Setup handles internal initialization, whereas Pre-Start is designed for user-defined hooks that need to run before any input processing begins.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}