# What Is the Cache Aligner and How Does It Improve KV Cache Hit Rates in Headroom?

> Discover how the Cache Aligner stabilizes dynamic tokens in LLM requests to boost KV cache hit rates and improve performance in Headroom.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: deep-dive
- Published: 2026-06-14

---

**The Cache Aligner is a prompt preprocessing transform that stabilizes dynamic tokens in LLM requests, enabling the KV cache to recognize and reuse previously computed attention keys and values across requests that share identical static prefixes.**

Headroom is an open-source inference optimization framework. The **Cache Aligner** serves as the first transform in the default processing pipeline, specifically designed to boost **KV cache hit rates** by eliminating volatile tokens that otherwise force full recomputation of attention states.

## How the Cache Aligner Normalizes Prompts

The transform operates in three distinct stages to convert variable prompt prefixes into deterministic sequences that the KV cache can match.

### Extraction of Dynamic Tokens

In [`headroom/transforms/cache_aligner.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/cache_aligner.py), the extraction phase scans incoming prompts for **dynamic tokens** using regular-expression detectors. These include timestamps, UUIDs, session-specific identifiers, and short-term token counters. Because the KV cache operates on exact token sequences, even a single varying token breaks the literal prefix match and forces the LLM to recompute attention keys and values for the entire sequence.

### Normalisation with Stable Placeholders

The normalisation stage replaces each detected dynamic fragment with a **stable placeholder** such as `<DATE>`, `<UUID>`, or `<TOKEN_ID>`. This process leaves static text, system instructions, and user-provided content untouched. By transforming noisy prefixes into deterministic ones, identical static prompts map to the same KV-cache entry, increasing the probability of a cache hit.

### Re-assembly Before Model Execution

Finally, the re-assembly stage injects the placeholders back into the request before transmission to the LLM. The model receives valid plain text (placeholders are literal strings), while the **internal KV cache** sees a repeatable prefix. This allows the LLM to skip computation for the static portion of the prompt on subsequent calls, saving latency and reducing token costs.

## Configuration and Implementation

The Cache Aligner is implemented in [`headroom/transforms/cache_aligner.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/cache_aligner.py) and positioned as the first step in the processing pipeline defined in [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py).

### Enabling the Transform

You activate the Cache Aligner via the Headroom configuration file. By default, the transform is enabled and processes prompts before they reach the model.

```yaml

# headroom.yaml

transforms:
  - name: cache_aligner
    enabled: true          # Turn the transform on (default)

    # Optional custom patterns – see the configuration wiki for details

    # patterns:

    #   - regex: '\b[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}\b'

    #     placeholder: '<UUID>'

```

### Python SDK Usage

When using the Headroom Python SDK, the Cache Aligner automatically preprocesses prompts based on your configuration.

```python
from headroom import HeadroomClient

client = HeadroomClient.from_config("headroom.yaml")

prompt = """\
You are an assistant.
User request at 2024-06-14T12:34:56Z: Summarise the following text…
"""

response = client.complete(prompt)
print(response.text)

```

With the `cache_aligner` active, the timestamp transforms into `<DATE>` before reaching the model, allowing the KV cache to match this request against previous ones sharing the same static prefix.

### Debugging Transformations

To inspect how the Cache Aligner modifies your prompts, enable debug mode in the client.

```python
client = HeadroomClient.from_config("headroom.yaml", debug=True)

# The client logs the intermediate transformed prompt:

#   >>> Transformed prompt: "You are an assistant.\nUser request at <DATE>: Summarise …"

```

## Performance Impact on KV Cache Hit Rates

According to the benchmark suite in [`benchmarks/prefix_cache_benchmark.py`](https://github.com/chopratejas/headroom/blob/main/benchmarks/prefix_cache_benchmark.py), enabling the Cache Aligner can raise cache-hit ratios from low-single-digits to **30%–50%** depending on workload characteristics. This improvement directly translates to reduced latency and lower token costs, as the LLM avoids recomputing attention keys and values for static prompt prefixes.

The performance gains occur because the Cache Aligner collapses request variability. Without normalization, each unique timestamp or session ID creates a distinct cache entry, rendering the KV cache ineffective for otherwise identical prompts.

## Summary

- The **Cache Aligner** is the first transform in Headroom's pipeline, implemented in [`headroom/transforms/cache_aligner.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/cache_aligner.py).
- It improves **KV cache hit rates** by replacing dynamic tokens (timestamps, UUIDs) with stable placeholders before the LLM processes the request.
- The transform enables the KV cache to recognize identical static prefixes across requests, bypassing redundant computation for repeated patterns.
- Benchmarks demonstrate **30%–50%** cache hit rate improvements when the aligner is enabled.
- Configuration occurs via [`headroom.yaml`](https://github.com/chopratejas/headroom/blob/main/headroom.yaml) with optional custom regex patterns for domain-specific dynamic tokens.

## Frequently Asked Questions

### What types of dynamic tokens does the Cache Aligner detect?

The Cache Aligner detects timestamps, UUIDs, session-specific identifiers, and short-term token counters using configurable regular expressions. You can extend the default patterns in [`headroom.yaml`](https://github.com/chopratejas/headroom/blob/main/headroom.yaml) to match domain-specific volatile strings such as request IDs or temporary authentication tokens.

### How does the Cache Aligner differ from standard KV caching?

Standard KV caching relies on exact token sequence matching; if any token varies, the cache misses. The Cache Aligner preprocesses prompts to normalize these variations before they reach the cache, effectively collapsing multiple unique requests into identical cacheable prefixes that the KV cache can recognize and reuse.

### Can I customize the regex patterns for token detection?

Yes. The Cache Aligner supports custom regex patterns defined in the configuration file. Specify a regex and corresponding placeholder in the `patterns` section of the `cache_aligner` transform configuration to handle specialized dynamic content in your prompts, as documented in [`wiki/configuration.md`](https://github.com/chopratejas/headroom/blob/main/wiki/configuration.md).

### What performance gains can I expect from enabling the Cache Aligner?

Based on the benchmark suite in [`benchmarks/prefix_cache_benchmark.py`](https://github.com/chopratejas/headroom/blob/main/benchmarks/prefix_cache_benchmark.py), the Cache Aligner typically improves KV cache hit rates from low-single-digits to **30%–50%**, resulting in proportional latency reductions and cost savings for workloads with repeated prompt patterns that differ only by dynamic metadata.