# Project Structure of Headroom: Understanding the Polyglot Repository Layout

> Explore the Headroom project structure. Discover how its polyglot repository efficiently organizes Python CLI, Rust core, and TypeScript SDK for optimal development and performance.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: architecture
- Published: 2026-06-21

---

**Headroom organizes its Python CLI, Rust proxy core, and TypeScript SDK into modular directories, with the main Python package in `headroom/`, the high-performance proxy in `crates/headroom-core/`, and developer tooling in `scripts/` and `docker/`.**

The `chopratejas/headroom` repository is a mixed-language codebase that combines a Python command-line interface with a Rust-based proxy engine. Understanding the headroom project structure is essential for contributing to the transform pipeline, extending the proxy server, or integrating the TypeScript SDK.

## Top-Level Directory Layout

The repository root contains twelve primary directories and configuration files that separate concerns by language and function:

- `headroom/` – Primary Python package containing the CLI entry point and transform pipeline
- `crates/headroom-core/` – Rust implementation of the proxy core and signal processing library
- `sdk/` – Official TypeScript SDK packaged for npm
- `benchmarks/` – Automated performance tests for transforms and proxy modes
- `docker/` – Docker Compose files and bake scripts for local development
- `wiki/` – Markdown documentation and troubleshooting guides
- `examples/` – Sample applications including Vercel AI SDK and LangChain demos
- `scripts/` – Shell and PowerShell helpers for installation and CI
- `plugins/` – Optional extensions including OpenClaw, Hermes, and OAuth2
- `tests/` – Pytest suite covering transforms and integration tests
- `REALIGNMENT/` – Design decision documents and architectural roadmap

Key configuration files at the root include [`pyproject.toml`](https://github.com/chopratejas/headroom/blob/main/pyproject.toml) for Python dependencies, [`Cargo.toml`](https://github.com/chopratejas/headroom/blob/main/Cargo.toml) for the Rust workspace, and `Dockerfile` for containerization.

## Python Package Architecture (headroom/)

The `headroom/` directory implements the CLI and transform pipeline. According to the source code, this package contains several critical sub-modules:

### CLI and Entry Points

The `headroom/cli/` subdirectory houses the Click-based command-line interface. The [`headroom/cli/wrap.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cli/wrap.py) file handles the `headroom` entry point, parsing arguments and dispatching to the appropriate handlers.

### Transform Pipeline

The `headroom/transforms/` directory contains the core processing logic. Each file implements a specific compression or analysis transform:

- [`log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/log_compressor.py) – Compresses log output
- [`search_compressor.py`](https://github.com/chopratejas/headroom/blob/main/search_compressor.py) – Optimizes search queries  
- [`tag_protector.py`](https://github.com/chopratejas/headroom/blob/main/tag_protector.py) – Protects sensitive tags during processing

The [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) module orchestrates these transforms sequentially, as shown in the usage example below.

### Utility Modules

Supporting functionality resides in:

- [`headroom/utils.py`](https://github.com/chopratejas/headroom/blob/main/headroom/utils.py) – Generic helper functions for file I/O and path handling
- [`headroom/update_check.py`](https://github.com/chopratejas/headroom/blob/main/headroom/update_check.py) – Auto-update logic contacting the upstream release feed

## Rust Core Implementation (crates/headroom-core/)

The Rust components deliver high-performance signal processing and proxy capabilities. Located in `crates/headroom-core/`, this crate contains two primary domains:

### Signal Processing

The `crates/headroom-core/src/signals/` directory implements the core signal processing library. These modules handle the low-level transformations that process text streams before they reach the proxy server.

### Proxy Server

The `crates/headroom-core/src/proxy/` directory contains the proxy server implementation. This component receives incoming requests and forwards them through the transform pipeline, bridging the Rust core with the Python transforms.

## SDK and Integration Examples

The repository provides first-party SDKs and sample implementations in dedicated directories:

### TypeScript SDK

The `sdk/` directory contains the official TypeScript SDK, distributed via npm. This package wraps the HTTP API exposed by the Rust proxy, allowing JavaScript applications to leverage Headroom's compression capabilities.

### Sample Applications

The `examples/` directory includes production-ready demonstrations:

- Vercel AI SDK integration patterns
- LangChain compatibility demos
- macOS launch-agent configurations

## Development and Testing Infrastructure

Headroom separates operational code from development utilities:

### Testing and Benchmarks

The `tests/` directory contains the Pytest suite covering core transforms, memory queries, and CLI wrappers. Performance validation occurs in `benchmarks/`, where [`benchmarks/run_benchmarks.py`](https://github.com/chopratejas/headroom/blob/main/benchmarks/run_benchmarks.py) serves as the entry point for automated relevance and throughput testing.

### Containerization and Scripts

Local development relies on [`docker/docker-compose.yml`](https://github.com/chopratejas/headroom/blob/main/docker/docker-compose.yml) to spin up the Rust proxy alongside the Python CLI. The `scripts/` directory provides installation helpers and CI orchestration scripts for both Unix and Windows environments.

## Working with the Codebase

The following examples demonstrate how the project structure translates to actual usage across the three languages.

### Python Transform Pipeline

To invoke transforms from the Python package:

```python
from headroom.transforms.pipeline import Pipeline

# Build a pipeline that compresses logs and then tags protected words

pipeline = Pipeline([
    "log_compressor",
    "tag_protector",
])

original = "User password is secret123"
compressed = pipeline.run(original)
print(compressed)

```

### Rust Signal Processing

When extending the Rust core, you interact with the signal processor:

```rust
use headroom_core::signals::{Signal, Processor};

let mut proc = Processor::new();
proc.add_signal(Signal::new("log_compress"));
let output = proc.process("This is a long log line ...");
println!("{}", output);

```

### TypeScript SDK Usage

To consume the proxy from TypeScript:

```typescript
import { HeadroomClient } from "headroom-sdk";

const client = new HeadroomClient({ endpoint: "http://localhost:8000" });
const result = await client.compress("Long documentation text...");
console.log(result);

```

## Summary

- **Headroom** organizes code by language: Python in `headroom/`, Rust in `crates/headroom-core/`, and TypeScript in `sdk/`.
- The **transform pipeline** resides in `headroom/transforms/` and is orchestrated by [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py).
- The **Rust proxy core** splits between `crates/headroom-core/src/signals/` for processing and `crates/headroom-core/src/proxy/` for the server.
- **Configuration** is managed via [`pyproject.toml`](https://github.com/chopratejas/headroom/blob/main/pyproject.toml) for Python and [`Cargo.toml`](https://github.com/chopratejas/headroom/blob/main/Cargo.toml) for Rust.
- **Documentation** lives in `wiki/` and `REALIGNMENT/`, while `examples/` provides integration patterns.

## Frequently Asked Questions

### What is the purpose of the headroom/transforms/ directory?

The `headroom/transforms/` directory houses the core compression and analysis logic of the Python package. Each module, such as [`log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/log_compressor.py) and [`tag_protector.py`](https://github.com/chopratejas/headroom/blob/main/tag_protector.py), implements a specific transformation that can be chained together via [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) to process text before it reaches the proxy.

### How does the Rust proxy integrate with the Python CLI?

The Rust proxy in `crates/headroom-core/src/proxy/` runs as a standalone server that receives HTTP requests. The Python CLI can communicate with this proxy to offload high-performance signal processing tasks, while the Python layer handles the transform pipeline orchestration and user interface.

### Where are the build configurations defined?

Python dependencies and build settings are defined in [`pyproject.toml`](https://github.com/chopratejas/headroom/blob/main/pyproject.toml) at the repository root, while Rust uses [`crates/headroom-core/Cargo.toml`](https://github.com/chopratejas/headroom/blob/main/crates/headroom-core/Cargo.toml) as its crate manifest. The `Dockerfile` and [`docker/docker-compose.yml`](https://github.com/chopratejas/headroom/blob/main/docker/docker-compose.yml) containers standardize the deployment environment across both languages.

### What is the difference between the wiki/ and REALIGNMENT/ directories?

The `wiki/` directory contains user-facing documentation, troubleshooting guides, and design notes intended for general consumption. The `REALIGNMENT/` directory stores project-level architectural decisions, roadmaps, and internal design documents that guide the long-term evolution of the codebase.