# Exact Token Savings for Memory File Compression in Caveman

> Discover exact token savings for memory file compression with Caveman. Reduce input token counts by up to 59.6% on average to optimize LLM prompts.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: performance
- Published: 2026-07-11

---

**The `/caveman-compress` command reduces input token counts by approximately 46% on average when compressing project memory files, with individual savings ranging from 36.9% to 59.6% depending on file content.**

The Caveman project by JuliusBrussee provides a specialized compression utility that rewrites verbose project memory files into a compact "caveman-speak" format. This transformation directly reduces the token budget consumed when the LLM loads compressed files like [`CLAUDE.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.md) or [`todo-list.md`](https://github.com/JuliusBrussee/caveman/blob/main/todo-list.md) at the start of each session.

## How Memory File Compression Works

According to the source code in [`src/plugins/opencode/commands/caveman-compress.md`](https://github.com/JuliusBrussee/caveman/blob/main/src/plugins/opencode/commands/caveman-compress.md), the compression algorithm rewrites natural language documentation into a condensed format while preserving semantic meaning. The `caveman-compress` skill documentation in [`skills/caveman-compress/README.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/README.md) confirms this process targets **input tokens only**—the compressed files are read at session startup, reducing context window usage without affecting output generation costs.

## Exact Token Savings Breakdown

The repository's README documents precise measurements for five representative memory files:

| Memory file | Original tokens | Compressed tokens | Savings |
|-------------|-----------------|-------------------|---------|
| [`claude-md-preferences.md`](https://github.com/JuliusBrussee/caveman/blob/main/claude-md-preferences.md) | 706 | 285 | **59.6%** |
| [`project-notes.md`](https://github.com/JuliusBrussee/caveman/blob/main/project-notes.md) | 1,145 | 535 | **53.3%** |
| [`claude-md-project.md`](https://github.com/JuliusBrussee/caveman/blob/main/claude-md-project.md) | 1,122 | 636 | **43.3%** |
| [`todo-list.md`](https://github.com/JuliusBrussee/caveman/blob/main/todo-list.md) | 627 | 388 | **38.1%** |
| [`mixed-with-code.md`](https://github.com/JuliusBrussee/caveman/blob/main/mixed-with-code.md) | 888 | 560 | **36.9%** |
| **Average** | **898** | **481** | **46%** |

These figures represent the reduction in tokens sent to the LLM when the compressed versions are loaded. Files with verbose natural language (like preferences documentation) achieve higher compression ratios than files containing code blocks.

## Using the Caveman-Compress Command

Compress individual memory files using the CLI command defined in the OpenCode plugin:

```bash

# Compress a single memory file (e.g., CLAUDE.md)

caveman-compress CLAUDE.md

```

Batch process multiple project files with a shell loop:

```bash

# Compress all supported memory files in the current project

for f in CLAUDE.md todos.md preferences.md; do
  caveman-compress "$f"
done

```

## Calculating Token Savings Programmatically

The repository uses whitespace tokenization to measure savings. You can replicate this calculation using Python:

```python

# Example: programmatically compute token savings

import json, pathlib

def token_count(text: str) -> int:
    # Simple whitespace tokenisation (the same logic the repo uses)

    return len(text.split())

def compress_and_report(file_path: str):
    original = pathlib.Path(file_path).read_text()
    # Simulate compression by calling the CLI (omitted here)

    # compressed = subprocess.check_output(["caveman-compress", file_path])

    # For illustration, assume a 46% reduction:

    compressed = " " .join(original.split()[:int(0.54 * token_count(original))])
    saved_pct = 100 * (1 - token_count(compressed) / token_count(original))
    print(f"{file_path}: {saved_pct:.1f}% tokens saved")

compress_and_report("CLAUDE.md")

```

## Summary

- **Average savings**: 46% reduction in input tokens across all memory files
- **Range**: 36.9% to 59.6% depending on content type (code-heavy files compress less)
- **Scope**: Affects only input tokens loaded at session start; output tokens remain unchanged
- **Target files**: [`CLAUDE.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.md), [`todo-list.md`](https://github.com/JuliusBrussee/caveman/blob/main/todo-list.md), [`project-notes.md`](https://github.com/JuliusBrussee/caveman/blob/main/project-notes.md), and other project memory files
- **Implementation**: Defined in [`src/plugins/opencode/commands/caveman-compress.md`](https://github.com/JuliusBrussee/caveman/blob/main/src/plugins/opencode/commands/caveman-compress.md) and documented in [`skills/caveman-compress/README.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/README.md)

## Frequently Asked Questions

### What is the exact token savings for memory file compression in Caveman?

The average token savings is **46%**, with specific files ranging from **36.9%** (for code-heavy files) to **59.6%** (for verbose preference documentation). These savings are calculated using whitespace tokenization and measured against the original file sizes.

### Does caveman-compress affect output tokens or only input tokens?

Compression affects **only input tokens**. The reduction applies to the context loaded when the LLM reads the compressed memory files at the start of a session. Output token generation costs remain identical to uncompressed workflows.

### How do I compress multiple memory files at once?

Use a shell loop to batch process files. The command `caveman-compress` accepts individual file paths, so you can iterate over [`CLAUDE.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.md), [`todos.md`](https://github.com/JuliusBrussee/caveman/blob/main/todos.md), [`preferences.md`](https://github.com/JuliusBrussee/caveman/blob/main/preferences.md), and other project memory files in a single script.

### Where is the compression logic implemented in the Caveman repository?

The CLI command definition resides in [`src/plugins/opencode/commands/caveman-compress.md`](https://github.com/JuliusBrussee/caveman/blob/main/src/plugins/opencode/commands/caveman-compress.md), while the skill documentation and token savings receipts are maintained in [`skills/caveman-compress/README.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/README.md) and the main [`README.md`](https://github.com/JuliusBrussee/caveman/blob/main/README.md) respectively.