# How to Get Detailed Feature Information for the caveman-compress Skill

> Discover detailed feature information for the caveman-compress skill. Learn how this skill rewrites memory files into a terse caveman style, reducing tokens by 46% via a validation-backed pipeline.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: how-to-guide
- Published: 2026-07-03

---

**The caveman-compress skill rewrites natural-language memory files into a terse "caveman" style while preserving technical elements, achieving approximately 46% token reduction through a validation-backed pipeline implemented in the `scripts/` directory.**

The `caveman-compress` skill is a specialized utility within the JuliusBrussee/caveman repository that optimizes natural-language memory files for token efficiency. This skill compresses prose in files like [`CLAUDE.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.md) and [`todos.md`](https://github.com/JuliusBrussee/caveman/blob/main/todos.md) by removing filler words and hedging while strictly preserving code blocks, URLs, and file paths. Understanding the complete feature set requires examining both the declarative [`SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/SKILL.md) specification and the implementation logic in the Python scripts.

## Core Purpose and Trigger Mechanisms

The primary function of the `caveman-compress` skill is to reduce token count in natural-language memory files while maintaining semantic integrity. According to the [`skills/caveman-compress/SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/SKILL.md) specification in JuliusBrussee/caveman, the skill is invoked via the slash command `/caveman-compress <filepath>` within a Caveman-enabled chat environment, or by requesting the agent to "compress memory file".

## Compression Rules and Preservation Guarantees

### What Gets Removed

The compression algorithm targets specific linguistic patterns defined in [`SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/SKILL.md). The skill strips **articles** (`a`, `an`, `the`), **filler words** (`just`, `really`, `basically`), **pleasantries** (`sure`, `of course`), **hedging phrases** (`it might be worth...`), **redundant phrasing** (`in order to` → `to`), and **connective fluff** (`however`, `furthermore`).

### What Stays Preserved

The [`scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/validate.py) module enforces strict preservation guarantees. **Code blocks** (triple backtick fences), **inline code** (single backticks), **URLs**, **file paths**, **commands**, **technical terms**, **proper nouns**, **dates**, **version numbers**, and **environment variables** remain completely unchanged. Additionally, the skill preserves markdown structure including headings, bullet hierarchy, numbered lists, tables, and front-matter.

## Process Flow and Pipeline Architecture

The execution pipeline follows a four-step process defined in [`skills/caveman-compress/scripts/__main__.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/__main__.py):

1. **Detection**: The [`scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/detect.py) module determines if the file extension is supported (`.md`, `.txt`, `.typ`, `.typst`, `.tex`, or no extension) and identifies mixed-content files.
2. **Compression**: The [`scripts/compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/compress.py) module calls Claude to perform the compression according to the rules defined in [`SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/SKILL.md).
3. **Validation**: The [`scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/validate.py) script verifies that preserved elements (code fences, inline code, URLs) remain unchanged after compression.
4. **Retry Logic**: If validation fails, the CLI attempts a targeted fix rather than full recompression, retrying up to two times before returning an error.

## Safety Boundaries and File Type Detection

The [`scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/detect.py) implementation enforces strict boundary conditions. The skill operates **only** on natural-language extensions and **never** touches source code files (`.py`, `.js`, `.ts`, `.json`, `.yaml`, `.toml`, `.sh`). It also skips any file ending with [`.original.md`](https://github.com/JuliusBrussee/caveman/blob/main/.original.md) to prevent recursive processing.

According to [`skills/caveman-compress/SECURITY.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/SECURITY.md), the skill performs **local file I/O only**: it reads the supplied file, writes the compressed version, and creates a backup `<file>.original.md`. No network calls, subprocess executions, or external file access occur beyond the named file.

## Performance Characteristics and Benchmarking

The [`scripts/benchmark.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/benchmark.py) suite measures token-saving performance across various file types. Benchmarks show an average **46% token reduction** per memory file, with savings persisting across sessions. This metric validates the skill's effectiveness for managing large context windows in the JuliusBrussee/caveman ecosystem.

## How to Access Implementation Details

To explore additional nuances such as validation logic, retry policies, or benchmark methodology, examine the heavily commented scripts in `skills/caveman-compress/scripts/`:

- **Inspect validation flow**: Review [`scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/validate.py) to see the regex patterns that enforce "preserve exactly" constraints for code fences and inline snippets.
- **Read detection logic**: Examine [`scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/detect.py) to understand accepted extensions and mixed-content handling.
- **Review benchmark suite**: Analyze [`scripts/benchmark.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/benchmark.py) for concrete token savings numbers across various file types.

## Practical Usage Examples

### Using the Slash Command

```text
/caveman-compress CLAUDE.md

```

This overwrites [`CLAUDE.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.md) with the compressed version and creates [`CLAUDE.original.md`](https://github.com/JuliusBrussee/caveman/blob/main/CLAUDE.original.md) as a backup.

### Running from the CLI

```bash
cd skills/caveman-compress
python3 -m scripts /absolute/path/to/CLAUDE.md

```

The [`scripts/__main__.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/__main__.py) entry point executes the full detection-compression-validation pipeline.

### Programmatic Invocation

```python
from pathlib import Path
from skills.caveman_compress.scripts import compress

file_path = Path("/absolute/path/to/CLAUDE.md")
compress.run(file_path)

```

The `compress.run` wrapper in [`scripts/compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/compress.py) executes the same pipeline as the CLI.

### Inspecting the Backup

```bash
cat CLAUDE.original.md

```

This displays the pre-compression source for audit purposes.

## Summary

- The `caveman-compress` skill reduces tokens by approximately 46% by removing filler words and hedging from natural-language files while preserving all technical elements.
- The pipeline in [`scripts/__main__.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/__main__.py) follows a detect-compress-validate-retry pattern with strict safety boundaries enforced by [`scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/detect.py) and [`scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/validate.py).
- Only files with extensions `.md`, `.txt`, `.typ`, `.typst`, `.tex`, or no extension are processed; source code files are explicitly excluded.
- Original files are backed up as `<filename>.original.md` before overwriting, with all operations restricted to local file I/O.
- Detailed feature specifications reside in [`skills/caveman-compress/SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/SKILL.md), while implementation logic is documented in the `scripts/` directory.

## Frequently Asked Questions

### What file types does the caveman-compress skill support?

The skill processes files with extensions `.md`, `.txt`, `.typ`, `.typst`, `.tex`, or no extension, as implemented in [`scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/detect.py). It explicitly excludes source code files including `.py`, `.js`, `.ts`, `.json`, `.yaml`, `.toml`, and `.sh`, and skips any file ending with [`.original.md`](https://github.com/JuliusBrussee/caveman/blob/main/.original.md) to prevent recursive compression.

### How does the validation system ensure technical content preservation?

The [`scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/scripts/validate.py) module checks that preserved elements—specifically code blocks (triple backtick fences), inline code (single backticks), and URLs—remain unchanged after compression. If validation fails, the system attempts a targeted fix rather than full recompression, retrying up to two times before returning an error to the user.

### What happens to the original file when using caveman-compress?

Before overwriting, the skill automatically creates a backup copy named `<filename>.original.md` in the same directory as the source file. This backup contains the complete pre-compression content for audit purposes, and the skill skips any file already ending with [`.original.md`](https://github.com/JuliusBrussee/caveman/blob/main/.original.md) to prevent processing its own backups.

### Where can I find the complete specification for the caveman-compress skill?

The authoritative feature specification resides in [`skills/caveman-compress/SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/SKILL.md) within the JuliusBrussee/caveman repository. This document defines the purpose, trigger commands, compression rules, preservation guarantees, and boundary conditions. Implementation details are documented in the `scripts/` subdirectory, particularly [`__main__.py`](https://github.com/JuliusBrussee/caveman/blob/main/__main__.py), [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py), [`validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/validate.py), and [`detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/detect.py).