What Kind of Data Does the Caveman-Compress Skill Handle?

The caveman-compress skill exclusively processes natural-language memory files—such as CLAUDE.md, todos.md, and preferences.md—compressing verbose prose while preserving every technical artifact including code blocks, URLs, and shell commands.

The caveman-compress skill is a specialized utility within the JuliusBrussee/caveman repository designed to shrink documentation files for downstream LLM consumption. Understanding exactly what kind of data the caveman-compress skill handles ensures safe operation on your prose-oriented files without risking corruption of technical content.

Supported File Types and Extensions

The skill recognizes specific file extensions that indicate prose-oriented content. According to the skill definition in skills/caveman-compress/SKILL.md (lines 106-108), accepted file types include:

• .md (Markdown)
• .txt (Plain text)
• .typ and .typst (Typst documents)
• .tex (LaTeX files)
• Files without any extension

The skill explicitly excludes code and configuration files such as .py, .js, .json, .env, and .lock files to prevent accidental modification of source code or sensitive configurations.

Data Preservation Rules

The compression algorithm applies aggressive text reduction only to natural language, while strictly preserving technical elements. The preservation rules defined in skills/caveman-compress/SKILL.md include:

Code Blocks and Inline Technical Content

All code blocks—whether fenced with triple backticks or indented—remain completely untouched (lines 48-51). This includes:

• Inline code wrapped in single backticks (lines 52-54)
• Shell commands such as npm install, git commit, or docker build (lines 54-55)
• Environment variables like $HOME or $PATH (lines 55-58)

URLs, Paths, and Identifiers

The skill maintains the integrity of URLs and markdown links (lines 52-53), file paths (e.g., /src/..., ./config.yaml) (lines 53-54), technical terms, proper nouns, dates, and version numbers (lines 55-58).

Markdown Structure

Document organization remains intact, including:

• Headings and hierarchy
• Bullet lists and numbered lists
• Tables
• Front-matter

As specified in lines 59-65, the skill preserves the structural scaffolding that makes markdown documents navigable and machine-readable.

Compression Scope and Boundaries

The skill targets specific linguistic patterns for reduction. According to lines 40-47 and 66-70, caveman-compress removes:

• Filler words and pleasantries
• Redundant phrases
• Connective fluff (e.g., "however", "additionally", "in order to")

When processing files that mix prose with code, the skill only rewrites the prose sections while leaving code sections verbatim (lines 107-109). Anything inside a code fence is copied exactly without shortening, reordering, or comment removal (lines 73-80).

Safety Mechanisms and Backup Strategy

Before applying any modifications, the skill creates a safety backup. As implemented in skills/caveman-compress/scripts/compress.py (lines 6-7, 110-111), the original file is preserved as <filename>.original.md in the same directory. This ensures you can always recover the uncompressed version if the output requires manual adjustment.

Practical Usage Examples

Running from the Command Line

Execute the compression script directly on supported files:

# From the repository root

python3 -m skills/caveman-compress/scripts/compress.py /absolute/path/to/CLAUDE.md

The script performs the following operations:

1. Detects the file type (markdown)
2. Calls Claude to produce a compressed version
3. Validates that all code blocks, URLs, and commands remain intact
4. Writes the compressed text back to the original file
5. Stores a backup as CLAUDE.md.original.md

Before and After Compression

Original (example.md):

# Project Overview

You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production.

```bash
npm install
npm test

**Compressed ([`example.md`](https://github.com/JuliusBrussee/caveman/blob/main/example.md) after caveman-compress):**

```markdown

# Project Overview

Run tests before push to main. Catch bugs early, prevent broken prod deploys.

```bash
npm install
npm test

Notice that the heading and code block remain unchanged, while the verbose prose has been condensed to essential actions only.

### Using the Slash Command

Within the caveman environment, trigger compression using:

```text
/caveman-compress README.md

This command invokes the same pipeline defined in skills/caveman-compress/scripts/compress.py, updating README.md in-place while preserving the backup file.

Summary

• The caveman-compress skill handles natural-language memory files including .md, .txt, .typ, .typst, and .tex extensions.
• It preserves all technical content including code blocks, inline code, URLs, file paths, shell commands, and markdown structure.
• The skill removes only filler words and redundant prose, reducing token count for LLM contexts.
• Safety backups are automatically created as <filename>.original.md before any modifications occur.
• Code and configuration files (.py, .js, .json, .env, .lock) are explicitly excluded from processing.

Frequently Asked Questions

Does caveman-compress modify code blocks within markdown files?

No. According to the skill definition in skills/caveman-compress/SKILL.md (lines 73-80), anything inside a code fence is copied exactly without shortening, reordering, or comment removal. Both fenced and indented code blocks remain completely untouched during compression.

What file extensions are explicitly excluded from caveman-compress?

The skill refuses to process files with extensions that indicate source code or configuration data. As defined in lines 106-108, excluded extensions include .py, .js, .json, .env, .lock, and similar technical file types. Only prose-oriented extensions like .md, .txt, .typ, .typst, and .tex are accepted.

How does caveman-compress protect against data loss?

Before writing any compressed content, the skill automatically creates a backup of the original file with the suffix .original.md (lines 6-7, 110-111). This backup mechanism ensures you can always restore the uncompressed version if the algorithm produces undesirable results or removes context you wish to preserve.

Can caveman-compress handle files with mixed prose and code content?

Yes. When processing files that contain both natural language and code, the skill intelligently identifies boundaries and only compresses the prose sections (lines 107-109). Code blocks, inline code snippets, and technical syntax remain in their original form, allowing you to safely compress documentation that includes embedded examples or configuration snippets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}