How to Get Detailed Feature Information for the caveman-compress Skill

The caveman-compress skill rewrites natural-language memory files into a terse "caveman" style while preserving technical elements, achieving approximately 46% token reduction through a validation-backed pipeline implemented in the scripts/ directory.

The caveman-compress skill is a specialized utility within the JuliusBrussee/caveman repository that optimizes natural-language memory files for token efficiency. This skill compresses prose in files like CLAUDE.md and todos.md by removing filler words and hedging while strictly preserving code blocks, URLs, and file paths. Understanding the complete feature set requires examining both the declarative SKILL.md specification and the implementation logic in the Python scripts.

Core Purpose and Trigger Mechanisms

The primary function of the caveman-compress skill is to reduce token count in natural-language memory files while maintaining semantic integrity. According to the skills/caveman-compress/SKILL.md specification in JuliusBrussee/caveman, the skill is invoked via the slash command /caveman-compress <filepath> within a Caveman-enabled chat environment, or by requesting the agent to "compress memory file".

Compression Rules and Preservation Guarantees

What Gets Removed

The compression algorithm targets specific linguistic patterns defined in SKILL.md. The skill strips articles (a, an, the), filler words (just, really, basically), pleasantries (sure, of course), hedging phrases (it might be worth...), redundant phrasing (in order to → to), and connective fluff (however, furthermore).

What Stays Preserved

The scripts/validate.py module enforces strict preservation guarantees. Code blocks (triple backtick fences), inline code (single backticks), URLs, file paths, commands, technical terms, proper nouns, dates, version numbers, and environment variables remain completely unchanged. Additionally, the skill preserves markdown structure including headings, bullet hierarchy, numbered lists, tables, and front-matter.

Process Flow and Pipeline Architecture

The execution pipeline follows a four-step process defined in skills/caveman-compress/scripts/__main__.py:

1. Detection: The scripts/detect.py module determines if the file extension is supported (.md, .txt, .typ, .typst, .tex, or no extension) and identifies mixed-content files.
2. Compression: The scripts/compress.py module calls Claude to perform the compression according to the rules defined in SKILL.md.
3. Validation: The scripts/validate.py script verifies that preserved elements (code fences, inline code, URLs) remain unchanged after compression.
4. Retry Logic: If validation fails, the CLI attempts a targeted fix rather than full recompression, retrying up to two times before returning an error.

Safety Boundaries and File Type Detection

The scripts/detect.py implementation enforces strict boundary conditions. The skill operates only on natural-language extensions and never touches source code files (.py, .js, .ts, .json, .yaml, .toml, .sh). It also skips any file ending with .original.md to prevent recursive processing.

According to skills/caveman-compress/SECURITY.md, the skill performs local file I/O only: it reads the supplied file, writes the compressed version, and creates a backup <file>.original.md. No network calls, subprocess executions, or external file access occur beyond the named file.

Performance Characteristics and Benchmarking

The scripts/benchmark.py suite measures token-saving performance across various file types. Benchmarks show an average 46% token reduction per memory file, with savings persisting across sessions. This metric validates the skill's effectiveness for managing large context windows in the JuliusBrussee/caveman ecosystem.

How to Access Implementation Details

To explore additional nuances such as validation logic, retry policies, or benchmark methodology, examine the heavily commented scripts in skills/caveman-compress/scripts/:

• Inspect validation flow: Review scripts/validate.py to see the regex patterns that enforce "preserve exactly" constraints for code fences and inline snippets.
• Read detection logic: Examine scripts/detect.py to understand accepted extensions and mixed-content handling.
• Review benchmark suite: Analyze scripts/benchmark.py for concrete token savings numbers across various file types.

Practical Usage Examples

Using the Slash Command

/caveman-compress CLAUDE.md

This overwrites CLAUDE.md with the compressed version and creates CLAUDE.original.md as a backup.

Running from the CLI

cd skills/caveman-compress
python3 -m scripts /absolute/path/to/CLAUDE.md

The scripts/__main__.py entry point executes the full detection-compression-validation pipeline.

Programmatic Invocation

from pathlib import Path
from skills.caveman_compress.scripts import compress

file_path = Path("/absolute/path/to/CLAUDE.md")
compress.run(file_path)

The compress.run wrapper in scripts/compress.py executes the same pipeline as the CLI.

Inspecting the Backup

cat CLAUDE.original.md

This displays the pre-compression source for audit purposes.

Summary

• The caveman-compress skill reduces tokens by approximately 46% by removing filler words and hedging from natural-language files while preserving all technical elements.
• The pipeline in scripts/__main__.py follows a detect-compress-validate-retry pattern with strict safety boundaries enforced by scripts/detect.py and scripts/validate.py.
• Only files with extensions .md, .txt, .typ, .typst, .tex, or no extension are processed; source code files are explicitly excluded.
• Original files are backed up as <filename>.original.md before overwriting, with all operations restricted to local file I/O.
• Detailed feature specifications reside in skills/caveman-compress/SKILL.md, while implementation logic is documented in the scripts/ directory.

Frequently Asked Questions

What file types does the caveman-compress skill support?

The skill processes files with extensions .md, .txt, .typ, .typst, .tex, or no extension, as implemented in scripts/detect.py. It explicitly excludes source code files including .py, .js, .ts, .json, .yaml, .toml, and .sh, and skips any file ending with .original.md to prevent recursive compression.

How does the validation system ensure technical content preservation?

The scripts/validate.py module checks that preserved elements—specifically code blocks (triple backtick fences), inline code (single backticks), and URLs—remain unchanged after compression. If validation fails, the system attempts a targeted fix rather than full recompression, retrying up to two times before returning an error to the user.

What happens to the original file when using caveman-compress?

Before overwriting, the skill automatically creates a backup copy named <filename>.original.md in the same directory as the source file. This backup contains the complete pre-compression content for audit purposes, and the skill skips any file already ending with .original.md to prevent processing its own backups.

Where can I find the complete specification for the caveman-compress skill?

The authoritative feature specification resides in skills/caveman-compress/SKILL.md within the JuliusBrussee/caveman repository. This document defines the purpose, trigger commands, compression rules, preservation guarantees, and boundary conditions. Implementation details are documented in the scripts/ subdirectory, particularly __main__.py, compress.py, validate.py, and detect.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}