# How caveman-compress Preserves Code Blocks and URLs When Compressing Markdown

> Discover how caveman-compress protects code blocks and URLs during Markdown compression. Learn about its unique preservation techniques.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: how-to-guide
- Published: 2026-07-07

---

**caveman-compress preserves code blocks and URLs by isolating YAML front-matter, enforcing strict structural rules via LLM prompting, stripping outer markdown fences, and running post-compression validation with automatic repair.**

The `caveman-compress` tool in the [JuliusBrussee/caveman](https://github.com/JuliusBrussee/caveman) repository compresses natural-language prose in Markdown files while ensuring that every code snippet, inline backtick, URL, heading, and file path remains untouched. This article examines the five-step pipeline implemented in [`skills/caveman-compress/scripts/compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/compress.py) that guarantees structural integrity during compression.

## Step 1: Isolate YAML Front-Matter with split_frontmatter()

Before any compression occurs, `caveman-compress` separates the document's YAML front-matter from its body to prevent accidental modification of metadata.

The `split_frontmatter()` function (lines 28–40 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py)) detects content before the first `---` delimiter, removes it from the compression input, and stores it for later reassembly. This ensures that Jekyll-style headers, TOML blocks, or other front-matter configurations remain exactly as they were in the original file.

## Step 2: Enforce Structural Rules via build_compress_prompt()

The core protection mechanism lies in the `build_compress_prompt()` function (lines 73–80 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py)). This function constructs a **"preserve-structure" prompt** that explicitly instructs Claude not to modify:

- Code fences (triple backticks)
- Inline backticks
- URLs
- Headings
- File paths

By sending only the document body (without front-matter) alongside these strict prohibitions, the LLM returns compressed natural-language prose while leaving all technical elements intact.

## Step 3: Remove LLM Wrapper Fencing with strip_llm_wrapper()

Large language models occasionally wrap their entire response in markdown code fences for emphasis. The `strip_llm_wrapper()` function (lines 17–22 and 106–112 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py)) uses the `OUTER_FENCE_REGEX` pattern to detect and remove these outer wrappers.

This step is critical because it prevents nested fencing that could corrupt the document's actual code blocks. After stripping the outer fence, inner code fences remain exactly as they were in the original text.

## Step 4: Validate and Repair with validate() and build_fix_prompt()

After receiving the compressed text, `caveman-compress` runs a validation loop to catch any violations. The `validate()` function (lines 90–101 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py)) checks for specific errors including:

- **"Code block mismatch"**: When opening and closing fence counts differ
- **"Missing URL"**: When URLs detected in the original are absent in the output
- **"Heading alteration"**: When heading structures change unexpectedly

If `validate()` detects errors, the system calls `build_fix_prompt()` (lines 190–205 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py)) to generate a repair request. This second prompt asks Claude to restore the exact original fragments while maintaining the compression. The process retries until validation passes or reaches the maximum attempt limit.

## Step 5: Reassemble the Final Document

Once validation passes, the tool writes the final output. Lines 96–98 and 129–132 in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py) concatenate the preserved front-matter with the validated compressed body using `filepath.write_text(frontmatter + compressed_body)`.

This guarantee means that a code block like:

```python
def process_data():
    return "compressed"

```

Or a URL like `https://example.com/api` will appear identically in the output file, even as surrounding paragraphs shrink significantly.

## Using caveman-compress: CLI and Programmatic Examples

You can invoke the compression engine from the command line or integrate it directly into Python applications.

Compress a single file in place:

```bash

# From the repository root

$ python skills/caveman-compress/scripts/compress.py docs/guide.md
Processing: /path/to/docs/guide.md
Compressing with Claude...
Validation passed

```

Use the tool programmatically:

```python
from pathlib import Path
from skills.caveman_compress.scripts.compress import compress_file

# Returns True if compression succeeded, False otherwise

result = compress_file(Path("docs/guide.md"))
print("Compressed?", result)

```

The `compress_file()` function orchestrates the entire pipeline, returning a boolean status that indicates whether the file was successfully compressed and validated.

## Key Source Files and Their Roles

| File | Purpose |
|------|---------|
| [`skills/caveman-compress/scripts/compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/compress.py) | Main orchestration containing `split_frontmatter()`, `build_compress_prompt()`, `strip_llm_wrapper()`, `validate()`, and `build_fix_prompt()` |
| [`skills/caveman-compress/scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/detect.py) | Implements `should_compress()` to determine if a file contains enough natural language to warrant compression |
| [`skills/caveman-compress/scripts/validate.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/validate.py) | Parses compressed output and emits specific error codes like "code block mismatch" or "missing URL" for the repair loop |
| [`skills/caveman-compress/SKILL.md`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/SKILL.md) | Documentation covering usage patterns and architectural decisions |

## Summary

- **Front-matter isolation** via `split_frontmatter()` protects metadata before compression begins.
- **Strict LLM prompting** in `build_compress_prompt()` explicitly forbids modifying code blocks, URLs, and structural elements.
- **Wrapper removal** using `strip_llm_wrapper()` and `OUTER_FENCE_REGEX` prevents fence nesting artifacts.
- **Validation and repair** through `validate()` and `build_fix_prompt()` create a self-healing system that restores exact original fragments when errors occur.
- **Verbatim reassembly** ensures the final file contains unchanged technical elements alongside compressed prose.

## Frequently Asked Questions

### What happens if Claude accidentally modifies a code block?

If `validate()` detects a code block mismatch or alteration, `caveman-compress` triggers a repair loop using `build_fix_prompt()`. This second prompt instructs Claude to restore the exact original code fence while maintaining the compression of surrounding text. The system retries until validation passes or exhausts the configured attempt limit.

### How does caveman-compress handle YAML front-matter?

The `split_frontmatter()` function removes anything before the first `---` delimiter before sending content to the LLM. This front-matter is stored separately and concatenated back to the compressed body during the final write operation, ensuring that metadata fields remain completely untouched.

### Can I use caveman-compress programmatically in Python?

Yes. Import `compress_file()` from `skills.caveman_compress.scripts.compress` and pass a `pathlib.Path` object to the function. It returns a boolean indicating success and handles the entire pipeline including validation and repair automatically.

### What types of validation errors trigger the repair loop?

The `validate()` function specifically checks for code block mismatches (uneven fence counts), missing URLs (when original URLs disappear from output), and altered headings. Any of these violations feed into `build_fix_prompt()` to generate targeted repair instructions.