# Can Caveman Compress Text in Non-English Languages Without Translation?

> Discover if Caveman can compress non-English text without translation. Learn how Caveman's language-agnostic pipeline preserves original text while reducing bloat.

- Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman)
- Tags: deep-dive
- Published: 2026-07-08

---

**Yes.** Caveman's compression pipeline is fully language-agnostic and preserves the original language verbatim while reducing only stylistic bloat.

When working with multilingual documentation, you need a compression tool that respects linguistic boundaries. The Caveman project (`JuliusBrussee/caveman`) solves this by treating compression as a style-only operation, ensuring you can compress text in non-English languages without any risk of translation or character alteration.

## How Caveman Detects Compressible Files

Caveman determines whether to compress a file based purely on its extension and content type, not the human language of the text. This language-agnostic detection happens in [`skills/caveman-compress/scripts/detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/detect.py), which classifies files as "natural language" regardless of whether they contain English, Portuguese, or Japanese text.

### The Compressible Extensions List

The `COMPRESSIBLE_EXTENSIONS` set defines which file types enter the compression pipeline:

```python
COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst", ".typ", ".typst", ".tex"}

```

As implemented in [`detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/detect.py), this check runs before any content analysis occurs. If your file matches one of these extensions, the system proceeds to compression without attempting to detect the specific language of the prose.

## Strict Preservation Rules in the Compression Pipeline

Once a file is flagged for compression, [`skills/caveman-compress/scripts/compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/skills/caveman-compress/scripts/compress.py) orchestrates the process by building a prompt with strict "do NOT modify" constraints. These rules explicitly forbid the LLM from altering raw text content, including non-English characters.

### The build_compress_prompt Function

Inside [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py), the `build_compress_prompt` function constructs a prompt containing these mandatory safeguards:

```text
STRICT RULES:
- Do NOT modify anything inside ``` code blocks
- Do NOT modify anything inside inline backticks
- Preserve ALL URLs exactly
- Preserve ALL headings exactly
- Preserve file paths and commands

```

These constraints ensure that Claude shrinks only stylistic verbosity—removing filler words and simplifying sentence structure—while keeping every non-English character, kanji, or accent mark intact.

### The "Speak Your Tongue" Guarantee

According to the official `README.md` documentation, Caveman explicitly promises never to translate content:

> **Speak your tongue.** Caveman keeps your language. Write Portuguese, caveman grunt Portuguese. Spanish, French, same. It compresses the *style*, never translates.

This architectural decision means the compression algorithm treats Unicode text identically regardless of language family.

## Practical Examples for Multilingual Compression

You can invoke compression on non-English files using the standard CLI command. The pipeline handles the language preservation automatically.

Compress a Portuguese markdown file:

```bash
caveman-compress docs/guia_pt.md

```

Compress a Japanese text file while preserving all kanji and kana:

```bash
caveman-compress notes/日本語メモ.txt

```

Both commands execute the same three-step process:

1. `detect.py` returns `natural_language` based on the file extension
2. `compress.py` builds the constraint-heavy prompt via `build_compress_prompt`
3. Claude returns a reduced-token version containing the original non-English text unchanged

To inspect the exact prompt sent to Claude for debugging purposes:

```python
python -c "from skills.caveman-compress.scripts.compress import build_compress_prompt; \
print(build_compress_prompt('Este é um texto em português que será comprimido.'))"

```

## Summary

- **Language-agnostic detection**: [`detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/detect.py) uses file extensions, not language analysis, to identify compressible files.
- **Strict preservation rules**: The prompt in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py) explicitly forbids modifying raw text, URLs, headings, and code blocks.
- **No translation risk**: The README's "Speak your tongue" section guarantees Caveman only compresses style, never translates languages.
- **Unicode support**: The pipeline handles Japanese, Chinese, Arabic, and other non-Latin scripts without character alteration.

## Frequently Asked Questions

### Will Caveman accidentally translate my Spanish text to English?

No. Caveman explicitly forbids translation in its compression logic. According to the [`README.md`](https://github.com/JuliusBrussee/caveman/blob/main/README.md) "Speak your tongue" section, the tool compresses stylistic elements only, ensuring Spanish remains Spanish and French remains French throughout the process.

### Does Caveman support Japanese, Chinese, and Arabic characters?

Yes. The compression pipeline handles full Unicode text, including kanji, hanzi, and right-to-left scripts. Because [`detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/detect.py) classifies files based on extension rather than character encoding, and [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py) instructs the LLM to preserve all raw text exactly, your non-Latin characters remain untouched.

### How does Caveman prevent the LLM from rewriting my content?

The `build_compress_prompt` function in [`compress.py`](https://github.com/JuliusBrussee/caveman/blob/main/compress.py) injects strict constraints into every request: "Do NOT modify anything inside code blocks," "Preserve ALL URLs exactly," and "Preserve file paths and commands." These rules apply to all text content, ensuring the model only removes verbosity, not substance.

### What file types can I compress with Caveman?

Caveman processes files matching the `COMPRESSIBLE_EXTENSIONS` set defined in [`detect.py`](https://github.com/JuliusBrussee/caveman/blob/main/detect.py), which includes `.md`, `.txt`, `.markdown`, `.rst`, `.typ`, `.typst`, and `.tex` files. Any natural language file with these extensions enters the compression pipeline regardless of the human language used.