Can Caveman Compress Text in Non-English Languages Without Translation?

Yes. Caveman's compression pipeline is fully language-agnostic and preserves the original language verbatim while reducing only stylistic bloat.

When working with multilingual documentation, you need a compression tool that respects linguistic boundaries. The Caveman project (JuliusBrussee/caveman) solves this by treating compression as a style-only operation, ensuring you can compress text in non-English languages without any risk of translation or character alteration.

How Caveman Detects Compressible Files

Caveman determines whether to compress a file based purely on its extension and content type, not the human language of the text. This language-agnostic detection happens in skills/caveman-compress/scripts/detect.py, which classifies files as "natural language" regardless of whether they contain English, Portuguese, or Japanese text.

The Compressible Extensions List

The COMPRESSIBLE_EXTENSIONS set defines which file types enter the compression pipeline:

COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst", ".typ", ".typst", ".tex"}

As implemented in detect.py, this check runs before any content analysis occurs. If your file matches one of these extensions, the system proceeds to compression without attempting to detect the specific language of the prose.

Strict Preservation Rules in the Compression Pipeline

Once a file is flagged for compression, skills/caveman-compress/scripts/compress.py orchestrates the process by building a prompt with strict "do NOT modify" constraints. These rules explicitly forbid the LLM from altering raw text content, including non-English characters.

The build_compress_prompt Function

Inside compress.py, the build_compress_prompt function constructs a prompt containing these mandatory safeguards:

STRICT RULES:
- Do NOT modify anything inside ``` code blocks
- Do NOT modify anything inside inline backticks
- Preserve ALL URLs exactly
- Preserve ALL headings exactly
- Preserve file paths and commands

These constraints ensure that Claude shrinks only stylistic verbosity—removing filler words and simplifying sentence structure—while keeping every non-English character, kanji, or accent mark intact.

The "Speak Your Tongue" Guarantee

According to the official README.md documentation, Caveman explicitly promises never to translate content:

Speak your tongue. Caveman keeps your language. Write Portuguese, caveman grunt Portuguese. Spanish, French, same. It compresses the style, never translates.

This architectural decision means the compression algorithm treats Unicode text identically regardless of language family.

Practical Examples for Multilingual Compression

You can invoke compression on non-English files using the standard CLI command. The pipeline handles the language preservation automatically.

Compress a Portuguese markdown file:

caveman-compress docs/guia_pt.md

Compress a Japanese text file while preserving all kanji and kana:

caveman-compress notes/日本語メモ.txt

Both commands execute the same three-step process:

1. detect.py returns natural_language based on the file extension
2. compress.py builds the constraint-heavy prompt via build_compress_prompt
3. Claude returns a reduced-token version containing the original non-English text unchanged

To inspect the exact prompt sent to Claude for debugging purposes:

python -c "from skills.caveman-compress.scripts.compress import build_compress_prompt; \
print(build_compress_prompt('Este é um texto em português que será comprimido.'))"

Summary

• Language-agnostic detection: detect.py uses file extensions, not language analysis, to identify compressible files.
• Strict preservation rules: The prompt in compress.py explicitly forbids modifying raw text, URLs, headings, and code blocks.
• No translation risk: The README's "Speak your tongue" section guarantees Caveman only compresses style, never translates languages.
• Unicode support: The pipeline handles Japanese, Chinese, Arabic, and other non-Latin scripts without character alteration.

Frequently Asked Questions

Will Caveman accidentally translate my Spanish text to English?

No. Caveman explicitly forbids translation in its compression logic. According to the README.md "Speak your tongue" section, the tool compresses stylistic elements only, ensuring Spanish remains Spanish and French remains French throughout the process.

Does Caveman support Japanese, Chinese, and Arabic characters?

Yes. The compression pipeline handles full Unicode text, including kanji, hanzi, and right-to-left scripts. Because detect.py classifies files based on extension rather than character encoding, and compress.py instructs the LLM to preserve all raw text exactly, your non-Latin characters remain untouched.

How does Caveman prevent the LLM from rewriting my content?

The build_compress_prompt function in compress.py injects strict constraints into every request: "Do NOT modify anything inside code blocks," "Preserve ALL URLs exactly," and "Preserve file paths and commands." These rules apply to all text content, ensuring the model only removes verbosity, not substance.

What file types can I compress with Caveman?

Caveman processes files matching the COMPRESSIBLE_EXTENSIONS set defined in detect.py, which includes .md, .txt, .markdown, .rst, .typ, .typst, and .tex files. Any natural language file with these extensions enters the compression pipeline regardless of the human language used.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}