# How to Build the Caveman Compression Engine CLI: Complete Setup Guide > Build the Caveman compression engine CLI easily by cloning the JuliusBrussee caveman repository and running go build. Get your native binary for Compress, Detect, Retrieve, and Stats operations. - Repository: [Julius Brussee/caveman](https://github.com/JuliusBrussee/caveman) - Tags: how-to-guide - Published: 2026-08-22 --- **Build the Caveman compression engine CLI by cloning the repository and running `go build ./engine/cmd/caveman-engine` to produce a native binary that implements the Compress, Detect, Retrieve, and Stats operations.** The Caveman compression engine is a self-contained Go binary designed for deterministic text compression in AI workflows. Learning how to build the Caveman compression engine CLI from source gives you full control over the compression pipeline, from the core engine logic to the SQLite-backed recovery store. This guide walks through the build process using the actual source structure found in the `JuliusBrussee/caveman` repository. ## Prerequisites Before building, ensure your environment meets the following requirements: - **Go 1.22 or later** – The project uses Go modules and requires a recent toolchain. - **Git** – Needed to clone the repository. Verify your Go installation: ```bash go version ``` ## Build Instructions Follow these steps to compile the native binary from the source files located in [`engine/cmd/caveman-engine/main.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/cmd/caveman-engine/main.go). 1. Clone the repository and navigate to the root directory: ```bash git clone https://github.com/JuliusBrussee/caveman.git cd caveman ``` 2. Download and verify dependencies: ```bash go mod tidy ``` 3. Build the CLI binary: ```bash go build ./engine/cmd/caveman-engine ``` This produces an executable at `./engine/cmd/caveman-engine/caveman-engine` (platform-specific naming may vary). 4. Verify the build: ```bash ./engine/cmd/caveman-engine/caveman-engine --help ``` ## Architectural Overview Understanding the source layout helps when customizing or debugging the build. ### Entry Point and Command Dispatch The CLI entry point resides in [[`engine/cmd/caveman-engine/main.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/cmd/caveman-engine/main.go)](https://github.com/JuliusBrussee/caveman/blob/main/engine/cmd/caveman-engine/main.go). The `main` function parses sub-commands (`compress`, `detect`, `retrieve`, `stats`) and dispatches to dedicated handlers like `runCompress` and `runDetect`. ### Engine Core The core logic lives in [`engine/engine.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/engine.go), which defines the `Engine` type. This struct provides the stable API methods: - `Compress` – Applies registered compressors to input bytes. - `RetrieveQuery` – Fetches original data from CCR handles. - `Detect` – Identifies content types. - `Stats` – Returns aggregate compression metrics. ### Compressor Registry The default set of 15 compressors (JSON, log, code, diff, and others) is registered by `compressors.Default()` in [`engine/compressors/registry.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/compressors/registry.go). Each compressor implements a pure byte transform, while the engine adds safety checks, token accounting, and optional recovery storage. ### Recovery Store (CCR) When lossy compression produces smaller payloads, the original bytes are stored in a local SQLite database at `~/.caveman/ccr.db`. The `openEngine` helper in [`main.go`](https://github.com/JuliusBrussee/caveman/blob/main/main.go) initializes this store via `ccr.Open(ccrPath())`. If the store is unavailable or exceeds budget constraints, the engine returns the original input unchanged. ## CLI Usage Examples Once built, the binary reads from **stdin** (capped at 64 MiB as defined by `maxStdinBytes` in [`main.go`](https://github.com/JuliusBrussee/caveman/blob/main/main.go)) and writes compressed output to **stdout**, with JSON statistics emitted to **stderr**. Compress a JSON payload: ```bash cat payload.json | ./engine/cmd/caveman-engine/caveman-engine compress > compressed.bin ``` Detect content type: ```bash echo "Hello" | ./engine/cmd/caveman-engine/caveman-engine detect ``` Retrieve original bytes using a CCR handle: ```bash ./engine/cmd/caveman-engine/caveman-engine retrieve abcdef12345 > original.json ``` Display aggregate statistics: ```bash ./engine/cmd/caveman-engine/caveman-engine stats ``` List compressor capabilities: ```bash ./engine/cmd/caveman-engine/caveman-engine registry ``` ## Runtime Configuration Configure the CLI behavior using environment variables: - **CAVEMAN_CCR_DB** – Override the default recovery store path (`~/.caveman/ccr.db`). - **CAVEMAN_ENGINE_BIN** – Path to the binary, respected by the JavaScript launcher. The CLI enforces a **64 MiB** input limit on stdin. Streams exceeding this limit are rejected with an error. ## Summary - Build the Caveman compression engine CLI using `go build ./engine/cmd/caveman-engine` from the repository root. - The binary entry point is [`engine/cmd/caveman-engine/main.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/cmd/caveman-engine/main.go), which dispatches to sub-command handlers. - Core compression logic resides in [`engine/engine.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/engine.go) with support from [`engine/compressors/registry.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/compressors/registry.go). - Recovery data is stored in a SQLite database managed by [`engine/ccr/store.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/ccr/store.go). - The CLI supports four stable operations: **Compress**, **Detect**, **Retrieve**, and **Stats**. ## Frequently Asked Questions ### What Go version is required to build the Caveman compression engine CLI? The project requires **Go 1.22 or later** to compile successfully. Verify your version with `go version` before building, as earlier versions may fail to resolve modern module dependencies. ### Where is the compression logic implemented in the source code? The core compression logic is implemented in [`engine/engine.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/engine.go), specifically within the `Engine` type methods. The `Compress` method orchestrates the 15 registered compressors defined in [`engine/compressors/registry.go`](https://github.com/JuliusBrussee/caveman/blob/main/engine/compressors/registry.go). ### How does the CLI handle data recovery for lossy compression? When a lossy compressor produces a smaller payload, the engine stores the original bytes in a SQLite database at `~/.caveman/ccr.db` (configurable via `CAVEMAN_CCR_DB`). The `retrieve` sub-command uses `ccr.Open()` and `RetrieveQuery` to fetch the original data using a unique handle. ### Can I install the binary globally on my system? Yes. Move the built binary to a directory in your `PATH`, or set the `CAVEMAN_ENGINE_BIN` environment variable to point to the binary location for use with the JavaScript launcher.