# How the Artifact Format and Zstd Compression Work in Codebase-Memory-MCP

> Discover how Codebase-Memory-MCP uses Zstd compression and artifact formats. Learn about SQLite packaging, atomic writes, and frame-size validation for corrupted file prevention.

- Repository: [Martin Vogel/codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)
- Tags: internals
- Published: 2026-07-13

---

**Codebase-Memory-MCP packages SQLite databases into Zstandard-compressed artifacts with JSON metadata and Git-safe attributes, using atomic writes and frame-size validation to prevent corruption and resource exhaustion.**

The Codebase-Memory-MCP project enables teams to share code intelligence through a portable **artifact format** that encapsulates entire SQLite databases. This system leverages **Zstandard (zstd)** compression to minimize storage footprint while maintaining fast export and import operations. Understanding how the artifact structures its compressed payload, metadata, and Git integration is essential for teams sharing codebases across repositories.

## Anatomy of the Artifact Format

Each artifact consists of three distinct components that work together to ensure portability and safety.

### The Compressed Payload (*.zst)

The core of the artifact is a Zstandard-compressed snapshot of the SQLite database. In [`src/pipeline/artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/pipeline/artifact.c), the `prepare_snapshot_db()` function (lines 443-475) creates this snapshot using `VACUUM INTO` to generate a clean, compact database copy. For "best" quality exports, the system drops user-created indexes to improve compressibility before compression begins.

### JSON Metadata (artifact.json)

Alongside the compressed file sits [`artifact.json`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.json), generated by `write_metadata()` (lines 31-72 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)). This JSON document stores critical context including the schema version, original uncompressed size, compression level, Git HEAD hash, timestamps, and node/edge counts. This metadata enables version validation and integrity checks during import.

### Git Configuration (.gitattributes)

The `ensure_gitattributes()` function (lines 76-100 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)) generates a `.gitattributes` entry marking the artifact as binary and forcing the `merge=ours` driver. This prevents Git from attempting to merge conflicting artifact versions, automatically keeping the local copy during merge operations.

## Export Pipeline: Creating Artifacts

The export process follows a rigorous pipeline to ensure atomic, verifiable artifacts.

### Database Snapshot Preparation

The export begins with `prepare_snapshot_db()` copying the live database to a temporary file. This isolation ensures the compression operates on a consistent point-in-time snapshot without locking the production database.

### Compression Strategy and Levels

Compression is handled by the thin wrapper functions in [`internal/cbm/zstd_store.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/internal/cbm/zstd_store.c). The `cbm_zstd_compress_bound()` function (lines 34-36) calculates the maximum compressed size using `ZSTD_compressBound()`, ensuring adequate buffer allocation. The actual compression occurs in `cbm_zstd_compress()` (lines 10-16), which supports two quality levels defined in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c) (lines 10-13): **ART_ZSTD_FAST** (level 3) for speed and **ART_ZSTD_BEST** (level 9) for maximum compression.

### Atomic File Operations

Both the compressed payload and metadata are written using `write_file_atomic()` (lines 59-89 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)). This helper ensures that partially written files never appear on disk, preventing repository corruption if the export process is interrupted.

## Import Pipeline: Restoring Artifacts

Importing artifacts prioritizes security through frame validation and bounded resource allocation.

### Frame-Size Validation and Safety Bounds

Rather than trusting the `original_size` field in metadata, the importer uses `cbm_zstd_frame_content_size()` (lines 26-32 in [`zstd_store.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/zstd_store.c)) to read the content-size directly from the zstd frame header. This value must be non-zero, must not exceed **ART_MAX_DECOMPRESSED_BYTES** (approximately 64 GB, defined at lines 23-24 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)), and must match the metadata record. This design prevents maliciously crafted archives from triggering memory exhaustion attacks.

### Decompression and Integrity Verification

The `cbm_zstd_decompress()` function (lines 18-23 in [`zstd_store.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/zstd_store.c)) performs the actual decompression into a precisely allocated buffer sized to the validated frame content. After decompression, `cbm_store_check_integrity_deep()` (lines 704-708 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)) executes a full SQLite integrity check, rejecting any corrupted artifacts before they enter the cache.

## Security and Safety Design

The artifact format implements multiple defense layers. **Deterministic sizing** via frame headers eliminates reliance on attacker-controlled metadata fields. **Bounded resources** through `ART_MAX_DECOMPRESSED_BYTES` caps memory allocation. **Atomic operations** guarantee transactional file writes, while the **Git attributes** configuration prevents merge conflicts that could corrupt the artifact.

## Code Examples

### Exporting an Artifact (Fast)

```c
// Export a DB snapshot with fast (zstd‑3) compression
int rc = cbm_artifact_export(
    "/path/to/db.sqlite",          // db_path
    "/path/to/repo",               // repo_path
    "my-project",                  // project_name
    CBM_ARTIFACT_FAST);            // quality flag (fast)

if (rc != 0) {
    fprintf(stderr, "Export failed: %s\n",
            cbm_artifact_export_last_error());
}

```

### Importing an Artifact

```c
// Import the artifact into the cache DB
int rc = cbm_artifact_import(
    "/path/to/repo",          // repo_path containing *.zst + artifact.json
    "/path/to/cache.db");     // cache_db_path (final DB location)

if (rc != 0) {
    fprintf(stderr, "Import failed\n");
}

```

### Direct Zstd Wrapper Usage

```c
// Simple wrapper usage – compress a buffer
size_t bound = cbm_zstd_compress_bound(src_len);
char *dst = malloc(bound);
int compressed_len = cbm_zstd_compress(src, src_len, dst, bound, 9); // level 9

// Decompress back
size_t orig_size = cbm_zstd_frame_content_size(dst, compressed_len);
char *orig = malloc(orig_size);
int64_t decompressed_len = cbm_zstd_decompress(dst, compressed_len, orig, orig_size);

```

## Summary

- The artifact format combines a **Zstandard-compressed payload** (`*.zst`), **JSON metadata**, and **Git attributes** configuration for safe team sharing.
- Export uses `prepare_snapshot_db()` and `cbm_zstd_compress()` with configurable levels (3 or 9), writing atomically via `write_file_atomic()`.
- Import validates frame sizes using `cbm_zstd_frame_content_size()` against `ART_MAX_DECOMPRESSED_BYTES` (64 GB limit) before decompression.
- Integrity verification runs `cbm_store_check_integrity_deep()` to ensure the SQLite database is corruption-free.
- The `.gitattributes` entry with `merge=ours` prevents Git merge conflicts on binary artifacts.

## Frequently Asked Questions

### What compression levels does Codebase-Memory-MCP support?

The system supports two compression levels defined in [`src/pipeline/artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/src/pipeline/artifact.c): **ART_ZSTD_FAST** (level 3) for quick exports and **ART_ZSTD_BEST** (level 9) for maximum space savings. The "best" mode also drops user-created indexes before compression to improve the compression ratio.

### How does the importer protect against maliciously crafted artifacts?

Instead of trusting the metadata's `original_size` field, the importer reads the content size directly from the zstd frame header using `cbm_zstd_frame_content_size()`. It validates this against `ART_MAX_DECOMPRESSED_BYTES` (approximately 64 GB) and the metadata record, preventing resource exhaustion attacks that would otherwise allocate excessive memory.

### Why does the artifact include a `.gitattributes` entry?

The `ensure_gitattributes()` function generates an entry marking the artifact as binary with `merge=ours`, which forces Git to always keep the local version during merges. This prevents accidental merge conflicts that could corrupt the compressed database or create invalid hybrid artifacts.

### Are file operations atomic to prevent corruption?

Yes. Both the compressed payload and JSON metadata are written using `write_file_atomic()` (lines 59-89 in [`artifact.c`](https://github.com/DeusData/codebase-memory-mcp/blob/main/artifact.c)), which ensures that files appear on disk only when fully written. This guarantees that partial writes or interrupted exports never leave the repository in an inconsistent state.