How to Create and Use Compression Contexts (CCtx) in Zstandard

You create a compression context in Zstandard by calling ZSTD_createCCtx(), optionally configure it with ZSTD_CCtx_setParameter(), then execute compression using either the one-shot ZSTD_compressCCtx() or the streaming sequence of ZSTD_compressBegin(), ZSTD_compressContinue(), and ZSTD_compressEnd(), and finally release resources with ZSTD_freeCCtx().

Creating and using compression contexts (CCtx) in the facebook/zstd library allows applications to reuse internal buffers and maintain compression state across multiple operations. A ZSTD_CCtx structure encapsulates all parameters, history, and working memory needed to compress data, eliminating allocation overhead when processing multiple buffers or streams.

What Is a Zstandard Compression Context?

A compression context (ZSTD_CCtx) is an opaque structure declared in lib/zstd.h that stores the entire state of a compression operation. According to the facebook/zstd source code, this includes compression level settings, dictionary references, internal buffers, and stream state. Using a context enables advanced features like streaming compression, dictionary attachment, and parameter tuning that are not available in the simple one-shot ZSTD_compress() function.

Creating and Initializing a CCtx

The lifecycle begins with context allocation. The standard method uses ZSTD_createCCtx(), which allocates memory using the library's default allocator.

#include "zstd.h"

ZSTD_CCtx* cctx = ZSTD_createCCtx();
if (cctx == NULL) {
    // Handle allocation failure
}

For applications requiring custom memory management, the library provides ZSTD_createCCtx_advanced(), declared in lib/zstd.h, which accepts a ZSTD_customMem structure containing user-defined allocation and deallocation functions. Both functions return a pointer to an opaque context object or NULL on failure.

Configuring Compression Parameters

Before compression, you can tune the context using ZSTD_CCtx_setParameter(). This function, defined in lib/zstd.h, accepts parameter enums such as ZSTD_c_compressionLevel, ZSTD_c_windowLog, and ZSTD_c_checksumFlag.

// Set compression level to 5
ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, 5);

// Enable checksum verification
ZSTD_CCtx_setParameter(cctx, ZSTD_c_checksumFlag, 1);

To use a pre-trained dictionary, call ZSTD_CCtx_loadDictionary() or ZSTD_CCtx_refCDict() before starting compression. These functions are implemented in lib/compress/zstd_compress_internal.h and allow the context to reference external dictionary data without copying it into internal storage.

Performing Compression

Zstandard offers two primary patterns for context-based compression: one-shot operations for discrete buffers and streaming operations for large or continuous data sources.

One-Shot Compression

For scenarios where the entire input is available in memory, use ZSTD_compressCCtx(). This convenience function handles the full sequence of begin, continue, and end operations internally.

const char* src = "Data to compress";
size_t srcSize = strlen(src);
size_t dstCapacity = ZSTD_compressBound(srcSize);
void* dst = malloc(dstCapacity);

size_t compressedSize = ZSTD_compressCCtx(
    cctx, dst, dstCapacity, src, srcSize, 5
);

if (ZSTD_isError(compressedSize)) {
    fprintf(stderr, "Error: %s\n", ZSTD_getErrorName(compressedSize));
}

The function signature in lib/zstd.h specifies:

• cctx: The compression context
• dst / dstCapacity: Output buffer and its size
• src / srcSize: Input data and length
• The final int parameter overrides the compression level set in the context for this specific operation only.

Streaming Compression

For large files or network streams, use the explicit streaming API to process data in chunks. This approach is utilized in tests/zstreamtest.c for robust handling of arbitrary input sizes.

#define CHUNK_SIZE 65536

FILE* fin = fopen("input.dat", "rb");
FILE* fout = fopen("output.zst", "wb");

void* inBuf = malloc(CHUNK_SIZE);
void* outBuf = malloc(ZSTD_compressBound(CHUNK_SIZE));

// Initialize the stream
size_t initResult = ZSTD_compressBegin(cctx, 3);  // Level 3
if (ZSTD_isError(initResult)) { /* handle error */ }

size_t bytesRead;
while ((bytesRead = fread(inBuf, 1, CHUNK_SIZE, fin)) > 0) {
    // Compress current chunk
    size_t compSize = ZSTD_compressContinue(
        cctx, outBuf, ZSTD_compressBound(CHUNK_SIZE), inBuf, bytesRead
    );
    
    if (ZSTD_isError(compSize)) {
        fprintf(stderr, "Compression error: %s\n", ZSTD_getErrorName(compSize));
        break;
    }
    
    fwrite(outBuf, 1, compSize, fout);
}

// Finalize the frame
size_t endSize = ZSTD_compressEnd(cctx, outBuf, ZSTD_compressBound(0), NULL, 0);
if (!ZSTD_isError(endSize)) {
    fwrite(outBuf, 1, endSize, fout);
}

free(inBuf);
free(outBuf);

The streaming workflow requires three distinct phases:

1. Begin: ZSTD_compressBegin() (or ZSTD_compressBegin_usingDict()) initializes the frame header and prepares internal state.
2. Continue: ZSTD_compressContinue() processes input chunks and writes compressed blocks to the output buffer.
3. End: ZSTD_compressEnd() writes the frame epilogue, including any remaining buffered data and the checksum if enabled.

Resetting and Reusing Contexts

To compress multiple independent streams without destroying the context, call ZSTD_CCtx_reset(). This function clears internal state while preserving allocated memory, avoiding the overhead of ZSTD_freeCCtx() followed by ZSTD_createCCtx().

// Reset for a new compression session
ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only);

// Alternative: Reset both session and parameters to defaults
ZSTD_CCtx_reset(cctx, ZSTD_reset_session_and_parameters);

The internal implementation in lib/compress/zstd_compress_internal.h ensures that dictionary references and parameter overrides are cleared according to the reset mode specified.

Cleaning Up Resources

When compression operations are complete, release the context to prevent memory leaks.

size_t freeResult = ZSTD_freeCCtx(cctx);
if (freeResult != 0) {
    // Context was not fully freed (should not happen with valid cctx)
}

Unlike standard library free(), ZSTD_freeCCtx() returns the total size of freed memory, which can be useful for debugging or memory tracking in applications defined in examples/common.h.

Summary

• Allocation: Use ZSTD_createCCtx() in lib/zstd.h to allocate a compression context, or ZSTD_createCCtx_advanced() for custom allocators.
• Configuration: Adjust compression parameters with ZSTD_CCtx_setParameter() or attach dictionaries using ZSTD_CCtx_loadDictionary().
• Execution: Choose ZSTD_compressCCtx() for one-shot operations, or the streaming trio of ZSTD_compressBegin(), ZSTD_compressContinue(), and ZSTD_compressEnd() for chunked processing.
• Reuse: Reset contexts between independent streams with ZSTD_CCtx_reset() to avoid allocation overhead.
• Cleanup: Always call ZSTD_freeCCtx() to release resources when the context is no longer needed.

Frequently Asked Questions

What is the difference between ZSTD_createCCtx and ZSTD_createCCtx_advanced?

ZSTD_createCCtx() uses the library's default memory allocator defined at compile time, while ZSTD_createCCtx_advanced() accepts a ZSTD_customMem structure containing function pointers for custom allocation, reallocation, and deallocation routines. The advanced variant is essential for applications that manage memory pools or require allocation tracking, as implemented in specialized wrappers like zlibWrapper/examples/zwrapbench.c.

Can I reuse a ZSTD_CCtx for multiple independent compressions?

Yes. After finishing a compression with ZSTD_compressEnd(), call ZSTD_CCtx_reset(cctx, ZSTD_reset_session_only) to clear the stream state while preserving allocated buffers. This allows you to start a new compression immediately without the overhead of freeing and recreating the context, a pattern demonstrated in tests/zstreamtest.c for high-throughput scenarios.

When should I use streaming compression instead of one-shot compression?

Use the streaming API (ZSTD_compressBegin, ZSTD_compressContinue, ZSTD_compressEnd) when processing data larger than available memory, handling network streams of unknown size, or compressing files that exceed your system's RAM. The one-shot ZSTD_compressCCtx() is optimal for discrete, in-memory buffers where the entire input is known upfront and fits comfortably in addressable memory.

How do I attach a dictionary to a compression context?

Call ZSTD_CCtx_loadDictionary(cctx, dict, dictSize) before ZSTD_compressBegin() (for streaming) or before ZSTD_compressCCtx() (for one-shot). This function copies dictionary data into the context's internal storage. Alternatively, use ZSTD_CCtx_refCDict() to reference a pre-digested dictionary object without copying, which is more efficient when compressing multiple streams with the same dictionary, as noted in the function declarations within lib/zstd.h.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}