ZSTD_c_windowLog: Controlling Sliding Window Size in Zstandard Compression

ZSTD_c_windowLog is a compression parameter that defines the size of the sliding window used by the Zstandard encoder as 2^windowLog bytes, limiting how far back the compressor can search for matching sequences to improve compression ratio.

The ZSTD_c_windowLog parameter is a critical configuration option in the facebook/zstd library that directly impacts compression efficiency and memory consumption. In the Zstandard source code, this parameter governs the encoder's historical buffer, balancing the trade-off between compression ratio and resource usage. Understanding how to tune ZSTD_c_windowLog enables developers to optimize compression for workloads with long-range repetitive patterns.

What Is ZSTD_c_windowLog?

ZSTD_c_windowLog is defined in lib/zstd.h as an enumerable compression parameter that controls the window size, which is the amount of previously processed data the encoder retains in memory to find duplicate sequences. The actual window size in bytes is calculated as 2^windowLog, meaning a windowLog value of 20 creates a 1 MiB window (2^20), while a value of 22 yields a 4 MiB window.

The parameter accepts values between ZSTD_WINDOWLOG_MIN and ZSTD_WINDOWLOG_MAX. If you specify a value outside this range, the encoder clamps it to the nearest valid boundary rather than failing, though it may emit a warning during parameter validation as seen in the test suite in tests/paramgrill.c.

Impact on Compression Performance

Compression Ratio and Match Distance

The maximum match distance is directly constrained by the window size. A larger window allows the encoder to find and reference duplicate sequences that occurred further back in the data stream, which significantly improves compression ratio on files with long-range repetitive patterns such as log files or database dumps. Conversely, a small window limits matches to recent data only, potentially missing distant repetitions.

Memory Usage and Allocation

Memory consumption scales proportionally with the window size. The encoder allocates internal buffers sized to the window, so increasing ZSTD_c_windowLog from 20 to 26 (64 MiB) increases the encoder's memory footprint accordingly. This affects both the compression context (ZSTD_CCtx) and, critically, the decoder's requirements discussed below.

Interaction with hashLog and chainLog

As implemented in the compressor's parameter-adjustment logic, ZSTD_c_windowLog influences the default values of hashLog and chainLog. When the window is small, these logarithmic values are automatically reduced so that hash tables fit efficiently within the window constraints, ensuring that look-up operations remain performant relative to the available history.

Configuring ZSTD_c_windowLog in Practice

You configure the parameter using the ZSTD_CCtx_setParameter API before compression begins.

Setting a Custom Window Size

#include <zstd.h>
#include <stdio.h>

int main(void) {
    ZSTD_CCtx *cctx = ZSTD_createCCtx();
    if (!cctx) return 1;

    /* Request a 4 MiB sliding window (log2 = 22) */
    size_t err = ZSTD_CCtx_setParameter(cctx, ZSTD_c_windowLog, 22);
    if (ZSTD_isError(err)) {
        fprintf(stderr, "Error setting windowLog: %s\n", ZSTD_getErrorName(err));
        return 1;
    }

    /* ... proceed with compression using ZSTD_compress2 or stream APIs ... */

    ZSTD_freeCCtx(cctx);
    return 0;
}

Querying the Effective Window Size

After setting parameters (or using defaults), retrieve the actual value that will be used:

int effectiveLog;
ZSTD_CCtx_getParameter(cctx, ZSTD_c_windowLog, &effectiveLog);
printf("Effective windowLog = %d (window = %zu bytes)\n",
       effectiveLog, (size_t)1 << effectiveLog);

The tests/zstreamtest.c file contains unit tests exercising these set and get operations, validating that parameter persistence works correctly across compression contexts.

Decoder-Side Considerations

The decoder must be configured to accept the window size used during encoding. While the default decoder settings typically accept windows up to ZSTD_WINDOWLOG_MAX, you can explicitly set a maximum to limit memory exposure on resource-constrained systems:

ZSTD_DCtx *dctx = ZSTD_createDCtx();
ZSTD_DCtx_setParameter(dctx, ZSTD_d_windowLogMax, 26);   /* allow up to 64 MiB */

If the decoder's maximum window size is smaller than the encoder's ZSTD_c_windowLog, decompression will fail with an error indicating the frame requires a larger window than permitted. This safety mechanism prevents memory exhaustion attacks and ensures predictable resource usage.

Summary

• ZSTD_c_windowLog controls the sliding window size as 2^windowLog bytes, determining how far back the encoder searches for matches.
• Larger windows improve compression ratio on data with distant repetitive patterns but increase memory usage proportionally.
• The parameter is set via ZSTD_CCtx_setParameter in lib/zstd.h and is validated against ZSTD_WINDOWLOG_MIN and ZSTD_WINDOWLOG_MAX limits.
• The encoder automatically adjusts hashLog and chainLog based on the window size to maintain efficient look-up tables.
• Decoders must use ZSTD_d_windowLogMax to accommodate the window size, or decompression will fail.

Frequently Asked Questions

What is the default value of ZSTD_c_windowLog?

The default value varies by compression level but typically falls between 20 and 27, depending on the strategy selected. You can query the default for a specific level using ZSTD_getCParams with the target compression level, which returns a ZSTD_compressionParameters struct containing the default window log.

How does ZSTD_c_windowLog affect decompression memory?

The decompression memory requirement is determined by the window size stored in the compressed frame header, which derives from the encoder's ZSTD_c_windowLog. The decoder allocates a buffer sized to this window, meaning a frame compressed with windowLog=26 requires roughly 64 MiB of decoder memory regardless of the input file size.

What happens if I set ZSTD_c_windowLog higher than necessary?

Setting the window log higher than the data's actual repetition distance wastes memory without improving compression ratio. The encoder in lib/zstd.h will still allocate the full window buffer, and the decoder must reserve equivalent memory, increasing resource consumption for no compression benefit.

Can ZSTD_c_windowLog be changed during streaming compression?

Yes, you can adjust ZSTD_c_windowLog between frames when using the streaming API (ZSTD_CStream). However, you cannot change it mid-frame because the window size is encoded in the frame header. The test cases in tests/zstreamtest.c demonstrate resetting parameters between ZSTD_compressStream operations to compress different frames with distinct window sizes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}