# Optimizing Zstd Dictionary Training: Complete Parameter Guide

> Master Zstd dictionary training with our comprehensive parameter guide. Optimize compression efficiency and resource usage by tuning segment size, d-mer size, and more.

- Repository: [Meta/zstd](https://github.com/facebook/zstd)
- Tags: deep-dive
- Published: 2026-09-09

---

**Zstandard dictionary training exposes seven tunable parameters—segment size (k), d-mer size (d), frequency log size (f), acceleration level (accel), thread count (nbThreads), split point, and verbosity—defined in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) that directly control the trade-off between compression efficiency and training resource usage.**

Optimizing zstd dictionary training requires understanding how each parameter in the `ZDICT_params_t` structure influences the cover algorithm's search for optimal dictionary entries. The facebook/zstd repository provides flexible APIs in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) that allow fine-tuning from the default conservative settings to high-performance variants for specific data distributions.

## Core Training Parameters Defined in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h)

The `ZDICT_params_t` structure encapsulates all configuration options passed to training functions. These parameters are processed by the cover algorithm implementation in [`lib/dictBuilder/cover.h`](https://github.com/facebook/zstd/blob/main/lib/dictBuilder/cover.h) to construct the final dictionary.

### Segment Size (k)

**k** defines the length in bytes of segments that the trainer splits each sample into before analysis. The default value is approximately 2048 bytes, with a typical operational range of 16 to 2048+.

Larger values yield finer-grained segment analysis, potentially improving compression ratios at the cost of increased memory consumption and CPU usage during training. Conversely, reducing k speeds up training but may miss longer repetitive patterns.

### D-mer Size (d)

**d** specifies the number of bytes constituting a "d-mer" (substring) for frequency analysis. It must satisfy the constraint `0 < d ≤ k`, with a default of approximately 6 and typical range of 6 to 16.

Smaller d values increase the total number of d-mers analyzed, enriching the dictionary with more entries but significantly increasing processing time. Larger values reduce the search space, accelerating training but potentially overlooking short repetitive sequences.

### Frequency Table Size (f)

**f** represents the log₂ of the internal frequency table size used by the cover trainer, defaulting to approximately 20 (implying a table size of roughly 1 million entries).

Increasing f to 22 or higher provides more accurate statistical modeling of d-mer distributions, improving dictionary quality at the expense of higher memory allocation during the training phase.

### Acceleration Level (accel)

**accel** controls the aggressiveness of the optimization approximation, accepting values from 1 to 10 with a default of 1 (most accurate).

Higher acceleration levels reduce training time by accepting sub-optimal dictionary entries earlier in the search process. Setting `accel = 5` or higher is recommended for rapid prototyping, while `accel = 1` maximizes compression performance for production dictionaries.

### Multithreading (nbThreads)

**nbThreads** configures the number of CPU threads utilized during training, defaulting to 1 (single-threaded). This parameter only takes effect when zstd is built with multithreading support.

Increasing thread count linearly reduces wall-clock training time without altering the final dictionary contents, as the cover algorithm produces deterministic output regardless of parallelism.

### Training Sample Ratio (splitPoint)

**splitPoint** specifies the fraction of provided samples used for actual training versus validation, defaulting to 1.0 (100% training).

Setting values around 0.75 reserves 25% of samples for validation, providing a better estimate of dictionary generalization while reducing the training corpus. This is particularly useful when evaluating dictionary quality on held-out data.

## Dictionary Training Functions

Zstd provides two primary training entry points in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h), with prototypes found around lines 190-210 and 379-389.

### ZDICT_trainFromBuffer (Classic Cover)

The standard trainer implements the full cover algorithm with comprehensive frequency analysis. As noted in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) lines 190-210, this function allocates approximately 6 MiB of temporary workspace and performs exhaustive d-mer frequency counting.

### ZDICT_trainFromBuffer_fastCover

The fastCover variant, declared near line 389 in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h), uses aggressive approximations and smaller internal data structures to reduce memory usage to approximately 9 MiB per input byte while maintaining acceptable compression ratios. This variant is ideal for memory-constrained environments or when training on massive corpora.

## Code Examples for Parameter Tuning

### Basic Training with Defaults

The simplest approach uses `NULL` for parameters to accept system defaults:

```c
#include "zstd.h"
#include "zdict.h"

void *samples;          /* concatenated sample buffers */
size_t *sampleSizes;    /* array of individual sample sizes */
size_t nbSamples = 100;
unsigned char dictBuffer[64 * 1024];

size_t dictSize = ZDICT_trainFromBuffer(
    dictBuffer, sizeof(dictBuffer),
    samples,
    sampleSizes, nbSamples,
    NULL);  /* NULL → use default parameters */

if (ZDICT_isError(dictSize)) {
    fprintf(stderr, "Training error: %s\n", ZDICT_getErrorName(dictSize));
}

```

### Custom Parameter Configuration

For production optimization, populate the `ZDICT_params_t` structure defined in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) lines 314-331:

```c
#include "zdict.h"

ZDICT_params_t params = ZDICT_paramsDefault();

/* Optimize for high compression with moderate speed */
params.k = 4096;        /* Larger segments for better pattern matching */
params.d = 8;           /* Medium d-mer size */
params.f = 22;          /* Larger frequency table for accuracy */
params.accel = 2;       /* Slight speedup without major quality loss */
params.nbThreads = 4;   /* Parallel processing */
params.splitPoint = 0.80; /* Reserve 20% for validation */

size_t dictSize = ZDICT_trainFromBuffer(
    dictBuffer, sizeof(dictBuffer),
    samples, sampleSizes, nbSamples,
    &params);

```

### Fast Cover for Resource-Constrained Environments

When memory is limited, use the fastCover variant with higher acceleration:

```c
ZDICT_params_t fastParams = ZDICT_paramsDefault();
fastParams.accel = 5;   /* Higher acceleration for speed */

size_t dictSize = ZDICT_trainFromBuffer_fastCover(
    dictBuffer, sizeof(dictBuffer),
    samples, sampleSizes, nbSamples,
    &fastParams);

```

## Summary

- **Seven parameters** control zstd dictionary training: k (segment size), d (d-mer size), f (frequency log), accel (speed), nbThreads (parallelism), splitPoint (validation ratio), and notificationLevel (verbosity).
- **Default settings** in `ZDICT_paramsDefault()` prioritize compression ratio over speed, using k≈2048, d≈6, and accel=1.
- **Memory usage** scales with f and k; the classic trainer uses ~6 MiB workspace while fastCover uses ~9 MiB per input byte.
- **Thread safety** allows nbThreads > 1 for faster wall-clock times without changing output determinism.
- **Source references** for all parameters are located in [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) (lines 314-331 for the structure, lines 190-210 and 379-389 for function prototypes).

## Frequently Asked Questions

### What is the optimal segment size (k) for dictionary training?

The optimal **k** depends on your data's natural block size. For log files or network packets, match k to the average record size (typically 1024-4096 bytes). According to the facebook/zstd source code in [`lib/dictBuilder/cover.h`](https://github.com/facebook/zstd/blob/main/lib/dictBuilder/cover.h), values below 16 provide minimal benefit, while exceeding 4096 offers diminishing returns for most corpora.

### How does the acceleration parameter affect dictionary quality?

The **accel** parameter implements early-termination heuristics in the cover algorithm. As implemented in [`lib/dict.h`](https://github.com/facebook/zstd/blob/main/lib/dict.h), accel values above 3 reduce the exhaustive search depth for d-mer selections, potentially missing 5-15% of optimal entries but reducing training time by 40-60%. For final production dictionaries, accel=1 is recommended.

### Can I use multiple threads to speed up dictionary training?

Yes, setting **nbThreads** greater than 1 in `ZDICT_params_t` enables parallel frequency counting and segment analysis. The implementation in `lib/dictBuilder/` divides the sample space evenly across threads, producing identical output regardless of thread count due to deterministic merge operations in the cover algorithm.

### What is the difference between cover and fastCover training?

**cover** (`ZDICT_trainFromBuffer`) performs exhaustive statistical analysis using the full parameters struct including **f** (frequency table size), while **fastCover** (`ZDICT_trainFromBuffer_fastCover`) bypasses the large frequency array allocation and uses probabilistic sampling. According to [`lib/zdict.h`](https://github.com/facebook/zstd/blob/main/lib/zdict.h) lines 379-389, fastCover ignores the f parameter and requires approximately 60% less memory, making it suitable for embedded systems or training dictionaries exceeding 100 MB of source samples.