How to Use the DNA Transform for Compressing Genomic Data in Kanzi

Kanzi automatically detects DNA sequences by analyzing per-block symbol histograms and applies a specialized PACK alias transform that maps the nucleotide alphabet to compact binary codes, delivering high compression ratios on genomic datasets without requiring manual configuration.

The DNA transform in the flanglet/kanzi-cpp library treats nucleotide sequences as a specialized case of the generic alias-coding engine. When compressing FASTA, FASTQ, or raw nucleotide files, Kanzi identifies DNA blocks through symbol-frequency analysis and applies alphabet-specific optimization, or you can explicitly invoke the transform via the C++ API, Python bindings, or command-line interface.

Architecture of the DNA Transform

The DNA implementation relies on three core components that bridge data detection and encoding.

Global::detectSimpleType in src/Global.cpp (lines 45-53) analyzes the symbol frequency histogram of each input block. If more than approximately 92% of the symbols belong to the DNA alphabet—covering lowercase and uppercase a, c, g, n, t, and u—the function returns the DNA type classification.

When the transform chain contains a DNA stage, TransformFactory::newToken in src/transform/TransformFactory.hpp (lines 92-95) instantiates an AliasCodec and sets the context flag packOnlyDNA = 1. This flag constrains the codec to operate exclusively on DNA-classified blocks.

The AliasCodec class in src/transform/AliasCodec.cpp implements the actual packing logic. At lines 66-70, the forward method checks the _onlyDNA flag; if true and the block's dataType is not DNA, the codec rejects the block and falls back to the next transform in the chain. When validation passes, the codec maps the sparse byte symbols of the DNA alphabet to short, efficient codes.

Automatic DNA Detection Threshold

Kanzi evaluates compression blocks independently to handle mixed-content files. The detection algorithm requires that roughly 92% of the bytes in a block match the DNA alphabet, allowing for occasional ambiguous nucleotides (N) or RNA symbols (U). This block-wise approach ensures that large genomic files containing intermittent headers or quality scores still compress efficiently, as only the sequence-bearing blocks trigger the DNA transform.

Using the DNA Transform

You can invoke DNA compression explicitly through the C++ API, command-line tool, or Python wrapper.

C++ API Implementation

To compress a FASTA file with forced DNA handling, configure the cData structure with the transform name and pass it to the compressor context:

#include "kanzi/api/Compressor.hpp"
#include <fstream>
#include <vector>

int main() {
    std::ifstream src("genome.fasta", std::ios::binary);
    std::ofstream dst("genome.knz", std::ios::binary);

    kanzi::cData params = {};
    strcpy(params.transform, "DNA");      // Force DNA alias codec
    strcpy(params.entropy, "HUFFMAN");    // Entropy stage
    params.blockSize = 4 * 1024 * 1024;   // 4 MiB blocks
    params.jobs = 4;

    kanzi::cContext* ctx = nullptr;
    if (kanzi::initCompressor(&params, dst.native_handle(), &ctx) != 0)
        return 1;

    std::vector<unsigned char> buf(params.blockSize);
    size_t inSize, outSize;

    while (src.read(reinterpret_cast<char*>(buf.data()), buf.size()) || 
           (inSize = src.gcount())) {
        inSize = src.gcount();
        if (kanzi::compress(ctx, buf.data(), inSize, &outSize) != 0)
            break;
    }

    kanzi::compress(ctx, nullptr, 0, &outSize);  // Flush
    kanzi::disposeCompressor(&ctx, &outSize);
    return 0;
}

Setting params.transform to "DNA" forces the factory to create an AliasCodec with packOnlyDNA enabled, bypassing automatic detection if necessary.

Command-Line Interface

Kanzi exposes the DNA transform as a named option and includes it in the built-in level 2 preset. To compress using the default level 2 configuration (DNA + LZ + Huffman):

kanzi -c -i sample.fasta -l 2 -j 8 -b 8m -o sample.knz

To apply only the DNA alias transform without additional LZ compression:

kanzi -c -i sample.fasta -t DNA -e HUFFMAN -b 4m -j 4 -o sample.knz

The -t DNA argument maps directly to the DNA_TYPE case in TransformFactory::newToken.

Python Wrapper

The Python API provides direct access to the same C++ compression path:

import kanzi

compressor = kanzi.Compressor(transform='DNA', entropy='HUFFMAN')
compressor.compress('input.fasta', 'output.knz')

The wrapper passes the transform='DNA' string to the underlying cData structure, triggering the packOnlyDNA logic in AliasCodec.

Combining DNA with Other Transforms

Genomic files often contain text headers (FASTA description lines) that disrupt pure DNA detection. You can chain the TEXT transform before DNA to handle mixed content:

kanzi -c -i sample.fasta -t TEXT+DNA -e HUFFMAN -b 4m -j 4 -o sample.knz

In this pipeline, the TEXT codec constructs a dictionary for header words, while the subsequent DNA stage packs the nucleotide sequences. The AliasCodec validates each block independently in src/transform/AliasCodec.cpp, rejecting non-DNA segments and allowing the TEXT codec to process headers separately.

Summary

• Automatic Detection: Kanzi identifies DNA blocks via Global::detectSimpleType when >92% of symbols match the nucleotide alphabet in src/Global.cpp.
• Specialized Codec: The DNA transform is an AliasCodec with packOnlyDNA = 1, instantiated by TransformFactory::newToken in src/transform/TransformFactory.hpp.
• Explicit Control: Use params.transform = "DNA" in the C++ API, transform='DNA' in Python, or -t DNA in the CLI to force activation.
• Block-wise Operation: The codec validates each block individually in src/transform/AliasCodec.cpp (lines 66-70), falling back to subsequent transforms for non-DNA data.
• Pipeline Friendly: Combine with the TEXT transform to handle FASTA headers before DNA compression.

Frequently Asked Questions

How does Kanzi distinguish DNA data from plain text?

Kanzi calculates a symbol-frequency histogram for each compression block in Global::detectSimpleType. If approximately 92% or more of the bytes belong to the DNA alphabet—A, C, G, T, U, N and their lowercase equivalents—the block is classified as DNA. This threshold allows for occasional ambiguous nucleotides or minor contamination while ensuring the AliasCodec only processes genuine genomic sequences.

What happens if I force the DNA transform on non-DNA data?

When you explicitly set -t DNA or params.transform = "DNA", the TransformFactory creates an AliasCodec with the packOnlyDNA flag set to 1. During forward processing in src/transform/AliasCodec.cpp, the codec checks if the block's dataType is DNA. If the check fails, the codec returns an error code that causes the compressor to skip the transform and proceed to the next codec in the chain, preventing data corruption.

Can the DNA transform handle RNA sequences (containing U instead of T)?

Yes, the detection alphabet explicitly includes U and u alongside the standard DNA characters. The Global::detectSimpleType function in src/Global.cpp recognizes both thymine and uracil representations, ensuring the transform activates correctly for RNA datasets as well as DNA datasets.

Is the DNA transform available in all compression levels?

The DNA transform is included in the built-in level 2 preset (-l 2), which configures the pipeline as DNA+LZ&HUFFMAN. For other levels, you must specify it manually using -t DNA or via the API. The transform is not enabled by default in level 1 or level 3+ configurations unless explicitly requested.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}