Using the Kanzi EXE Transform for x86 and ARM64 Executable Compression

The Kanzi EXE transform preprocesses x86/AMD64 and ARM64 executable files by normalizing relative branch instructions into absolute addresses, significantly reducing entropy before the final compression stage.

Kanzi’s EXE transform provides architecture-aware preprocessing for Windows PE, Linux ELF, and macOS Mach-O binaries. Implemented in src/transform/EXECodec.cpp and declared in src/transform/EXECodec.hpp, this specialized codec automatically detects executable formats and applies platform-specific transformations to improve compression ratios.

How the EXE Transform Works

The EXE transform operates at the block level, parsing executable headers to locate code segments and rewriting branch instructions to create repetitive patterns that compress efficiently.

Architecture Detection and Header Parsing

When a data block enters the forward() method, the detectType() function (lines 468-560 in src/transform/EXECodec.cpp) scans the buffer to identify executable formats. This routine uses Magic::getType to determine the file format, then calls parseHeader() (lines 562-689) to extract the code segment boundaries (codeStart and codeEnd) and determine the target architecture.

The detection logic returns architecture identifiers: X86 (0x40) for x86/AMD64 or ARM64 (0x20) for ARM64 binaries. This byte value directs the transform to invoke the appropriate processing routine.

x86 and AMD64 Transformation Logic

For x86 and AMD64 executables, the forwardX86() method (lines 113-190) processes the code region sequentially. It identifies relative CALL (0xE8) and JMP (0xE9) instructions, along with two-byte conditional jumps (0F 8?).

For each branch instruction found, the transform:

• Calculates the absolute target address from the relative displacement
• Applies an XOR mask using MASK_ADDRESS (0xF0F0F0F0)
• Stores the masked absolute address in place of the original relative offset

This conversion transforms variable relative offsets into near-constant absolute address patterns, dramatically improving compressibility for the downstream entropy coder.

ARM64 Transformation Logic

The forwardARM() method (lines 194-270) handles 32-bit ARM64 branch instructions including unconditional branches (B/BL) and conditional jumps (CBZ/CBNZ, currently disabled). The algorithm:

• Decodes the 26-bit signed offset embedded in branch instructions
• Converts the relative offset to an absolute target address
• Re-encodes the instruction with a normalized offset

This process mirrors the x86 approach, converting position-dependent branch targets into consistent address representations that exhibit lower entropy.

Inverse Transformation and Metadata

The transform stores metadata in the output stream to enable perfect reconstruction. The first byte records the architecture type (X86 or ARM64), followed by two 32-bit values indicating codeStart and the transformed region length.

During decompression, the inverse() method dispatches to inverseX86() (lines 318-389) or inverseARM() (lines 391-466) based on this header. These routines reverse the address masking and offset calculations, restoring the original relative branch instructions byte-for-byte. The transformation is non-lossy, guaranteeing identical output after a complete round-trip.

Implementing EXE Compression in C++

Integrate the EXE transform into your compression pipeline using Kanzi’s C++ API by specifying the codec string in the Compressor constructor.

Compressing Executables

#include "kanzi/api/Compressor.hpp"
#include "kanzi/api/Context.hpp"
#include <fstream>

int main()
{
    kanzi::Context ctx;
    // Chain EXE transform with Run-Length, Text, and FPAQ entropy coding
    kanzi::Compressor cmp(ctx, "EXE+RLT+TEXT+UTF+FPAQ");
    
    std::ifstream in("program.exe", std::ios::binary);
    std::ofstream out("program.kanzi", std::ios::binary);
    
    cmp.compress(in, out);
    return 0;
}

The codec string "EXE+RLT+TEXT+UTF+FPAQ" configures the pipeline:

• EXE: Preprocesses executable branch instructions
• RLT: Applies Run-Length Transform for repeated patterns
• TEXT+UTF: Processes embedded text strings
• FPAQ: Fast predictive arithmetic entropy coding

Decompressing Executables

#include "kanzi/api/Decompressor.hpp"
#include "kanzi/api/Context.hpp"
#include <fstream>

int main()
{
    kanzi::Context ctx;
    kanzi::Decompressor dcmp(ctx, "EXE+RLT+TEXT+UTF+FPAQ");
    
    std::ifstream in("program.kanzi", std::ios::binary);
    std::ofstream out("program_restored.exe", std::ios::binary);
    
    dcmp.decompress(in, out);
    return 0;
}

The identical codec string ensures the inverse EXE transformation executes during decompression, restoring original relative offsets.

Command-Line Usage

The kanzi executable supports EXE transform via the -t or --transform flag, as documented in src/app/Kanzi.cpp (lines 139-145).

Compress an ARM64 binary:

kanzi -c -i myArm64.bin -o myArm64.kanzi -t EXE+RLT+TEXT+UTF+FPAQ

Decompress:

kanzi -d -i myArm64.kanzi -o myArm64_restored.bin

Integration with the Compression Pipeline

The EXE transform registers processed blocks with the data type Global::EXE, stored in the block context within src/io/CompressedOutputStream.cpp. This metadata signals downstream entropy codecs (such as FPAQ or ANS) that the block has undergone executable-specific preprocessing.

In src/app/BlockCompressor.cpp, the default transform strings assemble the complete pipeline, while src/transform/AliasCodec.cpp demonstrates how EXE is treated as a special data type within Kanzi’s aliasing framework.

Summary

• The EXE transform in src/transform/EXECodec.cpp specializes in compressing x86/AMD64 and ARM64 executables by normalizing branch instructions.
• Architecture detection via detectType() and parseHeader() automatically handles PE, ELF, and Mach-O formats.
• x86 processing uses forwardX86() to mask relative CALL/JMP targets with MASK_ADDRESS (0xF0F0F0F0).
• ARM64 processing via forwardARM() normalizes 26-bit branch offsets into absolute addresses.
• The transform is reversible through inverseX86() and inverseARM(), ensuring lossless compression.
• Specify the transform in API calls or command-line tools using the codec string starting with "EXE+...".

Frequently Asked Questions

How does the EXE transform detect whether a file is x86 or ARM64?

The detectType() function in src/transform/EXECodec.cpp (lines 468-560) examines the file magic using Magic::getType, then parseHeader() (lines 562-689) parses PE, ELF, or Mach-O headers to extract the architecture field. The method returns 0x40 for x86/AMD64 and 0x20 for ARM64, which the forward() method uses to dispatch to the appropriate transformation routine.

Can the EXE transform be used with other compression codecs besides FPAQ?

Yes. While the examples show "EXE+RLT+TEXT+UTF+FPAQ", you can substitute FPAQ with any entropy coder supported by Kanzi, such as ANS or Range coding. The EXE transform appears first in the chain to reduce entropy, followed by other transforms like RLT (Run-Length Transform), before the final entropy coding stage.

Is the EXE transform lossless for executable files?

Yes. The transformation is completely non-lossy. During decompression, inverseX86() (lines 318-389) or inverseARM() (lines 391-466) reads the architecture byte and code segment metadata stored at the beginning of the transformed block to restore the original relative branch offsets, producing byte-for-byte identical output.

Does the EXE transform work with 32-bit ARM binaries or only ARM64?

The current implementation focuses on ARM64 (AArch64) as indicated by the forwardARM() implementation in src/transform/EXECodec.cpp (lines 194-270), which processes 26-bit branch offsets specific to the ARM64 instruction set. The detection logic identifies ARM64 via the ARM64 constant (0x20). 32-bit ARM (ARMv7) support is not explicitly detailed in the current codebase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}