Why Do Older .cact Archives Fail to Load in Newer Needle Engine Versions

Older .cact archives trigger a RuntimeError because the archive format is strictly coupled to the specific Needle engine version that produced it, and the loader enforces exact header version matching to prevent undefined behavior from evolving binary layouts.

The cactus-compute/needle repository provides a fast, single-file LLM inference engine that relies on proprietary .cact archives to store quantized model weights and metadata. Understanding why older .cact archives fail to load in newer Needle engine versions is essential for maintaining compatibility across toolchain updates and preventing production outages.

How the Needle Engine Validates .cact Versions

In needle/__init__.py, the Needle class constructor attempts to load the specified weights via a memory-mapped file. During initialization, the engine inspects the archive header to verify the format version. If the version encoded in the .cact header does not match the version expected by the current engine binary, the loader immediately raises:

RuntimeError: failed to load weights from old.cact - the .cact archive format is tied to the engine version, and this version cannot read it.

This verification occurs before any tensor data is accessed, ensuring that incompatible binary layouts cannot cause memory corruption or silent failures.

Technical Causes of Archive Incompatibility

The .cact format is intentionally tightly coupled to the engine implementation because three core components evolve between releases:

• Binary Tensor Representation: The on-disk layout of quantized weights, including stride ordering and memory alignment, changes to optimize inference performance or support new hardware instructions. Archives produced by older quantizers store tensors in layouts that newer engines no longer recognize.

• Quantization Metadata Schema: The JSON manifest embedded within the archive tracks per-layer bit depths, scaling factors, and compression formats. Newer engines may require manifest fields that older archives omit, causing validation failures during the loading sequence in needle/__init__.py.

• C++ ABI and Data Structures: The engine is written in C++17 with strict ABI expectations. Even minor changes to internal data structures in the native layer break binary compatibility, making it impossible for the new engine to interpret the old archive's raw byte stream without risking undefined behavior.

Resolving Version Mismatch Errors

Since the engine is completely stateless and performs no runtime conversion, the only solution is to rebuild the archive using the current toolchain. According to the documentation in doc/finetuning.md, you must regenerate the .cact file with the needle build command:

needle build checkpoints/needle2.pkl --lora adapter.pkl --out compatible.cact

This regenerates the archive with the correct header version, updated quantization metadata, and tensor layouts that match the current engine's expectations.

Example: Handling Load Failures

Attempting to load an archive from an older engine version raises an immediate exception:

import needle

try:
    agent = needle.Needle(weights="old_model.cact")
except RuntimeError as e:
    print(f"Compatibility error: {e}")

To resolve, rebuild the archive with the current needle package:

needle build checkpoints/base.pkl --lora trained_adapter.pkl --out fixed_model.cact

Then load successfully:

agent = needle.Needle(weights="fixed_model.cact")
result = agent.run("Verify the fix works.")

Summary

• The .cact format uses a versioned header that must exactly match the engine binary in needle/__init__.py.
• Changes to tensor layouts, quantization schemas, and C++ ABIs break backward compatibility.
• The loader rejects mismatched archives with a specific RuntimeError before mapping weight data.
• Rebuilding with needle build generates a compatible archive with the correct format version and metadata schema.

Frequently Asked Questions

Can I patch an old .cact file to work with a new engine?

No. The engine does not support runtime conversion or binary patching. The binary representation of tensors and the internal data structure layout differ significantly between versions, making automated conversion impossible without access to the original checkpoint files.

Where is the version check performed in the Needle source code?

The version validation occurs in needle/__init__.py when the Needle class constructor processes the weights parameter. The loader inspects the archive header before memory-mapping the weight data, ensuring strict version coupling.

What components inside a .cact archive change between engine versions?

The archive contains three evolving components: the quantized weight tensors (binary layout), the JSON quantization metadata (schema fields), and the header version number itself. Any of these can trigger a compatibility failure if they do not align with the current engine's expectations.

How do I prevent .cact compatibility issues in production deployments?

Always rebuild your archives immediately after upgrading the needle package. Store original checkpoints (.pkl files) and LoRA adapters rather than relying solely on .cact archives for long-term storage, as explicitly recommended in doc/finetuning.md to avoid the "failed to load weights" pitfall.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}