Needle 2 Model File Size: Understanding the 14 MB Compressed Binary

The compressed Needle 2 model is a 14 MB self-contained binary that contains all model weights and runs inference in approximately 28 MB of RAM, requiring no additional files.

The cactus-compute/needle repository distributes Needle 2 as a foundation tool-calling model optimized for tiny devices. According to the source documentation and package metadata, the compressed Needle 2 model file size is exactly 14 MB, making it one of the most compact fully-functional language model binaries available.

Where the 14 MB File Size Is Documented

The 14 MB specification appears consistently across the repository's core documentation files.

pyproject.toml Package Metadata

In pyproject.toml, the package description explicitly defines the model's footprint: "A 14 MB foundation tool-calling model for tiny devices". This metadata drives the package distribution configuration and confirms the compressed binary size that users download via pip.

README.md and llms.txt Specifications

The README.md states: "The whole model is a single 14 MB binary that runs a full session in about 28 MB of RAM". Similarly, llms.txt notes that the inference engine fetches as a 14 MB binary from Hugging Face. Both files confirm that the entire model ships as one self-contained file without separate weight directories.

Loading the Compressed Binary in Python

The needle package automatically handles downloading and caching the 14 MB binary. In needle/model/run.py, the runtime initializes the model by loading this single binary into memory.

# Load Needle 2 and run inference using the 14 MB binary

from needle import Needle

# The 14 MB binary downloads automatically on first initialization

needle = Needle(model="cactus-needle")

# Execute a prompt - runs in ~28 MB RAM total

response = needle.run("Translate the following to French: Hello, how are you?")
print(response)

Memory Architecture and Runtime Footprint

While the compressed file size is 14 MB, the runtime memory footprint expands to approximately 28 MB during active sessions. This efficient memory expansion ratio is implemented in needle/model/architecture.py, which manages weight decompression and cache allocation without extracting files to disk. The single-binary approach eliminates filesystem overhead from multiple shard files typical of larger transformer models.

Fine-Tuning and Export Without Size Bloat

Despite supporting parameter-efficient fine-tuning via LoRA adapters, the export process maintains the original compression ratio. The export() function in needle/model/export.py merges trained adapters back into the base weights while preserving the ~14 MB binary size.

# Fine-tune with LoRA and export while maintaining 14 MB size

from needle import Needle, LoRA, export

needle = Needle(model="cactus-needle")
lora = LoRA(needle)

# Fine-tuning adapts the model without duplicating the base weights

lora.fine_tune(train_data, epochs=3, learning_rate=1e-4)

# Export merges adapters - output remains approximately 14 MB

export_path = export(needle, output_dir="exported_model")
print(f"Exported 14 MB binary saved to {export_path}")

The needle/model/finetune.py module implements Low-Rank Adaptation (LoRA) to ensure that training only modifies small adapter matrices rather than the full 14 MB weight set, preventing file size inflation during customization.

Summary

• The compressed Needle 2 model file size is 14 MB, documented in pyproject.toml, README.md, and llms.txt.
• The binary is self-contained and requires no additional model weight files.
• Runtime memory consumption is approximately 28 MB of RAM during inference sessions.
• Fine-tuning via LoRA and export operations preserve the ~14 MB binary size.
• Source implementation resides in needle/model/run.py, needle/model/architecture.py, and needle/model/export.py.

Frequently Asked Questions

What is the exact file size of the compressed Needle 2 model?

The compressed Needle 2 model is exactly 14 MB as specified in the pyproject.toml package metadata and confirmed in the repository's README.md. This single binary contains all model weights required for inference.

How much RAM does the Needle 2 model require at runtime?

According to the source documentation in README.md, the model runs a full inference session in approximately 28 MB of RAM. This represents roughly double the compressed file size due to decompressed weight caching and activation storage during forward passes.

Does fine-tuning the Needle 2 model increase its file size?

No. When using the LoRA (Low-Rank Adaptation) implementation in needle/model/finetune.py, fine-tuning trains small adapter matrices rather than duplicating the base weights. The export() function in needle/model/export.py merges these adapters while maintaining the original ~14 MB binary footprint.

Where is the Needle 2 binary downloaded from?

The llms.txt file indicates that the 14 MB inference engine binary is automatically fetched from Hugging Face when initializing the Needle class. The needle Python package handles caching after the initial download.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}