Needle 2 Model Binary Size: 14 MB Deep Dive and Implementation Guide

The Needle 2 model binary is exactly 14 MB, making it one of the most compact full-capability language models available for local deployment.

This article examines how the cactus-compute/needle repository achieves this remarkably small footprint while delivering complete inference sessions in approximately 28 MB of RAM. We'll explore the technical implementation, loading mechanisms, and deployment options based on the actual source code.

Needle 2 Model Binary Size Specifications

The 14 MB binary size is explicitly documented in the project's README at line 3:

"The whole model is a single 14 MB binary that runs a full session in about 28 MB of RAM." — README.md

This compact packaging represents a deliberate engineering trade-off. The development team optimized for edge deployment scenarios where network bandwidth and storage constraints matter more than raw parameter count.

How the 14 MB Size Is Achieved

Three architectural decisions contribute to the minimal footprint:

• CQ2-bit quantization — aggressive weight compression implemented in needle/model/architecture.py
• Simple Attention Network — reduced complexity architecture that eliminates redundant attention heads
• Single-file packaging — the needle/model/export.py module bundles all weights and inference code into one executable binary

Loading the 14 MB Needle 2 Binary in Python

The primary interface for working with the model is needle/model/run.py. This module handles automatic binary fetching, caching, and inference execution.

Automatic Binary Loading

from needle.model.run import Model

# Initialise the model (the binary is fetched/cached automatically)

model = Model()

# Run a simple inference

output = model.generate("What is the capital of France?")
print(output)

The Model class in needle/model/run.py performs several operations on first initialization:

1. Checks ~/.cache/needle/ for existing needle.bin
2. Downloads the 14 MB binary from the configured repository if missing
3. Maps the binary into memory for inference execution

Custom Binary Location

For deployments requiring explicit binary management, use the download_binary utility:

from needle.model.run import download_binary

# Store the 14 MB binary at a custom location

binary_path = download_binary(cache_dir="/my/models")
print(f"Binary saved to {binary_path}")

The returned binary_path points to the exact 14 MB file, allowing verification of integrity via file size checks or checksum validation.

Command-Line Deployment

The needle CLI provides the simplest path to running the 14 MB model without Python code:

# Install the package

pip install cactus-needle

# The CLI automatically downloads the 14 MB binary on first run

needle run "Summarize the plot of *The Matrix*"

The CLI implementation wraps the same needle/model/run.py infrastructure, ensuring consistent behavior across interfaces.

Key Source Files Behind the 14 MB Binary

Understanding the binary's composition requires examining four critical files:

File	Role
needle/model/run.py	Core runtime that loads the 14 MB binary and exposes the inference API
needle/model/architecture.py	Defines the model architecture (Simple Attention Network, CQ2-bit quantization)
needle/model/export.py	Handles exporting and packaging the binary for distribution
README.md	Documents the 14 MB size specification

Architecture Implementation Details

The needle/model/architecture.py file contains the quantization schemes that enable 14 MB storage. The CQ2-bit quantization reduces per-parameter storage to 2 bits while maintaining functional accuracy through calibrated scaling factors.

Export and Packaging Pipeline

needle/model/export.py performs final binary generation. This module:

• Serializes quantized weights to a flat buffer format
• Embeds the inference runtime as position-independent code
• Applies optional compression for network transfer (decompressed to 14 MB on disk)

Memory and Storage Comparison

Metric	Value	Context
Disk size	14 MB	Single self-contained file
RAM usage (full session)	~28 MB	Including activations and KV cache
Typical LLaMA-7B	~13 GB	900× larger storage requirement
Typical GPT-2 Medium	~1.5 GB	100× larger storage requirement

The 14 MB Needle 2 model binary achieves roughly three orders of magnitude size reduction compared to standard transformer implementations through architectural innovation rather than mere pruning.

Deployment Scenarios for the 14 MB Binary

The compact size enables specific use cases impractical for larger models:

• Browser extensions — 14 MB download acceptable for one-time installation
• Mobile applications — fits within app store size guidelines without OTA complications
• Embedded systems — deploys to microcontrollers with external flash storage
• Serverless functions — cold-start download completes within typical timeout windows

Verifying Your Binary Size

To confirm correct download of the full 14 MB file:

import os
from needle.model.run import download_binary

binary_path = download_binary()
size_mb = os.path.getsize(binary_path) / (1024 * 1024)

assert abs(size_mb - 14.0) < 0.5, f"Unexpected binary size: {size_mb:.2f} MB"
print(f"Verified: {size_mb:.2f} MB binary present")

Minor variation (< 0.5 MB) accounts for platform-specific padding in the executable format.

Summary

• The Needle 2 model binary is 14 MB, documented explicitly in README.md
• Automatic loading via needle.model.run.Model() fetches and caches the binary to ~/.cache/needle/
• CQ2-bit quantization and Simple Attention Network architecture in needle/model/architecture.py enable this compression
• Custom download locations supported through download_binary(cache_dir=...)
• Full inference sessions operate within 28 MB RAM, preserving the storage-to-memory efficiency ratio

Frequently Asked Questions

Where does the 14 MB Needle 2 binary get stored on disk?

By default, the binary caches to ~/.cache/needle/needle.bin. The download_binary() function accepts a cache_dir parameter for custom locations. The needle/model/run.py module manages all path resolution automatically.

Is the 14 MB size the compressed or uncompressed binary?

The 14 MB figure represents the uncompressed on-disk size after download and extraction. Network transfer may use additional compression, but the functional binary occupies 14 MB when loaded for inference.

Why does RAM usage (28 MB) exceed the binary size (14 MB)?

The doubled RAM footprint includes the 14 MB weight storage plus activation tensors, key-value cache for context windows, and temporary buffers for forward passes. The needle/model/architecture.py implementation specifically optimizes this working set.

Can I run Needle 2 without downloading the 14 MB binary?

No. The binary contains all model parameters and the optimized inference runtime. However, the download occurs automatically on first use and the 14 MB transfer completes quickly on standard connections.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}