# Needle 2 Model Binary Size: 14 MB Deep Dive and Implementation Guide

> Discover the 14 MB Needle 2 model binary size. This guide offers a deep dive into its implementation and showcases its compact, full-capability design for local deployment.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-31

---

**The Needle 2 model binary is exactly 14 MB**, making it one of the most compact full-capability language models available for local deployment.

This article examines how the cactus-compute/needle repository achieves this remarkably small footprint while delivering complete inference sessions in approximately 28 MB of RAM. We'll explore the technical implementation, loading mechanisms, and deployment options based on the actual source code.

## Needle 2 Model Binary Size Specifications

The **14 MB binary size** is explicitly documented in the project's README at line 3:

> "The whole model is a single **14 MB binary** that runs a full session in about 28 MB of RAM." — [README.md](https://github.com/cactus-compute/needle/blob/main/README.md)

This compact packaging represents a deliberate engineering trade-off. The development team optimized for edge deployment scenarios where network bandwidth and storage constraints matter more than raw parameter count.

### How the 14 MB Size Is Achieved

Three architectural decisions contribute to the minimal footprint:

- **CQ2-bit quantization** — aggressive weight compression implemented in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)
- **Simple Attention Network** — reduced complexity architecture that eliminates redundant attention heads
- **Single-file packaging** — the [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) module bundles all weights and inference code into one executable binary

## Loading the 14 MB Needle 2 Binary in Python

The primary interface for working with the model is [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py). This module handles automatic binary fetching, caching, and inference execution.

### Automatic Binary Loading

```python
from needle.model.run import Model

# Initialise the model (the binary is fetched/cached automatically)

model = Model()

# Run a simple inference

output = model.generate("What is the capital of France?")
print(output)

```

The `Model` class in [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) performs several operations on first initialization:

1. Checks `~/.cache/needle/` for existing `needle.bin`
2. Downloads the 14 MB binary from the configured repository if missing
3. Maps the binary into memory for inference execution

### Custom Binary Location

For deployments requiring explicit binary management, use the `download_binary` utility:

```python
from needle.model.run import download_binary

# Store the 14 MB binary at a custom location

binary_path = download_binary(cache_dir="/my/models")
print(f"Binary saved to {binary_path}")

```

The returned `binary_path` points to the exact 14 MB file, allowing verification of integrity via file size checks or checksum validation.

## Command-Line Deployment

The needle CLI provides the simplest path to running the 14 MB model without Python code:

```bash

# Install the package

pip install cactus-needle

# The CLI automatically downloads the 14 MB binary on first run

needle run "Summarize the plot of *The Matrix*"

```

The CLI implementation wraps the same [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) infrastructure, ensuring consistent behavior across interfaces.

## Key Source Files Behind the 14 MB Binary

Understanding the binary's composition requires examining four critical files:

| File | Role |
|------|------|
| [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) | Core runtime that loads the 14 MB binary and exposes the inference API |
| [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) | Defines the model architecture (Simple Attention Network, CQ2-bit quantization) |
| [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) | Handles exporting and packaging the binary for distribution |
| [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) | Documents the 14 MB size specification |

### Architecture Implementation Details

The [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) file contains the quantization schemes that enable 14 MB storage. The CQ2-bit quantization reduces per-parameter storage to 2 bits while maintaining functional accuracy through calibrated scaling factors.

### Export and Packaging Pipeline

[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) performs final binary generation. This module:

- Serializes quantized weights to a flat buffer format
- Embeds the inference runtime as position-independent code
- Applies optional compression for network transfer (decompressed to 14 MB on disk)

## Memory and Storage Comparison

| Metric | Value | Context |
|--------|-------|---------|
| **Disk size** | 14 MB | Single self-contained file |
| **RAM usage (full session)** | ~28 MB | Including activations and KV cache |
| **Typical LLaMA-7B** | ~13 GB | 900× larger storage requirement |
| **Typical GPT-2 Medium** | ~1.5 GB | 100× larger storage requirement |

The **14 MB Needle 2 model binary** achieves roughly three orders of magnitude size reduction compared to standard transformer implementations through architectural innovation rather than mere pruning.

## Deployment Scenarios for the 14 MB Binary

The compact size enables specific use cases impractical for larger models:

- **Browser extensions** — 14 MB download acceptable for one-time installation
- **Mobile applications** — fits within app store size guidelines without OTA complications
- **Embedded systems** — deploys to microcontrollers with external flash storage
- **Serverless functions** — cold-start download completes within typical timeout windows

## Verifying Your Binary Size

To confirm correct download of the full 14 MB file:

```python
import os
from needle.model.run import download_binary

binary_path = download_binary()
size_mb = os.path.getsize(binary_path) / (1024 * 1024)

assert abs(size_mb - 14.0) < 0.5, f"Unexpected binary size: {size_mb:.2f} MB"
print(f"Verified: {size_mb:.2f} MB binary present")

```

Minor variation (< 0.5 MB) accounts for platform-specific padding in the executable format.

## Summary

- **The Needle 2 model binary is 14 MB**, documented explicitly in [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md)
- Automatic loading via `needle.model.run.Model()` fetches and caches the binary to `~/.cache/needle/`
- **CQ2-bit quantization** and **Simple Attention Network** architecture in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) enable this compression
- Custom download locations supported through `download_binary(cache_dir=...)`
- Full inference sessions operate within **28 MB RAM**, preserving the storage-to-memory efficiency ratio

## Frequently Asked Questions

### Where does the 14 MB Needle 2 binary get stored on disk?

By default, the binary caches to `~/.cache/needle/needle.bin`. The `download_binary()` function accepts a `cache_dir` parameter for custom locations. The [`needle/model/run.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/run.py) module manages all path resolution automatically.

### Is the 14 MB size the compressed or uncompressed binary?

The 14 MB figure represents the uncompressed on-disk size after download and extraction. Network transfer may use additional compression, but the functional binary occupies 14 MB when loaded for inference.

### Why does RAM usage (28 MB) exceed the binary size (14 MB)?

The doubled RAM footprint includes the 14 MB weight storage plus activation tensors, key-value cache for context windows, and temporary buffers for forward passes. The [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) implementation specifically optimizes this working set.

### Can I run Needle 2 without downloading the 14 MB binary?

No. The binary contains all model parameters and the optimized inference runtime. However, the download occurs automatically on first use and the 14 MB transfer completes quickly on standard connections.