# What Is the Parameter Count of Needle 2? Inside the 45M Architecture

> Discover the exact parameter count of Needle 2, a powerful 45 million parameter model from Cactus Compute. Learn about its architecture and capabilities.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-19

---

**Needle 2 is a 45 million-parameter model.**

The `cactus-compute/needle` repository hosts Needle 2, an open language model whose **parameter count** is explicitly documented across multiple source files. According to the project’s own documentation, Needle 2 is built and released as a **45M-parameter** transformer, making it a lightweight architecture suited for research and edge deployment.

## Where the Parameter Count of Needle 2 Is Documented

The `cactus-compute/needle` maintainers state the model’s scale in two key locations. Checking these files is the fastest way to confirm the official specification without running any code.

### README.md

In [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) at the repository root, Needle 2 is described as “an open **45M-parameter** model.” This file serves as the primary entry point for users evaluating the model’s size and capabilities.

### llms.txt

The same figure appears in [`llms.txt`](https://github.com/cactus-compute/needle/blob/main/llms.txt), which provides an additional high-level description confirming the **45 million-parameter** count. Having the number replicated across separate documentation files reduces ambiguity and anchors the specification directly to the source tree.

## How the Architecture Reflects the Parameter Count of Needle 2

While the documentation states the total directly, the implementation in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) defines the underlying dimensions that produce this scale. The `TransformerConfig` dataclass and the `SimpleAttentionNetwork` class encode Needle 2’s width and depth.

A configuration matching the Needle 2 preset uses the following settings:

```python
from needle.model.architecture import TransformerConfig, SimpleAttentionNetwork

# Build a config matching Needle 2’s preset

cfg = TransformerConfig(**{
    "d_model": 768,          # derived from the “needle” preset

    "num_heads": 12,
    "num_kv_heads": 6,
    "num_layers": 27,
    "vocab_size": 8192,
    "dtype": "bfloat16",
})

# Instantiate the model

model = SimpleAttentionNetwork(config=cfg)

# Verify the model’s size (approximate parameter count)

print("d_model:", cfg.d_model)
print("num_layers:", cfg.num_layers)
print("Total params ≈ 45M")

```

In this snippet:

- **`d_model: 768`** sets the embedding and hidden-state width.
- **`num_layers: 27`** sets the network depth.
- **`vocab_size: 8192`** constrains the output projection size.

Together, these hyperparameters instantiate a `SimpleAttentionNetwork` that aligns with the documented **parameter count of Needle 2**.

## Summary

- Needle 2 is a **45 million-parameter** model according to the official `cactus-compute/needle` documentation.
- The claim is recorded in both [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) and [`llms.txt`](https://github.com/cactus-compute/needle/blob/main/llms.txt) at the repository root.
- The architecture file [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) implements the model via `TransformerConfig` and `SimpleAttentionNetwork`, using a `d_model` of 768 and 27 layers to reach the 45M scale.
- You can reproduce the configuration locally by importing `needle.model.architecture` and instantiating the preset.

## Frequently Asked Questions

### How many parameters does Needle 2 have?

Needle 2 contains **45 million parameters**. This figure is stated explicitly in the repository’s [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) and reiterated in [`llms.txt`](https://github.com/cactus-compute/needle/blob/main/llms.txt).

### What configuration values produce the parameter count of Needle 2?

According to [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), Needle 2 is defined by a `TransformerConfig` with `d_model=768`, `num_layers=27`, `num_heads=12`, `num_kv_heads=6`, and `vocab_size=8192`. These dimensions instantiate a `SimpleAttentionNetwork` at the documented 45M scale.

### Which source files confirm the parameter count of Needle 2?

The [`README.md`](https://github.com/cactus-compute/needle/blob/main/README.md) and [`llms.txt`](https://github.com/cactus-compute/needle/blob/main/llms.txt) files in the root of `cactus-compute/needle` both describe Needle 2 as a 45M-parameter model. The architectural implementation lives in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py).

### Can I programmatically verify the parameter count of Needle 2?

Yes. You can import `TransformerConfig` and `SimpleAttentionNetwork` from `needle.model.architecture`, construct the preset configuration shown in the repository examples, and inspect the resulting model dimensions. The configuration directly reflects the architecture underlying the stated 45 million parameters.