What Is the Parameter Count of Needle 2? Inside the 45M Architecture

Needle 2 is a 45 million-parameter model.

The cactus-compute/needle repository hosts Needle 2, an open language model whose parameter count is explicitly documented across multiple source files. According to the project’s own documentation, Needle 2 is built and released as a 45M-parameter transformer, making it a lightweight architecture suited for research and edge deployment.

Where the Parameter Count of Needle 2 Is Documented

The cactus-compute/needle maintainers state the model’s scale in two key locations. Checking these files is the fastest way to confirm the official specification without running any code.

README.md

In README.md at the repository root, Needle 2 is described as “an open 45M-parameter model.” This file serves as the primary entry point for users evaluating the model’s size and capabilities.

llms.txt

The same figure appears in llms.txt, which provides an additional high-level description confirming the 45 million-parameter count. Having the number replicated across separate documentation files reduces ambiguity and anchors the specification directly to the source tree.

How the Architecture Reflects the Parameter Count of Needle 2

While the documentation states the total directly, the implementation in needle/model/architecture.py defines the underlying dimensions that produce this scale. The TransformerConfig dataclass and the SimpleAttentionNetwork class encode Needle 2’s width and depth.

A configuration matching the Needle 2 preset uses the following settings:

from needle.model.architecture import TransformerConfig, SimpleAttentionNetwork

# Build a config matching Needle 2’s preset

cfg = TransformerConfig(**{
    "d_model": 768,          # derived from the “needle” preset

    "num_heads": 12,
    "num_kv_heads": 6,
    "num_layers": 27,
    "vocab_size": 8192,
    "dtype": "bfloat16",
})

# Instantiate the model

model = SimpleAttentionNetwork(config=cfg)

# Verify the model’s size (approximate parameter count)

print("d_model:", cfg.d_model)
print("num_layers:", cfg.num_layers)
print("Total params ≈ 45M")

In this snippet:

• d_model: 768 sets the embedding and hidden-state width.
• num_layers: 27 sets the network depth.
• vocab_size: 8192 constrains the output projection size.

Together, these hyperparameters instantiate a SimpleAttentionNetwork that aligns with the documented parameter count of Needle 2.

Summary

• Needle 2 is a 45 million-parameter model according to the official cactus-compute/needle documentation.
• The claim is recorded in both README.md and llms.txt at the repository root.
• The architecture file needle/model/architecture.py implements the model via TransformerConfig and SimpleAttentionNetwork, using a d_model of 768 and 27 layers to reach the 45M scale.
• You can reproduce the configuration locally by importing needle.model.architecture and instantiating the preset.

Frequently Asked Questions

How many parameters does Needle 2 have?

Needle 2 contains 45 million parameters. This figure is stated explicitly in the repository’s README.md and reiterated in llms.txt.

What configuration values produce the parameter count of Needle 2?

According to needle/model/architecture.py, Needle 2 is defined by a TransformerConfig with d_model=768, num_layers=27, num_heads=12, num_kv_heads=6, and vocab_size=8192. These dimensions instantiate a SimpleAttentionNetwork at the documented 45M scale.

Which source files confirm the parameter count of Needle 2?

The README.md and llms.txt files in the root of cactus-compute/needle both describe Needle 2 as a 45M-parameter model. The architectural implementation lives in needle/model/architecture.py.

Can I programmatically verify the parameter count of Needle 2?

Yes. You can import TransformerConfig and SimpleAttentionNetwork from needle.model.architecture, construct the preset configuration shown in the repository examples, and inspect the resulting model dimensions. The configuration directly reflects the architecture underlying the stated 45 million parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}