Default Parameters for the 'needle' Preset in TransformerConfig

The needle preset configures TransformerConfig with d_model=768, num_heads=12, num_kv_heads=6, num_layers=27, and engram_layers=(2, 15) as default architectural values sourced from needle/model/architecture.py.

The needle preset provides a pre-tuned transformer architecture for the cactus-compute/needle repository's Needle model. When applied to a TransformerConfig instance, these default parameters initialize the model's embedding dimensions, attention heads, and layer topology without requiring explicit per-field configuration.

Default Parameter Values for the 'needle' Preset

The needle preset supplies the following architectural defaults when loaded into TransformerConfig:

• d_model: 768 — The dimensionality of the model's embeddings and hidden states
• num_heads: 12 — The number of attention heads in the multi-head attention mechanism
• num_kv_heads: 6 — The number of key-value heads for grouped query attention
• num_layers: 27 — The total number of transformer decoder layers
• engram_layers: (2, 15) — Specific layer indices where engram (memory) mechanisms are activated

These values are hardcoded in the PRESETS dictionary within needle/model/architecture.py at lines 38‑41 according to the source code. When instantiating TransformerConfig with this preset, any unspecified fields retain these values while other parameters like vocab_size=8192 and max_seq_len=2048 use their own independent defaults.

Source Code Location

The preset definitions reside in needle/model/architecture.py, which contains the TransformerConfig dataclass and the PRESETS mapping. The needle preset is specifically defined in the PRESETS dictionary at lines 38‑41, making it available for both direct configuration instantiation and high-level model initialization.

# Conceptual view of the PRESETS structure in needle/model/architecture.py

PRESETS = {
    "needle": {
        "d_model": 768,
        "num_heads": 12,
        "num_kv_heads": 6,
        "num_layers": 27,
        "engram_layers": (2, 15),
    },
    # ... other presets

}

How to Use the 'needle' Preset

You can apply the default parameters by unpacking the PRESETS dictionary directly into TransformerConfig or by using the high-level Needle API.

Direct TransformerConfig Instantiation

Access the preset through the PRESETS export to create a configuration with the default parameters:

from needle.model.architecture import TransformerConfig, PRESETS

# Create a config using the "needle" preset

needle_cfg = TransformerConfig(**PRESETS["needle"])

print("d_model:", needle_cfg.d_model)           # 768

print("num_heads:", needle_cfg.num_heads)       # 12

print("num_kv_heads:", needle_cfg.num_kv_heads) # 6

print("num_layers:", needle_cfg.num_layers)     # 27

print("engram_layers:", needle_cfg.engram_layers)  # (2, 15)

High-Level Model Initialization

The public API exposes the preset through the Needle class constructor:

from needle import Needle

# Initialize a Needle model with the preset configuration

model = Needle(preset="needle")   # internally loads the same values

Summary

• The needle preset defines five core architectural parameters: d_model=768, num_heads=12, num_kv_heads=6, num_layers=27, and engram_layers=(2, 15).
• These defaults originate from the PRESETS dictionary in needle/model/architecture.py (lines 38‑41).
• You can instantiate configurations via TransformerConfig(**PRESETS["needle"]) or use Needle(preset="needle") for simplified model creation.
• Additional fields like vocab_size and max_seq_len maintain separate defaults independent of the architectural preset.

Frequently Asked Questions

What is the purpose of the engram_layers parameter in the needle preset?

The engram_layers parameter specifies which transformer layers implement the engram mechanism, set to (2, 15) in the needle preset. This tuple indicates that layers 2 and 15 incorporate specialized memory or retrieval components distinct from standard attention blocks.

Can I override individual parameters when using the needle preset?

Yes, you can override any default value by passing explicit keyword arguments after unpacking the preset. For example, TransformerConfig(**PRESETS["needle"], d_model=512) creates a configuration with the needle preset's 12 attention heads but a reduced embedding dimension of 512.

Where are additional configuration fields like vocab_size defined?

Fields such as vocab_size=8192 and max_seq_len=2048 are defined as dataclass defaults within the TransformerConfig class itself in needle/model/architecture.py, separate from the PRESETS dictionary. They apply universally unless explicitly overridden.

Where are the needle preset defaults validated in the test suite?

The tests/test_build.py file contains test suites that validate preset handling and model construction, ensuring that TransformerConfig correctly applies the default parameters from PRESETS["needle"] during model initialization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}