Needle Default Model Parameters: d_model and num_layers Configuration Guide

The Needle framework initializes d_model to 512 and num_layers to 12 by default through the TransformerConfig dataclass, while also providing "needle" and "base" presets with 27 layers for production-scale deployments.

The Needle repository implements a streamlined Transformer architecture defined in needle/model/architecture.py. Understanding the default configurations for model parameters like d_model and num_layers ensures you start with appropriate capacity for your computational constraints before applying custom overrides.

Default Values in TransformerConfig

The source of truth for default model parameters resides in the TransformerConfig dataclass within needle/model/architecture.py (lines 60‑66). When instantiated without arguments, this configuration yields a compact model suitable for experimentation:

• d_model defaults to 512, establishing the hidden dimension size for token representations
• num_layers defaults to 12, specifying the total count of stacked Transformer blocks

These values strike a balance between expressiveness and computational efficiency for development environments.

Production-Ready Preset Configurations

Beyond the compact defaults, Needle ships with two predefined dictionaries in the PRESETS mapping (lines 38‑44) optimized for different deployment scenarios. Both presets increase the layer count to 27 while adjusting the hidden dimensions.

The "needle" Preset

The preset named "needle" configures a larger variant with the following parameters:

• d_model: 768
• num_layers: 27
• num_heads: 12
• num_kv_heads: 6
• engram_layers: (2, 15)

This configuration appears at lines 38‑41 in architecture.py and targets scenarios requiring higher representational capacity.

The "base" Preset

The "base" preset maintains the default hidden dimension while scaling depth:

• d_model: 512
• num_layers: 27
• num_heads: 8
• num_kv_heads: 4
• engram_layers: (2, 15)

Defined at lines 42‑44, this option increases parameter count primarily through depth rather than width.

Instantiating Configurations in Code

You can leverage these defaults and presets through the TransformerConfig constructor. The following examples demonstrate initialization patterns using the actual source implementation:

Use the built‑in defaults for rapid prototyping:

from needle.model.architecture import TransformerConfig

default_cfg = TransformerConfig()
print(default_cfg.d_model)      # → 512

print(default_cfg.num_layers)   # → 12

Load the "needle" preset for production workloads:

from needle.model.architecture import TransformerConfig, PRESETS

needle_cfg = TransformerConfig(**PRESETS["needle"])
print(needle_cfg.d_model)      # → 768

print(needle_cfg.num_layers)   # → 27

Override specific parameters while preserving preset foundations:

custom_cfg = TransformerConfig(**PRESETS["base"], d_model=1024)
print(custom_cfg.d_model)      # → 1024

print(custom_cfg.num_layers)   # → 27 (inherited from base preset)

How Parameters Propagate Through the Codebase

These configuration values drive model instantiation across the repository. In needle/model/run.py, the TransformerConfig object directly determines the tensor shapes during model construction. Similarly, needle/model/export.py references config.num_layers and config.d_model when serializing model weights, ensuring the exported architecture matches the initialization parameters.

Summary

• Default dimensions: d_model=512 and num_layers=12 via TransformerConfig in needle/model/architecture.py (lines 60‑66)
• Preset architectures: "needle" (768/27) and "base" (512/27) configurations defined at lines 38‑44
• Flexible initialization: Instantiate with defaults, unpack presets, or override individual parameters via keyword arguments
• Code integration: run.py and export.py consume these values for model construction and serialization

Frequently Asked Questions

What does d_model represent in Needle?

In the Needle framework, d_model represents the hidden dimension size for token embeddings and all subsequent Transformer block computations. This value determines the width of every feedforward layer and the head dimension calculations when divided by num_heads.

How do I switch from default settings to the needle preset?

Import the PRESETS dictionary alongside TransformerConfig from needle.model.architecture, then unpack the desired preset using the spread operator: TransformerConfig(**PRESETS["needle"]). This replaces the default 512/12 configuration with the 768/27 architecture.

Can I override individual parameters while using a preset?

Yes, you can override any parameter by passing it as a keyword argument after unpacking the preset. For example, TransformerConfig(**PRESETS["base"], d_model=1024) maintains the 27 layers from the base preset while increasing the hidden dimension to 1024.

Where are these default configurations used during model export?

The export logic in needle/model/export.py accesses config.num_layers and config.d_model to validate tensor shapes and metadata during serialization. These references ensure the exported checkpoint maintains alignment with the architectural configuration defined in TransformerConfig.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}