# Configurable Parameters for the Engram Layer in Needle: Complete Technical Reference

> Explore Engram layer configurable parameters in Needle. Discover settings like engram_orders, engram_slots, and engram_layers in TransformerConfig to optimize n-gram hashing and memory.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: api-reference
- Published: 2026-09-09

---

**TLDR:** The Engram layer in Needle exposes eight primary configurable parameters—including `engram_orders`, `engram_slots`, and `engram_layers`—defined in the `TransformerConfig` dataclass in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), which control n-gram hashing, per-order head allocation, memory slot capacity, and dilated convolution over value projections.

The Engram layer is a specialized memory‑augmented component in the Needle transformer architecture (cactus-compute/needle) that enhances context processing through hashed n‑gram tables. Tuning these **configurable parameters for the Engram layer** allows developers to balance memory consumption, computational throughput, and model capacity. This guide examines each parameter's location in the source code, default values, and runtime implications.

## Primary Configuration Parameters in TransformerConfig

The user‑facing configuration for the Engram layer resides in the `TransformerConfig` dataclass inside [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) (lines 70–76). These five fields determine the high‑level geometry of the memory tables.

### N‑gram Orders and Head Allocation

**`engram_orders`** defines the tuple of n‑gram sizes used to hash token sequences. According to the source definition at line 72, the default value is `(2, 3)`, meaning the model maintains separate hash tables for bigrams and trigrams.

**`engram_heads`** controls the number of attention heads allocated per n‑gram order. Defined at line 73 with a default of `0`, the value triggers automatic computation when left unset. The implementation calculates heads as `max(1, d_model // (len(engram_orders) * ENGRAM_SUB_DIM))`, ensuring the sub‑dimension per head remains consistent across model sizes.

### Memory Capacity and Slot Allocation

**`engram_slots`** sets the size of each hash table (number of slots). The default value of `8192` at line 74 directly impacts the model's ability to store unique n‑gram contexts. Larger values increase capacity but linearly increase memory consumption.

**`engram_layers`** specifies which transformer layers receive Engram memory injections. Defined at line 75 as a tuple of integers defaulting to `(2, 15)`, this parameter determines the depth at which the augmented memory mechanism operates.

### Convolution Configuration

**`engram_conv_dilation`** controls the dilation factor for the convolution applied to the value stream. While `engram_conv_taps` is fixed as a constant (see below), the dilation parameter at line 187 allows adjustment of the receptive field over cached values.

## Derived Constants and Internal Geometry

Several critical values are computed from the primary parameters above rather than being set directly.

### Sub‑Dimension and Table Count

The **`sub_dim`** (sub‑dimension) represents the width of each slice within an Engram table. The source code at lines 120–124 derives this as `d_model // (len(engram_orders) * engram_heads)`. This ensures that concatenating all table outputs restores the original model dimensionality.

The **`num_tables`** value, referenced at line 490, equals `len(engram_orders) * engram_heads`. This derived property represents the total count of parallel hash tables maintained by the layer.

### Fixed Architecture Constants

**`engram_conv_taps`** is a fixed architecture constant defined in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) at line 22 with a value of `4`. This setting controls the number of convolution taps applied to the value projection and is baked into the exported model binary format.

**`dtype`** specifies the numerical precision for Engram parameters. Implemented at line 188 of [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py), the layer defaults to `jnp.bfloat16` to optimize memory bandwidth during attention computations.

## Implementing Custom Engram Configurations

Developers can override defaults through the `TransformerConfig` dataclass or, for low‑level experimentation, instantiate the `Engram` module directly.

### Configuring via TransformerConfig

The standard entry point for adjusting **Engram layer parameters** is the model configuration object:

```python
from needle.model.architecture import TransformerConfig

# Customize Engram geometry for a 1B parameter model

cfg = TransformerConfig(
    d_model=1024,
    engram_orders=(2, 4, 8),      # Extend to 4-grams and 8-grams

    engram_heads=4,               # Override auto-computation

    engram_slots=16384,           # Double the default capacity

    engram_layers=(1, 5, 9),      # Inject at layers 1, 5, and 9

    engram_conv_dilation=2,       # Apply dilated convolution

)

# Instantiate model (pseudo-code; actual class varies by training script)

model = NeedleModel(cfg)

```

*Source:* `TransformerConfig` definition and default values—[[`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py)](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py#L70-L76).

### Direct Engram Module Instantiation

For unit testing or custom architectures, instantiate the `Engram` class directly using the derived geometry values:

```python
import jax.numpy as jnp
from needle.model.architecture import Engram

# Manually specify computed geometry

engram = Engram(
    d_model=512,
    num_tables=8,          # e.g., 2 orders × 4 heads

    slots=8192,
    sub_dim=16,            # Must equal d_model // (orders × heads)

    num_layers=27,         # Total transformer layers

    conv_dilation=1,
    dtype=jnp.bfloat16,
)

# Dummy forward inputs

indices = jnp.zeros((1, 10, 8), dtype=jnp.int32)   # (batch, seq, num_tables)

ngram_ok = jnp.ones_like(indices[..., :1])          # Validity mask

tap_ok = jnp.ones((4, 1))                          # ENGRAM_CONV_TAPS=4

k, v = engram(indices, ngram_ok, tap_ok, quant=False)

```

*Source:* `Engram` class signature—[[`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py)](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py#L81-L88).

## Summary

- **Primary parameters** (`engram_orders`, `engram_heads`, `engram_slots`, `engram_layers`, `engram_conv_dilation`) are defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) lines 72–75 and 187, controlling hashing strategy, memory capacity, and injection points.
- **Derived values** (`sub_dim`, `num_tables`) are computed from primary configs at lines 120–124 and 490, ensuring dimensional consistency across the model width.
- **Fixed constants** (`engram_conv_taps=4`, `dtype=jnp.bfloat16`) are hardcoded at [`export.py`](https://github.com/cactus-compute/needle/blob/main/export.py) line 22 and [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py) line 188.
- Override defaults via `TransformerConfig` for standard training workflows, or instantiate the `Engram` class directly for experimental architectures.

## Frequently Asked Questions

### What is the default memory capacity for each Engram hash table?

The default `engram_slots` value is **8192**, defined in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) at line 74. This sets the number of slots per hash table, with total memory consumption scaling by `num_tables × slots × sub_dim`.

### How does Needle automatically calculate the number of Engram heads?

When `engram_heads` is set to `0` (the default), the initialization code at line 73 computes heads as `max(1, d_model // (len(engram_orders) * ENGRAM_SUB_DIM))`. This formula divides the model dimension by the product of order count and a fixed sub‑dimension constant to determine parallel attention heads.

### Where is the convolution tap constant defined for Engram processing?

The **`engram_conv_taps`** constant is hardcoded to `4` in [`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py) at line 22. This value is serialized into the exported model header and determines the number of convolutional taps applied to value projections during inference.

### Can I use n‑gram orders other than the default (2, 3) in the Engram layer?

Yes. The `engram_orders` parameter accepts any tuple of integers. For example, configuring `(2, 4, 8)` enables bigram, 4‑gram, and 8‑gram hashing simultaneously, though this increases the total table count and memory requirements proportionally.