# How Needle 2 Computes Engram Indices: A Deep Dive into the Hashing Algorithm

> Learn how Needle 2 computes engram indices using a 32-bit hash function with bit-shifts and a prime multiplier to generate slot addresses.

- Repository: [Cactus Compute, Inc./needle](https://github.com/cactus-compute/needle)
- Tags: deep-dive
- Published: 2026-08-24

---

**Needle 2 computes engram indices by applying a deterministic 32-bit hash function that mixes token IDs with order-dependent bit-shifts and a prime multiplier, then reduces the result modulo the table size to generate per-head slot addresses.**

Engram indices are the addressing mechanism behind Needle 2's memory-efficient attention system. Unlike standard transformer architectures that compute attention solely from hidden states, Needle stores contextual information in *engrams*—learned tables indexed by hashed n-gram contexts. The computation of these indices is handled entirely within the `engram_indices` function in **[`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py)**, which transforms input token IDs into integer slot addresses through a series of bit-wise operations designed for reproducibility and collision resistance.

## The `engram_indices` Function Architecture

The core implementation resides in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) and accepts four parameters that define the hashing geometry:

- **`tokens`**: Input token IDs (integer array of shape `(batch, seq)`)
- **`orders`**: Tuple of n-gram orders (e.g., `(2, 3)` for bi-grams and tri-grams)
- **`heads`**: Number of attention heads (integer)
- **`slots`**: Size of each engram embedding table (integer)

The function returns a tensor of shape `(batch, seq, orders × heads)` containing the computed slot indices for every position, order, and attention head combination.

## Step-by-Step Index Computation

The hashing algorithm proceeds through nine distinct stages, each designed to maximize entropy while maintaining deterministic behavior across hardware platforms.

### 1. Type Conversion to Unsigned 32-bit Integers

The process begins by casting input tokens to `uint32` to ensure consistent bit-wise behavior across different hardware architectures and JAX backends.

```python
u = tokens.astype(jnp.uint32)

```

This guarantees that subsequent shift and XOR operations produce identical results on CPU, GPU, and TPU.

### 2. Per-Order and Per-Head Iteration

The algorithm iterates over each **engram order** (representing different n-gram window sizes) and each **attention head** to generate independent index streams. This nesting ensures that each head receives a unique hashing trajectory, significantly reducing collision probability across the parallel attention mechanisms.

```python
for oi, order in enumerate(orders):
    for h in range(heads):
        # Per-head seed derivation

        seed = (_ENGRAM_SEED * (oi * heads + h + 1)) & 0xFFFFFFFF
        acc = jnp.full_like(u, jnp.uint32(seed))

```

The seed incorporates the global constant `_ENGRAM_SEED` combined with the linearized order-head index, creating a unique starting point for each hash stream while maintaining reproducibility.

### 3. N-gram Hash Mixing

For each position within the current n-gram window, the accumulator undergoes a cryptographic-style mixing operation. The algorithm XORs the accumulator with the token ID shifted right by `j` positions, then multiplies by the prime constant `_ENGRAM_PRIME`.

```python
for j in range(order):
    acc = (acc ^ _shift_right(u, j)) * jnp.uint32(_ENGRAM_PRIME)

```

This step distributes token information across the 32-bit space, where `_shift_right` creates n-gram sensitivity by incorporating contextual positions through bit displacement rather than explicit window slicing.

### 4. Final Diffusion and Slot Reduction

After the mixing loop, the accumulator undergoes a final decorrelation step before modular reduction:

```python
acc = acc ^ (acc >> jnp.uint32(15))
idx.append((acc % jnp.uint32(slots)).astype(jnp.int32))

```

The XOR-shift operation (`acc ^ (acc >> 15)`) provides additional avalanche effect, ensuring that small changes in input tokens propagate to multiple output bits. The modulo operation maps the 32-bit hash into the concrete table address space `[0, slots-1]`.

### 5. Tensor Stacking

Finally, all per-head indices are concatenated along the last axis:

```python
return jnp.stack(idx, axis=1)

```

This produces the output tensor of shape `(batch, seq, orders × heads)`, where the last dimension contains the slot indices for every engram configuration at every sequence position.

## Practical Usage Examples

### Direct Index Computation

You can invoke the `engram_indices` function directly to inspect how specific token sequences map to table slots:

```python
import jax.numpy as jnp
from needle.model.architecture import engram_indices

# Sample token IDs (batch=1, sequence length=4)

tokens = jnp.array([[101, 2023, 2003, 102]])

# Engram configuration matching default model setup

orders = (2, 3)      # Bi-gram and tri-gram contexts

heads = 8            # Eight attention heads

slots = 8192         # Table size per head

indices = engram_indices(tokens, orders, heads, slots)
print(f"Shape: {indices.shape}")  # (1, 4, 16)

print(f"Slot range: [{indices.min()}, {indices.max()}]")  # Within [0, 8191]

```

### Integration with the Engram Module

Within the model forward pass, these indices feed directly into the `Engram` class instances. The `engram_geometry` helper extracts configuration parameters from the model config:

```python
from needle.model.architecture import Engram, engram_geometry

# Extract parameters from loaded configuration

orders, heads, slots = engram_geometry(model_config)

# Compute indices during forward pass

indices = engram_indices(input_tokens, orders, heads, model_config.engram_slots)

# Retrieve key-value pairs from each engram table

kv_outputs = [
    engram_layer(indices, ngram_mask=valid_ngrams, tap_mask=valid_taps)
    for engram_layer in self.engram_layers
]

```

## Integration with the Needle 2 Architecture

The computed indices serve as the addressing mechanism for several critical subsystems beyond the forward pass. In **[`needle/model/decode.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/decode.py)**, the indices are incorporated into the KV cache during autoregressive generation, allowing the model to retrieve previously computed engram contexts without recomputing hashes for the entire prefix.

Configuration management occurs in **[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)**, which extracts engram geometry (orders, heads, slots) from saved model checkpoints, ensuring that exported models preserve the exact hashing parameters used during training.

## Summary

- **Engram indices** are deterministic 32-bit hashes computed in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py) via the `engram_indices` function.
- The algorithm uses **type-safe uint32 operations**, **per-head seeding**, and **prime-based mixing** to generate unique slot addresses for each attention head and n-gram order.
- Indices are produced by XORing token IDs with shifted versions of themselves, multiplying by `_ENGRAM_PRIME`, applying final bit diffusion, and reducing modulo the table size.
- The resulting tensor of shape `(batch, seq, orders × heads)` feeds directly into the `Engram` module to retrieve context-aware key-value embeddings for the attention mechanism.

## Frequently Asked Questions

### What are engram indices used for in Needle 2?

Engram indices function as addresses into learned embedding tables that store contextual information. According to the source code in [`needle/model/architecture.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/architecture.py), these indices allow the model to retrieve n-gram specific key and value vectors for each attention head, effectively compressing context into fixed-size tables rather than computing full attention over the entire sequence history.

### Why does the algorithm use uint32 and bit-wise operations instead of standard hashing?

The implementation casts tokens to `uint32` and employs XOR shifts and prime multiplication to guarantee deterministic, hardware-independent behavior. Unlike Python's built-in `hash()` function, which may vary between sessions or platforms, the 32-bit integer operations in `engram_indices` produce identical slot addresses across CPU, GPU, and TPU devices, ensuring reproducible model behavior.

### How does per-head seeding prevent index collisions?

By deriving a unique seed for each combination of engram order and attention head using the formula `(_ENGRAM_SEED * (oi * heads + h + 1))`, the algorithm ensures that different heads hash the same token sequence into different areas of the slot space. This separation prevents multiple heads from competing for the same table entries, effectively partitioning the engram memory banks by attention head.

### Where is the engram geometry (orders, heads, slots) defined in the codebase?

The geometry parameters are extracted from model configurations in **[`needle/model/export.py`](https://github.com/cactus-compute/needle/blob/main/needle/model/export.py)** (lines 120-131) and utilized during initialization. The `engram_geometry` helper function parses these values from the model config, which are then passed to `engram_indices` during both training in [`architecture.py`](https://github.com/cactus-compute/needle/blob/main/architecture.py) and inference in [`decode.py`](https://github.com/cactus-compute/needle/blob/main/decode.py) (lines 156-165).