How the Stack Module Facilitates Cross-Layer Information Flow in Needle 2

The Stack class in Needle 2 enables high-bandwidth cross-layer information flow through highway-style gating, layer-wise shared parameters via nn.scan, and multi-lane residual connections that allow early-layer signals to propagate directly to deeper layers without degradation.

The Stack module serves as the central communication backbone in the cactus-compute/needle architecture, replacing traditional residual connections with sophisticated routing mechanisms. Unlike standard transformer stacks where information degrades with depth, this implementation creates persistent highways that allow early-layer features—such as Engram table memories—to propagate across all 27 layers while maintaining signal integrity.

Site Flag Preparation for Engram Routing

Before the forward pass begins, the Stack prepares binary routing matrices that identify which layers host Engram tables.

In needle/model/architecture.py (lines 86-92), when an engram_kv parameter is supplied, the Stack class creates a binary matrix marking specific layers as Engram hosts. These site flags function as addressing mechanisms, ensuring that memory-augmented signals route to the correct architectural depth during the cross-layer information flow.

Scan-Based Layer Execution with Shared Parameters

The core iteration mechanism relies on JAX’s nn.scan to execute layers while maintaining parameter sharing across depth.

Layer-Wise Parameter Sharing

Lines 110-122 of needle/model/architecture.py define a ScanBlock that repeatedly executes the _ScanBody class for each transformer layer. By scanning over layers with nn.scan, the Stack shares identical parameters—prefixed with mhc_*—across all depths. This weight sharing strategy horizontalizes information flow, allowing gradients and activations to treat the stack depth as a recurrent dimension rather than isolated stages.

Highway-Style Gating Mechanisms

Within each _ScanBody iteration, three distinct gates regulate how information moves between layers, creating multiple pathways for signal propagation.

Pre-Gate Activation Mixing

At lines 61-64, the input tensor x undergoes pre-gating (hpre), which mixes current activations with learned projections via mhc_phi_pre. This initial gate determines how much of the incoming signal enters the current layer's computation, preserving relevant patterns from earlier depths.

Post-Gate Residual Control

Following the attention and feed-forward computations, a post-gate (hpost) at lines 66-67 modulates the residual contribution. This gate controls the amplitude of the newly computed residual y before it merges with the main pathway, preventing early-layer signal saturation.

Residual-Gate with Sinkhorn Normalization

The final residual-gate (hres) at lines 68-72 employs _sinkhorn normalization to blend the original representation with the residual across multiple lanes. This mechanism enables cross-lane interactions, distributing information laterally within each layer before propagating it vertically to subsequent depths.

Multi-Lane Aggregation and Output Normalization

After completing the scan across all layers, the Stack collapses the multi-lane structure into a unified representation.

Lines 124-125 in needle/model/architecture.py execute this consolidation through two operations: averaging across lanes using jnp.mean(x, axis=2) followed by RMS normalization. This aggregation merges parallel processing streams back into a single hidden state, completing the cross-layer information flow cycle while maintaining numerical stability.

Practical Implementation Example

The following example demonstrates how to instantiate the Stack module and observe cross-layer hidden states during inference:

import jax.numpy as jnp
from needle.model.architecture import TransformerConfig, SimpleAttentionNetwork, Stack

# Configure model with cross-layer lanes and Engram layers

cfg = TransformerConfig(
    d_model=768,
    num_layers=27,
    num_heads=12,
    engram_layers=(2, 15),
    mhc_lanes=4
)

# Initialize the full network (Stack is embedded internally)

model = SimpleAttentionNetwork(cfg)

# Forward pass triggers cross-layer flow

tokens = jnp.arange(0, 64)[None, :]
logits = model(tokens)

# Extract intermediate states showing cross-layer propagation

hidden = model.hidden_cells(tokens)
print(f"Hidden shape: {hidden.shape}")  # (batch, seq, layers+1, d_model)

# Direct Stack invocation for advanced use

stack = Stack(cfg)
x = jnp.zeros((1, 10, cfg.d_model))
x_out, layer_hidden = stack(x, collect_hidden=True)

Summary

• The Stack module in needle/model/architecture.py creates site flags (lines 86-92) to route Engram-augmented information to specific layers.
• Layer-wise parameter sharing via nn.scan (lines 110-122) allows horizontal information flow across all transformer depths using shared mhc_* weights.
• Three highway gates—pre-gate (hpre), post-gate (hpost), and residual-gate (hres)—regulate signal propagation and prevent depth-based degradation.
• Multi-lane processing with Sinkhorn normalization enables lateral information exchange before vertical aggregation.
• Final RMS normalization and mean-pooling (lines 124-125) consolidate parallel lanes into coherent output representations.

Frequently Asked Questions

How does the Stack module differ from standard transformer residual connections?

Standard transformers use simple additive residuals that can dilute early-layer signals as depth increases. The Needle 2 Stack replaces this with highway-style gating that learns dynamically how much information to preserve, transform, or route between layers. According to the source code in needle/model/architecture.py, the three-gate system (pre, post, and residual) provides fine-grained control over information flow that static residual connections cannot achieve.

What role do the mhc_lanes play in cross-layer communication?

The mhc_lanes parameter creates parallel processing streams within each layer, allowing the model to maintain multiple representations simultaneously. As implemented in lines 68-72 of needle/model/architecture.py, these lanes interact through the residual-gate's Sinkhorn normalization, enabling lateral information exchange before the final aggregation step merges them back into a single hidden state.

Why does the Stack use nn.scan instead of a standard Python loop?

The nn.scan implementation in lines 110-122 enforces parameter sharing across all transformer layers through the mhc_* parameter set. This architectural choice treats layer depth as a recurrent dimension, ensuring that the same transformation applies at every layer while maintaining differentiable memory of previous states. This approach reduces parameter count and creates the horizontal information pathways characteristic of the Needle 2 architecture.

Where does the Engram memory integration occur in the Stack?

Engram tables integrate through the site flags prepared at lines 86-92 of needle/model/architecture.py. When engram_kv is provided, the Stack creates a binary matrix identifying which layers (specified in engram_layers) should receive external memory injections. These flags route information during the scan execution, allowing specific layers to access persistent memory stores while maintaining the highway communication structure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}