# How LingBot-Map Implements 3D RoPE for Temporal Consistency

> Discover how LingBot-Map enhances temporal consistency in streaming inference with its 3D RoPE implementation. Learn how global frame indices ensure consistent positional encoding.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: deep-dive
- Published: 2026-07-06

---

**LingBot-Map implements 3D RoPE by extending 2D rotary position embeddings with a temporal dimension in `WanRotaryPosEmbed`, applying global frame indices to tokens before they enter the KV cache to ensure consistent positional encoding across streaming inference.**

LingBot-Map is an open-source transformer-based architecture designed for streaming SLAM and map-building tasks. To maintain temporal consistency across video frames, the repository implements a specialized 3D Rotary Position Embedding (3D RoPE) mechanism that treats time, height, and width as independent rotational axes. This article examines how the `WanRotaryPosEmbed` class in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) leverages global frame indexing to prevent temporal drift during causal inference.

## What Is 3D RoPE in LingBot-Map?

### Extending 2D Rotary Position Embeddings

Traditional 2D RoPE encodes token positions by rotating feature vectors using sinusoidal matrices based on spatial coordinates. In [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py), the 2D implementation treats **height** and **width** independently, concatenating their rotated components. **3D RoPE** introduces a third rotational axis for the **temporal** dimension, assigning each video frame a distinct rotational signature based on its absolute position in the sequence.

### The WanRotaryPosEmbed Class Architecture

The core implementation resides in the `WanRotaryPosEmbed` class. The constructor receives the attention head dimension and splits it into three sub-dimensions for time (`t_dim`), height (`h_dim`), and width (`w_dim`), ensuring that `t_dim + h_dim + w_dim = head_dim`.

```python

# lingbot_map/layers/rope.py

class WanRotaryPosEmbed(nn.Module):
    """3D rotary position embedding (t, h, w)."""
    def __init__(self, attention_head_dim, patch_size, max_seq_len=1024,
                 theta=10000.0, fhw_dim=[20, 22, 22]):
        # Splits head_dim into t-, h-, w-sub-dimensions

        # Pre-computes separate 1-D RoPE tables for each axis

        # Stores complex-valued frequencies in self.freqs

```

The class pre-computes frequency tables for each axis using `get_1d_rotary_pos_embed`, concatenating them into `self.freqs` for efficient indexing during inference.

## How 3D RoPE Enables Temporal Consistency

### Global Frame Indexing vs. Relative Positioning

Temporal consistency requires that the same global frame index generates identical positional embeddings regardless of sliding window position. Unlike relative window indexing—which causes drift as frames shift through the buffer—3D RoPE uses absolute frame indices (`f_start`, `f_end`) to generate deterministic rotations. When the `CameraCausalHead` processes a chunk of frames, it calls `WanRotaryPosEmbed` with the global frame position:

```python

# lingbot_map/heads/camera_head.py (excerpt)

if self.rope3d is not None:
    pos3d = self.rope3d(
        ppf=S,               # Number of frames in current chunk

        pph=1, ppw=1,        # Camera token occupies 1×1 patch

        patch_start_idx=0,
        device=pose_tokens.device,
        f_start=self.frame_idx,         # Global start frame

        f_end=self.frame_idx + S,       # Global end frame

    )

```

This ensures that frame 100 always receives the same rotational encoding whether processed in isolation or as part of a sliding window.

### The KV Cache Integration

The positional embedding applies **before** keys enter the KV cache. In [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py), the attention block checks `if enable_3d_rope` and skips standard 2D RoPE application when the 3D variant is active. This prevents double-application of rotations when retrieving cached keys.

Because the temporal rotation is embedded at the moment of key creation, cached representations maintain their global temporal alignment. When the model queries past frames, the stored keys already contain the correct temporal rotation, ensuring consistent attention scores across streaming steps.

### Flag Propagation and Component Initialization

The `enable_3d_rope` flag propagates through the model hierarchy:

1. `GCTBase` ([`lingbot_map/models/gct_base.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_base.py), line 56) receives the configuration
2. `GCTStream` and `AggregatorStream` inherit the flag
3. `AggregatorStream` initializes the 3D RoPE module via `_init_3d_rope()` (lines 101-104 of [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py))
4. `CameraCausalHead` instantiates the embedding layer during initialization (lines 86-108 of [`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py))

This cascading configuration ensures that all components agree on whether to use temporal position encoding.

## Implementation Flow: From Model Config to Attention

The temporal consistency mechanism follows a strict initialization and execution pipeline:

1. **Model Configuration** – Set `enable_3d_rope=True` when instantiating `GCTStream` or `GCTBase`
2. **Aggregator Initialization** – `AggregatorStream` creates the shared `WanRotaryPosEmbed` instance in `_init_3d_rope()`
3. **Head Setup** – `CameraCausalHead` receives the 3D RoPE module and stores it as `self.rope3d`
4. **Position Generation** – During each forward pass, the camera head generates position tensors using global frame indices
5. **Attention Application** – The attention block receives the pre-computed 3D positions and applies them to queries and keys before caching

## Code Example: Enabling 3D RoPE in Practice

To utilize 3D RoPE for temporal consistency in your own inference pipeline:

```python
import torch
from lingbot_map.models.gct_stream_window_v2 import GCTStream
from lingbot_map.utils.geometry import setup_dummy_input

# Initialize model with 3D RoPE enabled

model = GCTStream(
    img_size=518,
    patch_size=14,
    embed_dim=1024,
    disable_global_rope=False,
    enable_3d_rope=True,  # Enable temporal consistency

    max_frame_num=1024,
)

# Create dummy streaming input (B=1, 3 frames, 1 camera token per frame)

tokens, mask = setup_dummy_input(num_frames=3)

# Forward pass with causal inference

with torch.no_grad():
    out = model(tokens, mask=mask, causal_inference=True)

# Output maintains temporal alignment across frames

print(out.shape)  # (B, S, output_dim)

```

The `setup_dummy_input` helper generates tensors of shape `[B, S, C]` where `S` represents the frame count, simulating the token embeddings produced by the vision backbone.

## Key Source Files and Architecture

| Component | File Path | Relevant Section |
|-----------|-----------|------------------|
| 3D RoPE Implementation | [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) | `WanRotaryPosEmbed` class definition (lines 79-106) |
| Camera Head Integration | [`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py) | Initialization and forward pass position generation (lines 86-108) |
| Aggregator Stream | [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py) | `_init_3d_rope()` method (lines 98-104) |
| Base Model Configuration | [`lingbot_map/models/gct_base.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_base.py) | `enable_3d_rope` constructor flag (lines 55-58) |
| Attention Mechanism | [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py) | Conditional RoPE application (lines 154-162) |
| CLI Demo | [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) | Argument parsing for feature flag (lines 365-371) |

## Summary

- **3D RoPE extends 2D rotary embeddings** by adding a temporal dimension through `WanRotaryPosEmbed` in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py)
- **Global frame indexing** ensures identical positional encoding for the same frame regardless of sliding window position
- **Pre-cache application** applies temporal rotations before keys enter the KV cache, preventing alignment drift
- **Flag propagation** cascades from `GCTBase` through `AggregatorStream` to `CameraCausalHead` for coordinated activation
- **Conditional attention** blocks avoid double-application by checking `enable_3d_rope` before applying standard 2D RoPE

## Frequently Asked Questions

### What is the difference between 2D RoPE and 3D RoPE in LingBot-Map?

2D RoPE encodes spatial height and width dimensions by rotating feature vectors with sinusoidal matrices based on patch grid coordinates. 3D RoPE adds a temporal dimension that assigns frame-specific rotations based on absolute frame indices, enabling the model to maintain consistent temporal relationships across streaming video sequences.

### Why must 3D RoPE be applied before the KV cache?

Applying 3D RoPE before caching ensures that stored keys contain the correct global temporal rotation. When the model retrieves cached keys during streaming inference, the temporal positions are already aligned with the query's current frame index. This prevents the "drift" that would occur if relative window positions were used or if rotations were applied twice.

### How does LingBot-Map prevent double-application of rotary embeddings?

The attention blocks in [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py) check the `enable_3d_rope` flag before applying standard 2D RoPE. When 3D RoPE is enabled, the temporal and spatial rotations are handled externally by `WanRotaryPosEmbed` and passed to the attention mechanism, ensuring the rotary transformation is applied exactly once per token.

### Can 3D RoPE be disabled for non-streaming inference?

Yes. The `enable_3d_rope` parameter in `GCTBase` and `GCTStream` controls this feature. When set to `False`, the model falls back to standard 2D RoPE, which is suitable for single-frame processing or scenarios where temporal consistency across cached frames is not required.