# What Is 3D RoPE and How Does It Achieve Temporal Consistency in LingBot-Map?

> Discover how 3D RoPE in LingBot-Map achieves temporal consistency by extending 2D rotary embeddings with a temporal dimension for aligned streaming inference.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: deep-dive
- Published: 2026-06-29

---

**3D RoPE extends standard 2‑D rotary position embeddings with a temporal dimension that encodes global frame indices, ensuring cached keys and queries remain aligned across streaming inference by rotating features once before they enter the KV cache.**

Robbyant's LingBot-Map is an open-source SLAM system that processes video streams through transformer blocks. To prevent temporal drift during sliding-window inference, the implementation introduces 3D Rotary Position Embedding (RoPE), which augments spatial coordinates with absolute temporal positioning to stabilize representations across frames.

## What Is 3D RoPE?

Standard Rotary Position Embedding (RoPE) encodes token positions by rotating feature vectors using sinusoidal matrices. While 2‑D RoPE treats height and width as independent axes, 3D RoPE introduces a third axis for time, allowing the model to distinguish temporal positions using absolute global indices rather than relative window offsets.

### The WanRotaryPosEmbed Implementation

The core logic resides in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) within the `WanRotaryPosEmbed` class. This module splits the attention head dimension into three distinct sub-dimensions for time (`t`), height (`h`), and width (`w`).

```python

# lingbot_map/layers/rope.py

class WanRotaryPosEmbed(nn.Module):
    """3D rotary position embedding (t, h, w)."""
    def __init__(self, attention_head_dim, patch_size, max_seq_len=1024,
                 theta=10000.0, fhw_dim=[20, 22, 22]):
        # head_dim is split into t-, h-, and w-sub-dimensions

        # Pre-computes separate 1-D RoPE tables for each axis via get_1d_rotary_pos_embed

        # Stores concatenated frequencies in self.freqs as complex-valued tensors

```

The `fhw_dim` parameter (default `[20, 22, 22]`) determines how many units of the head dimension are allocated to each axis, where `t_dim + h_dim + w_dim = head_dim`. For each axis, the constructor invokes `get_1d_rotary_pos_embed` to build sinusoidal frequency tables, then concatenates these into `self.freqs`—a tensor indexable by `(frame, row, column)` triples.

### Dimensional Isolation

By partitioning the embedding space, temporal rotations remain orthogonal to spatial patterns. For a configuration with `attention_head_dim=64`, the split might reserve 20 dimensions for temporal encoding, 22 for height, and 22 for width. This isolation prevents temporal positional information from interfering with spatial geometric reasoning.

## How 3D RoPE Achieves Temporal Consistency

Temporal consistency requires that identical frames receive identical embeddings regardless of when they are processed in a streaming pipeline. 3D RoPE accomplishes this through global indexing and strategic placement in the attention mechanism.

### Global Frame Indexing

Instead of using relative positions within a sliding window, `CameraCausalHead` passes absolute frame indices to the RoPE module. The head maintains `self.frame_idx` as a persistent global counter across inference steps:

```python

# lingbot_map/heads/camera_head.py

if self.rope3d is not None:
    pos3d = self.rope3d(
        ppf=S,                       # Number of frames in current chunk

        pph=1, ppw=1,                # Camera token occupies 1×1 patch

        patch_start_idx=0,
        device=pose_tokens.device,
        f_start=self.frame_idx,      # Global start frame (absolute)

        f_end=self.frame_idx + S,    # Global end frame (absolute)

    )

```

The `WanRotaryPosEmbed.__call__` method selects the frequency slice corresponding to the range `[f_start:f_end]`, returning a complex tensor of shape `[1, 1, S, head_dim//2]`. Because this encoding derives deterministically from the absolute frame number, the same physical frame processed in different windows receives an identical rotation.

### KV Cache Integration

The attention mechanism applies 3D RoPE **before** keys enter the KV cache. In [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py), the code checks the `enable_3d_rope` flag to prevent double-application during cache retrieval:

```python

# lingbot_map/layers/attention.py (lines 154-159)

if not enable_3d_rope:
    # Apply standard 2-D RoPE only when 3D is disabled

    apply_rotary_emb(...)

# Keys are now temporally encoded and stored in the cache

```

This ensures that when the model retrieves cached keys during streaming, all temporal rotations have already been applied. The cached representations remain fixed to the global timeline, eliminating the drift that would occur if relative window positions were used.

### Flag Propagation Chain

The `enable_3d_rope` boolean propagates through the model hierarchy to coordinate the feature:

- **Model Base**: Defined in `GCTBase.__init__` ([`lingbot_map/models/gct_base.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_base.py), line 56)
- **Aggregator**: `AggregatorStream._init_3d_rope()` instantiates the shared `WanRotaryPosEmbed` instance ([`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), lines 101-104)
- **Detection Head**: `CameraCausalHead` initializes `self.rope3d` when the flag is active ([`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py), lines 86-88)

When enabled, this chain ensures that all transformer blocks share the same 3D RoPE module and global frame counter.

## Practical Usage Example

To enable temporal consistency in a streaming configuration:

```python
from lingbot_map.models.gct_stream_window_v2 import GCTStream

# Initialize with 3D RoPE enabled

model = GCTStream(
    img_size=518,
    patch_size=14,
    embed_dim=1024,
    disable_global_rope=False,
    enable_3d_rope=True,        # Enable 3-D temporal RoPE

    max_frame_num=1024,
)

# Streaming inference maintains temporal consistency automatically

# The model internally tracks frame_idx and applies appropriate rotations

```

During forward passes, the `CameraCausalHead` automatically manages frame indexing and applies 3D rotations to camera tokens without requiring manual position management.

## Summary

- **3D RoPE** extends 2‑D rotary embeddings by adding a temporal dimension via the `WanRotaryPosEmbed` class in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py).
- **Dimensional splitting** uses the `fhw_dim` parameter to allocate specific head dimensions to time, height, and width axes.
- **Global indexing** relies on absolute frame positions (`f_start`, `f_end`) rather than relative offsets, preventing temporal drift across sliding windows.
- **Pre-cache application** encodes temporal rotations into keys before they enter the KV cache, with the attention layer skipping re-application when `enable_3d_rope` is active.
- **Architecture coordination** occurs through a flag propagation chain from `GCTBase` through `AggregatorStream` to `CameraCausalHead`.

## Frequently Asked Questions

### What is the difference between 2D and 3D RoPE in LingBot-Map?

2D RoPE treats height and width as independent spatial axes by concatenating separate rotary embeddings for each. 3D RoPE introduces a temporal component that reserves specific head dimensions for frame indices, enabling the model to distinguish temporal positions using absolute global time rather than just spatial coordinates.

### Why does temporal consistency matter for streaming SLAM?

Streaming SLAM processes video incrementally using a KV cache to avoid recomputing past states. Without temporal consistency, the same physical frame processed in different sliding windows would receive different positional encodings, causing the model to treat identical scenes as distinct events and accumulating positional drift over time.

### How does 3D RoPE interact with the KV cache?

3D RoPE applies temporal rotations to keys **before** they are stored in the KV cache, as implemented in [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py). When retrieving cached keys, the attention mechanism checks the `enable_3d_rope` flag and skips re-applying rotary embeddings. This ensures all cached keys remain on the same global temporal coordinate system regardless of when they were computed.

### Where is the temporal dimension size configured?

The allocation between temporal, height, and width dimensions is configured via the `fhw_dim` parameter in `WanRotaryPosEmbed.__init__` (default `[20, 22, 22]` in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py)). These values must sum to the total `attention_head_dim`, with the first element representing the temporal allocation.