# Maximum Sequence Length Supported by LingBot-Map: 1024 Token Limit Explained

> Discover the 1024 token limit for LingBot-Map and understand when state resetting is necessary for optimal performance. Learn about its default parameters.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: deep-dive
- Published: 2026-06-29

---

**LingBot-Map enforces a hard maximum sequence length of 1024 tokens (or frames) before requiring a state reset, governed by default parameters in the rotary position embedding layers and streaming aggregator.**

LingBot-Map is an open-source robotics foundation model designed for sequential visual-linguistic reasoning. Understanding the maximum sequence length supported by LingBot-Map is essential for deploying long-horizon tasks, as the architecture relies on pre-computed positional encodings that cap effective context windows at 1024 entries.

## Where the 1024 Limit is Defined

The 1024-token ceiling originates from two architectural components that must stay synchronized: the 2‑D Rotary Position Embedding (RoPE) system and the 3‑D streaming aggregator.

### Rotary Position Embedding (2‑D RoPE)

In [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py), the `RotaryPositionEmbedding2D` class initializes sinusoidal frequency tables using a default sequence length of 1024. The `_compute_frequency_components` method signature includes `seq_len: int = 1024`, which sets the upper bound for position indices that the model can distinguish. Attempting to process positions beyond this pre-computed range results in invalid embeddings or out-of-bounds errors.

### Streaming Aggregator (3‑D RoPE)

In [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), the `AggregatorStream` class manages the KV-cache for streaming inference. Its constructor accepts a `max_frame_num` parameter that defaults to 1024:

```python
def __init__(self, ..., max_frame_num: int = 1024, ...):

```

This value propagates through the model hierarchy—specifically to `gct_stream` and `gct_stream_window` classes—where it is stored as `self.max_frame_num`. When the frame counter exceeds this threshold, the aggregator triggers cache management routines.

## What Happens When the Limit is Exceeded

When the number of processed tokens or frames surpasses 1024, the internal KV-cache is considered full. According to the implementation in [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), the system handles overflow in one of two ways:

- **Sliding-window eviction**: If configured, older entries are discarded to maintain the 1024 most recent frames.
- **State reset**: The `clean_kv_cache()` method clears the entire cache, forcing the model to restart state tracking from the current frame.

This mechanism ensures that the RoPE embeddings remain valid, as the position indices cannot exceed the pre-computed frequency tables.

## How to Configure or Override the Limit

While 1024 is the default, you can instantiate components with custom limits. However, increasing `max_frame_num` requires corresponding adjustments to the RoPE frequency computations to avoid shape mismatches.

### Inspect Default Values

```python

# Verify the default 2‑D RoPE sequence length

from lingbot_map.layers.rope import RotaryPositionEmbedding2D

rope = RotaryPositionEmbedding2D()
print("Default max_seq_len:", rope._compute_frequency_components.__defaults__[1])

# Output: 1024

```

### Check Streaming Aggregator Defaults

```python
from lingbot_map.aggregator.stream import AggregatorStream

agg = AggregatorStream(enable_3d_rope=True)
print("Streaming max_frame_num:", agg.max_frame_num)

# Output: 1024

```

### Override with Caution

```python

# Create aggregator with custom limit (requires RoPE cache adjustment)

agg = AggregatorStream(enable_3d_rope=True, max_frame_num=2048)
print("Custom max_frame_num:", agg.max_frame_num)

# Output: 2048

```

**Note**: When overriding, you must also regenerate the RoPE frequency tables with the new `seq_len` to ensure positional encodings cover the extended range.

## Summary

- **LingBot-Map supports a maximum sequence length of 1024 tokens** before requiring state reset or cache eviction.
- The limit is defined in [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) (2‑D RoPE) and [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py) (streaming aggregator).
- Exceeding 1024 frames triggers `clean_kv_cache()` or sliding-window eviction to maintain positional encoding validity.
- Override via `max_frame_num` parameter requires synchronized adjustments to RoPE pre-computations.

## Frequently Asked Questions

### Can I increase the maximum sequence length beyond 1024?

Yes, you can pass a custom `max_frame_num` to `AggregatorStream` or adjust the `seq_len` parameter in `RotaryPositionEmbedding2D`. However, you must regenerate the sinusoidal frequency tables to match the new length, and you may need to adjust memory allocation for the KV-cache accordingly.

### What triggers a state reset in LingBot-Map?

A state reset occurs automatically when the frame counter exceeds `max_frame_num` (default 1024) and theKV-cache reaches capacity. The `clean_kv_cache()` method clears accumulated states, or the system evicts older frames if sliding-window mode is enabled.

### Is the 1024 limit configurable via command line?

Yes. The repository includes CLI parsers in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) that expose `--max_frame_num` as a command-line argument with a default value of 1024, allowing runtime configuration without modifying source code.

### Does the sequence limit apply to all LingBot-Map variants?

Yes. Both the base `gct_stream` and windowed `gct_stream_window` variants inherit the `max_frame_num` constraint from the streaming aggregator architecture, ensuring consistent behavior across model configurations.