Maximum Sequence Length Supported by LingBot-Map: 1024 Token Limit Explained

LingBot-Map enforces a hard maximum sequence length of 1024 tokens (or frames) before requiring a state reset, governed by default parameters in the rotary position embedding layers and streaming aggregator.

LingBot-Map is an open-source robotics foundation model designed for sequential visual-linguistic reasoning. Understanding the maximum sequence length supported by LingBot-Map is essential for deploying long-horizon tasks, as the architecture relies on pre-computed positional encodings that cap effective context windows at 1024 entries.

Where the 1024 Limit is Defined

The 1024-token ceiling originates from two architectural components that must stay synchronized: the 2‑D Rotary Position Embedding (RoPE) system and the 3‑D streaming aggregator.

Rotary Position Embedding (2‑D RoPE)

In lingbot_map/layers/rope.py, the RotaryPositionEmbedding2D class initializes sinusoidal frequency tables using a default sequence length of 1024. The _compute_frequency_components method signature includes seq_len: int = 1024, which sets the upper bound for position indices that the model can distinguish. Attempting to process positions beyond this pre-computed range results in invalid embeddings or out-of-bounds errors.

Streaming Aggregator (3‑D RoPE)

In lingbot_map/aggregator/stream.py, the AggregatorStream class manages the KV-cache for streaming inference. Its constructor accepts a max_frame_num parameter that defaults to 1024:

def __init__(self, ..., max_frame_num: int = 1024, ...):

This value propagates through the model hierarchy—specifically to gct_stream and gct_stream_window classes—where it is stored as self.max_frame_num. When the frame counter exceeds this threshold, the aggregator triggers cache management routines.

What Happens When the Limit is Exceeded

When the number of processed tokens or frames surpasses 1024, the internal KV-cache is considered full. According to the implementation in lingbot_map/aggregator/stream.py, the system handles overflow in one of two ways:

• Sliding-window eviction: If configured, older entries are discarded to maintain the 1024 most recent frames.
• State reset: The clean_kv_cache() method clears the entire cache, forcing the model to restart state tracking from the current frame.

This mechanism ensures that the RoPE embeddings remain valid, as the position indices cannot exceed the pre-computed frequency tables.

How to Configure or Override the Limit

While 1024 is the default, you can instantiate components with custom limits. However, increasing max_frame_num requires corresponding adjustments to the RoPE frequency computations to avoid shape mismatches.

Inspect Default Values

# Verify the default 2‑D RoPE sequence length

from lingbot_map.layers.rope import RotaryPositionEmbedding2D

rope = RotaryPositionEmbedding2D()
print("Default max_seq_len:", rope._compute_frequency_components.__defaults__[1])

# Output: 1024

Check Streaming Aggregator Defaults

from lingbot_map.aggregator.stream import AggregatorStream

agg = AggregatorStream(enable_3d_rope=True)
print("Streaming max_frame_num:", agg.max_frame_num)

# Output: 1024

Override with Caution

# Create aggregator with custom limit (requires RoPE cache adjustment)

agg = AggregatorStream(enable_3d_rope=True, max_frame_num=2048)
print("Custom max_frame_num:", agg.max_frame_num)

# Output: 2048

Note: When overriding, you must also regenerate the RoPE frequency tables with the new seq_len to ensure positional encodings cover the extended range.

Summary

• LingBot-Map supports a maximum sequence length of 1024 tokens before requiring state reset or cache eviction.
• The limit is defined in lingbot_map/layers/rope.py (2‑D RoPE) and lingbot_map/aggregator/stream.py (streaming aggregator).
• Exceeding 1024 frames triggers clean_kv_cache() or sliding-window eviction to maintain positional encoding validity.
• Override via max_frame_num parameter requires synchronized adjustments to RoPE pre-computations.

Frequently Asked Questions

Can I increase the maximum sequence length beyond 1024?

Yes, you can pass a custom max_frame_num to AggregatorStream or adjust the seq_len parameter in RotaryPositionEmbedding2D. However, you must regenerate the sinusoidal frequency tables to match the new length, and you may need to adjust memory allocation for the KV-cache accordingly.

What triggers a state reset in LingBot-Map?

A state reset occurs automatically when the frame counter exceeds max_frame_num (default 1024) and theKV-cache reaches capacity. The clean_kv_cache() method clears accumulated states, or the system evicts older frames if sliding-window mode is enabled.

Is the 1024 limit configurable via command line?

Yes. The repository includes CLI parsers in demo.py that expose --max_frame_num as a command-line argument with a default value of 1024, allowing runtime configuration without modifying source code.

Does the sequence limit apply to all LingBot-Map variants?

Yes. Both the base gct_stream and windowed gct_stream_window variants inherit the max_frame_num constraint from the streaming aggregator architecture, ensuring consistent behavior across model configurations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}