How to Reduce Bidirectional Scale Frames in LingBot-Map: A Memory Optimization Guide

Pass the --num_scale_frames argument with a value lower than the default 8 (e.g., --num_scale_frames 2) to shrink the initial bidirectional attention block and reduce GPU memory consumption during the scale estimation phase.

LingBot-Map processes an initial window of frames using bidirectional attention to estimate global scale before switching to streaming mode. By default, this scale estimation phase uses 8 bidirectional scale frames, which can create a significant activation peak on GPUs with limited VRAM. Reducing this value allows the model to run on constrained hardware while maintaining the core mapping functionality.

What Are Bidirectional Scale Frames?

In the streaming pipeline, the inference_streaming function in lingbot_map/models/gct_stream_window_v2.py extracts a fixed number of initial frames—controlled by the num_scale_frames parameter—to compute a global scale factor. These frames are processed together with a scale token that enables bidirectional attention among them, allowing the model to establish spatial relationships across multiple viewpoints before processing the remainder of the sequence in a causal manner.

Default Configuration and Memory Impact

The default value of num_scale_frames = 8 creates a relatively large activation peak during the initial forward pass. In gct_stream_window_v2.py, the code resolves the actual number of scale frames using scale_frames = min(num_scale_frames, S), where S represents the available frames. This bidirectional block stores intermediate activations for all 8 frames simultaneously, which directly impacts GPU memory usage.

How to Reduce Scale Frames via Command Line

The --num_scale_frames argument is exposed in the primary entry points: demo.py, demo_render/batch_demo.py, and the benchmark scripts.

Basic Inference with Reduced Scale Frames

python demo.py \
    --model_path /path/to/lingbot-map.pt \
    --image_folder /path/to/images/ \
    --num_scale_frames 2

This command overrides the default 8-frame bidirectional block with a 2-frame window, immediately cutting the activation memory for the initial scale phase.

Rendering Pipeline Configuration

When processing video through the rendering pipeline, apply the same flag:

python demo_render/batch_demo.py \
    --video_path /data/video.mp4 \
    --output_folder /data/outputs/ \
    --model_path /path/to/lingbot-map.pt \
    --num_scale_frames 2 \
    --mode windowed --window_size 128 \
    --keyframe_interval 13

Programmatic Configuration in Python

For benchmark scripts or custom implementations, set the value before initializing the inference:

args = parser.parse_args()
args.num_scale_frames = 2        # Reduce bidirectional scale phase

benchmark = LingBotMapBenchmark(...)
benchmark.run(args)

All internal calls forward this parameter to inference_streaming, where the scale frame count is resolved before building the attention blocks.

Performance and Accuracy Trade-offs

Reducing the bidirectional scale window affects three key metrics:

• Memory Usage: Lower peak VRAM consumption during the initial phase, as fewer intermediate activations are stored.
• Latency: Slightly faster start-up time, since the scale block processes fewer frames in its single forward pass.
• Accuracy: Potential modest loss of scale estimation accuracy in scenes requiring diverse viewpoints; however, values of 2–4 frames are typically sufficient for standard indoor and outdoor videos according to the repository documentation.

Summary

• LingBot-Map uses 8 bidirectional scale frames by default to estimate global scale before streaming.
• Reduce this value by passing --num_scale_frames N to demo.py or demo_render/batch_demo.py.
• The parameter is consumed in lingbot_map/models/gct_stream_window_v2.py by the inference_streaming function.
• Lower values decrease GPU memory usage and initial latency but may marginally affect scale accuracy in complex scenes.

Frequently Asked Questions

How low can I set --num_scale_frames without breaking the model?

You can set values as low as 2 or even 1, provided at least one frame is available for scale estimation. The code in gct_stream_window_v2.py uses min(num_scale_frames, S) to ensure it never requests more frames than exist in the input sequence. Values of 2–4 are officially recommended for limited GPU memory scenarios.

Does reducing scale frames affect the quality of the final 3D reconstruction?

The impact is generally limited to the initial scale estimation accuracy. While fewer frames provide less viewpoint diversity for global scale calculation, the subsequent streaming phase processes the full sequence. In practice, the difference is negligible for most indoor and outdoor videos, though complex scenes with extreme scale variations may benefit from the default 8 frames.

Where exactly is the bidirectional attention computed for scale frames?

The bidirectional attention occurs in the inference_streaming function within lingbot_map/models/gct_stream_window_v2.py. This function extracts the first num_scale_frames frames and processes them with a special scale token that allows full cross-attention between these initial frames, unlike the causal attention used for the remainder of the sequence.

Can I modify the default value without using the command-line argument?

Yes. You can modify the default directly in the model configuration or argument parser definitions in demo.py or demo_render/batch_demo.py. Alternatively, when using the benchmark API programmatically, set args.num_scale_frames before calling the benchmark runner, as shown in the code examples above.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}