How to Resolve Out-of-Memory Errors in LingBot-Map Using --offload_to_cpu and --num_scale_frames

Use --offload_to_cpu (enabled by default) to move per-frame predictions from GPU to CPU after each forward pass, and reduce --num_scale_frames from its default of 8 to a lower value like 2 to shrink the initial scale-phase memory spike.

LingBot-Map is a streaming 3-D reconstruction system that maintains a paged KV-cache growing with every stored keyframe. When processing long sequences on GPUs with limited VRAM, this cache can exceed available memory, causing out-of-memory (OOM) errors during inference. The repository provides two specific command-line flags to manage this memory budget without sacrificing reconstruction quality.

Understanding the Memory Bottleneck

The streaming architecture in Robbyant/lingbot-map stores intermediate activations and keyframe data in GPU memory throughout the reconstruction process. Two distinct phases consume VRAM:

1. The scale phase processes an initial batch of frames (default 8) to estimate global scene scale before the main streaming loop begins.
2. The KV-cache accumulates per-frame predictions and gradients for every keyframe stored during streaming.

When either the temporary activation memory from the scale phase or the persistent KV-cache grows too large, the process aborts with an OOM error. The flags --offload_to_cpu and --num_scale_frames target these specific bottlenecks.

Using --offload_to_cpu to Manage GPU Memory

The --offload_to_cpu flag controls whether dense per-frame predictions (depth maps, color maps, and their gradients) remain on the GPU after the forward pass.

This flag is enabled by default. When active, the implementation in demo.py immediately moves each prediction tensor to CPU memory via tensor.to('cpu'), freeing the GPU footprint that would otherwise accumulate across thousands of frames. This keeps GPU memory usage roughly constant regardless of sequence length or image resolution.

You can disable offloading only if you have abundant VRAM:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky \
    --no-offload_to_cpu

Tuning --num_scale_frames for the Scale Phase

The --num_scale_frames parameter controls how many frames the model processes during the initial scale estimation stage. The default value is 8, which provides accurate initial scale estimation but creates a temporary memory spike at the beginning of the pipeline.

Reducing this value lowers the peak activation memory during the scale phase, though it may produce slightly less accurate initial scale estimates. For GPUs with 8 GB of VRAM or when processing high-resolution sequences, reduce the value to 2 or 4:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky \
    --num_scale_frames 2

Combining Both Flags for Maximum Efficiency

For processing very long sequences on modest GPUs, combine both optimizations with a tuned keyframe interval. While --keyframe_interval limits how many frames persist in the KV-cache, --offload_to_cpu ensures non-keyframe predictions do not accumulate on the GPU, and --num_scale_frames minimizes the initial memory spike.

As implemented in the argument parser of demo.py, these flags work hand-in-hand to enable processing thousands of frames on GPUs with as little as 8 GB of VRAM:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky \
    --keyframe_interval 4 \
    --offload_to_cpu \
    --num_scale_frames 2

The same flag handling appears in benchmark/run.py for batch processing workflows.

Implementation Details

The memory management logic resides in the repository's CLI entry points. In demo.py, the argument parser registers these flags and forwards values to the LingBotMap inference class, which internally decides whether to invoke the CPU offloading after each forward pass.

According to the Performance & Memory section in README.md, the interaction between these flags and the KV-cache follows this priority:

• Offloading predictions maintains constant GPU usage by moving large per-frame tensors to host memory immediately after generation.
• Shrinking the scale phase reduces the temporary memory spike before the streaming loop begins.

Summary

• Enable --offload_to_cpu (default behavior) to move dense predictions and gradients to CPU memory after each frame, preventing accumulation on the GPU.
• Reduce --num_scale_frames from 8 to 2 or 4 to lower the peak memory consumption during the initial scale estimation phase.
• Combine with --keyframe_interval to limit KV-cache growth and enable processing long sequences on GPUs with limited VRAM.
• Reference demo.py and benchmark/run.py for the argument parser implementation and README.md for detailed memory budget explanations.

Frequently Asked Questions

What exactly does --offload_to_cpu move to CPU memory?

The flag triggers the tensor.to('cpu') operation on dense per-frame predictions including depth maps, color maps, and their associated gradients immediately after the forward pass completes. This prevents the full-resolution activations from occupying GPU memory throughout the streaming process, while the smaller KV-cache tensors remain on GPU for fast attention operations.

How low can I safely set --num_scale_frames?

You can reduce --num_scale_frames to as low as 2, though this trades off initial scale estimation accuracy for memory savings. The default of 8 provides robust global scale estimation, but values of 2-4 work well for most scenes while significantly reducing the temporary memory spike at startup.

When should I disable --offload_to_cpu with --no-offload_to_cpu?

Only disable CPU offloading if your GPU has abundant VRAM (typically 24 GB or more) and you require maximum inference speed. Keeping tensors on GPU eliminates the host-to-device transfer overhead, but causes memory usage to scale linearly with sequence length, inevitably leading to OOM errors on long sequences.

How do these flags interact with --keyframe_interval?

While --keyframe_interval controls how many historical frames persist in the paged KV-cache, --offload_to_cpu manages the memory footprint of non-keyframe predictions that would otherwise accumulate during the forward pass. Together, they provide complementary memory management: the keyframe interval limits permanent cache growth, while offloading prevents temporary activation buildup.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}