How to Customize Camera Paths in LingBot-Map's Offline Rendering Pipeline Using YAML Presets

You customize camera paths in LingBot-Map by defining a camera block in a YAML configuration file that specifies field-of-view, transition frames, and an ordered list of motion segments (follow, birdeye, flyby, or keyframes), then passing it to the offline pipeline via the --camera_config flag.

The LingBot-Map repository (Robbyant/lingbot-map) provides an offline rendering system for generating cinematic videos from reconstructed scenes. To customize camera paths in LingBot-Map's offline rendering pipeline using YAML presets, you manipulate the camera configuration section that defines how the virtual camera moves through the scene via keyframe-based interpolation.

Understanding the Camera Path Architecture

PipelineConfig and YAML Parsing

Located in demo_render/rgbd_render/config.py, the PipelineConfig class parses the complete YAML file and stores the camera block in cfg.camera. This dataclass supports merging multiple configuration files, allowing you to override specific sections without duplicating the entire config via the from_yaml() method.

CameraPath and Keyframe Generation

The CameraPath class in demo_render/rgbd_render/camera.py manages the actual motion. It holds a list of keyframes and provides __getitem__ access so the renderer can query exact camera poses for any frame index. The module also contains preset generators (follow, birdeye, flyby, keyframes) that translate high-level segment descriptions into concrete keyframes, and the build_camera_path function that orchestrates the translation of YAML configuration into a CameraPath object.

OfflinePipeline Execution

The OfflinePipeline in demo_render/rgbd_render/pipeline/offline.py orchestrates rendering. It instantiates the renderer, builds the camera path via build_camera_path(cfg.camera, scene), and iterates through each frame calling renderer.render_frame(scene, camera_path[i], ...) for every frame index.

YAML Configuration Structure for Camera Paths

The camera block in your YAML file supports three top-level fields:

• fov (float): Field-of-view in degrees applied to all frames (e.g., fov: 70.0)
• transition (int): Number of frames for smooth interpolation between segments (e.g., transition: 20)
• segments (list): Ordered list of motion presets, each requiring a mode parameter

Available Preset Modes

follow: Tracks the reconstructed camera trajectory from a side view.

• Required parameters: frames, back_offset, up_offset, look_offset, smooth_window

birdeye: Rises to a static top-down overview.

• Required parameters: frames, reveal_height_mult

flyby: Moves linearly between two world-space points while looking at a target.

• Required parameters: frames, start, end, look_at, smooth_window

keyframes: Directly specifies absolute camera poses without algorithmic generation.

• Required parameters: frames, keyframes (list of {pos: [x,y,z], wxyz: [w,x,y,z]} objects)

All frame ranges use relative indexing where -1 indicates the final frame of the sequence.

Creating a Custom Camera Path YAML Preset

Here is a complete example combining multiple motion types with smooth transitions:

camera:
  fov: 65.0
  transition: 15

  segments:
    # Follow the reconstruction trajectory for the first 2000 frames

    - mode: follow
      frames: [0, 2000]
      back_offset: 0.4
      up_offset: 0.1
      look_offset: 0.5
      smooth_window: 30

    # Transition to a bird's-eye view for the remainder

    - mode: birdeye
      frames: [2000, -1]
      reveal_height_mult: 3.0

    # Optional: Add a custom flyby segment

    - mode: flyby
      frames: [3500, 4000]
      start: [5.0, 1.0, 2.0]
      end: [5.0, 1.0, -2.0]
      look_at: [0.0, 0.0, 0.0]
      smooth_window: 20

How the interpolation works internally: When build_camera_path processes this configuration, it first calls the follow() generator to create keyframes that maintain a fixed offset behind the reconstructed trajectory, applying a smoothing window of 30 frames to reduce jitter. At frame 2000, the system inserts 15 transition frames (controlled by transition: 15) that linearly interpolate from the final follow pose to the birdeye position calculated at reveal_height_mult times the scene's bounding-box height above the center. The optional flyby segment creates a linear trajectory between start and end coordinates while maintaining orientation toward the look_at point.

Executing the Offline Pipeline with Your Preset

To render using your custom camera configuration:

python demo_render/rgbd_scan_render.py \
    --config demo_render/config/indoor.yaml \
    --camera_config my_camera.yaml \
    --input_folder example/university \
    --output out.mp4

The --camera_config flag merges your YAML file with the base configuration, overriding only the camera section while preserving scene, preprocessing, and rendering parameters from the base file defined in demo_render/config/indoor.yaml.

After completion, inspect the concrete path in the automatically generated <output>_config.yaml (e.g., out_config.yaml), which contains the flattened keyframe list for reproducibility:

cat out_config.yaml

Programmatic Camera Path Generation (Optional)

For dynamic generation without YAML files, use the preset generators directly:

from demo_render.rgbd_render.camera import CameraPath, follow, birdeye

# Build keyframes manually

keyframes = []
keyframes.extend(follow(
    frames=[0, 3000],
    back_offset=0.35,
    up_offset=0.12,
    look_offset=0.45,
    smooth_window=25,
    scene=scene,
    total_frames=5000,
))
keyframes.extend(birdeye(
    frames=[3000, -1],
    reveal_height_mult=2.5,
    scene=scene,
    total_frames=5000,
))

camera_path = CameraPath(keyframes, 5000, interpolation='linear')

Summary

• Customize camera paths in LingBot-Map by defining segments in a YAML file under the camera block with fov, transition, and segments parameters as implemented in demo_render/rgbd_render/config.py.
• Available modes in demo_render/rgbd_render/camera.py include follow, birdeye, flyby, and direct keyframes for precise control over camera motion.
• The build_camera_path function translates YAML segments into a CameraPath object that interpolates between keyframes using the specified transition frame count between segments.
• Use the --camera_config command-line flag to override camera settings when running demo_render/rgbd_scan_render.py.
• Final camera trajectories are archived in <output>_config.yaml for deterministic replay and debugging.

Frequently Asked Questions

What is the difference between follow and birdeye camera modes in LingBot-Map?

The follow mode generates a camera path that tracks alongside the reconstructed trajectory from a configurable side offset, maintaining relative position to the movement through the scene using parameters like back_offset and up_offset. The birdeye mode creates a static overhead view by positioning the camera at a height multiplier (reveal_height_mult) above the scene center, typically used for revealing the complete map layout after a traversal sequence.

How do I smoothly transition between different camera movements in the YAML preset?

Specify the transition parameter in the root camera block to define how many frames the system uses to interpolate between consecutive segments. For example, transition: 20 creates a 20-frame linear interpolation from the final pose of the first segment to the starting pose of the next segment, ensuring cinematic continuity between different motion types like follow and birdeye.

Can I combine manual keyframes with automated presets like flyby?

Yes, you can mix modes within the segments list. Define a segment with mode: keyframes containing explicit pos and wxyz coordinates, followed by a flyby or birdeye segment. The build_camera_path function in demo_render/rgbd_render/camera.py processes each segment sequentially and applies the global transition setting between different motion types to ensure smooth handoffs.

Where does LingBot-Map save the actual camera path after rendering?

The offline pipeline writes a complete snapshot of the interpolated camera path to <output>_config.yaml (e.g., out_config.yaml if your output was out.mp4). This file contains the total_frames, interpolation method, and the full list of keyframe positions and orientations, allowing you to reload the exact path using CameraPath.load() for future reproduction or inspection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}