# How to Implement Custom Camera Modes in LingBot-Map's Render Pipeline

> Learn to implement custom camera modes in LingBot-Map. Create a factory, register it, and expose parameters to enhance your rendering pipeline.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-06-29

---

**You implement custom camera modes in LingBot-Map by creating a factory function in [`demo_render/rgbd_render/camera.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/camera.py) that returns a `CameraPath`, registering it in the `build_camera_path` orchestrator, and exposing parameters through the `CameraSegment` dataclass in [`config.py`](https://github.com/Robbyant/lingbot-map/blob/main/config.py).**

LingBot-Map orchestrates camera trajectories through a config-driven pipeline that decouples motion planning from OpenGL rendering. The system processes `CameraSegment` definitions—loaded via `PipelineConfig.from_yaml` and `load_config`—to generate interpolated `CameraPath` objects, feeding per-frame `Camera` poses to the renderer. By extending the pipeline's factory methods, you can introduce entirely new camera behaviors without modifying the core rendering logic in [`demo_render/rgbd_render/renderer.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/renderer.py).

## Understanding the Camera Pipeline Architecture

The architecture separates concerns across four key components:

- **`Camera` and `CameraPath`** (in [`demo_render/rgbd_render/camera.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/camera.py)): Represent a single pose and an interpolated sequence of poses, respectively.
- **`CameraSegment` and `CameraConfig`** (in [`demo_render/rgbd_render/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/config.py)): Dataclasses that declare *what* mode to use, *when* (frame range), and *how* (mode-specific parameters).
- **`build_camera_path`** (in [`demo_render/rgbd_render/camera.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/camera.py)): Orchestrates segments, instantiates concrete paths per segment, and stitches them with optional transitions.
- **Render loop** (in [`demo_render/rgbd_render/renderer.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/renderer.py)): Consumes the final `CameraPath` and feeds each `Camera` to the OpenGL renderer.

## How Camera Modes Generate Motion

The pipeline processes segments through four distinct phases:

1. **Configuration**: You define a `CameraSegment` with a `mode` string (e.g., `follow`, `birdeye`, `static`, `pivot`) via YAML or CLI.
2. **Instantiation**: `build_camera_path` (lines 332-363) switches on `seg.mode` and calls a factory like `make_follow_path` or `make_birdeye_path`.
3. **Keyframe Generation**: Each factory returns a `CameraPath` capable of producing a `Camera` for any frame index via `CameraPath.get_camera`.
4. **Segment Blending**: The builder handles smooth transitions between adjacent segments using smoothstep interpolation.

## Implementing a Custom Mode: Step-by-Step

To add a custom mode such as **"orbit"** (camera circles around the scene center), follow this implementation path.

### Step 1: Extend the Configuration Dataclass

If your mode requires extra parameters, add fields to `CameraSegment` in [`demo_render/rgbd_render/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/config.py):

```python
@dataclass
class CameraSegment:
    mode: str
    frames: List[int]
    # ... existing fields ...

    orbit_radius: float = 2.0
    orbit_speed: float = 1.0

```

### Step 2: Implement the Factory Function

Create a factory function in [`demo_render/rgbd_render/camera.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/camera.py) that computes per-frame poses and returns a `CameraPath`:

```python
def make_orbit_path(c2w_poses: np.ndarray,
                    total_frames: int,
                    radius: float = 2.0,
                    speed: float = 1.0,
                    fov_deg: float = 60.0) -> CameraPath:
    """Orbit camera around scene centre on a horizontal circle."""
    centre = c2w_poses[:, :3, 3].mean(axis=0).astype(np.float32)
    up = np.array([0, 1, 0], dtype=np.float32)

    keyframes = []
    for fi in range(total_frames):
        angle = 2 * np.pi * (fi / total_frames) * speed
        eye = centre + radius * np.array([np.cos(angle), 0, np.sin(angle)],
                                         dtype=np.float32)
        keyframes.append({
            'frame': fi,
            'eye': eye.tolist(),
            'center': centre.tolist(),
            'up': up.tolist(),
            'fov': fov_deg,
        })
    return CameraPath(keyframes, total_frames, interpolation='linear')

```

### Step 3: Register in the Path Builder

Extend the mode cascade in `build_camera_path` (around line 690) to wire the factory to the configuration:

```python
elif seg.mode == 'orbit':
    path = make_orbit_path(
        scene.c2w_poses,
        total_frames,
        radius=getattr(seg, 'orbit_radius', 2.0),
        speed=getattr(seg, 'orbit_speed', 1.0),
        fov_deg=fov,
    )
    seg_paths.append((s, e, path))

```

### Step 4: Configure via YAML or CLI

Reference the new mode in your pipeline configuration:

```yaml
camera:
  fov: 60
  transition: 30
  segments:
    - mode: orbit
      frames: [0, -1]
      orbit_radius: 2.0
      orbit_speed: 1.0

```

Alternatively, pass flat arguments via CLI (mapped automatically by `_build_segments_from_flat_args`):

```bash
python demo_render/demo.py \
    --config my_config.yaml \
    --orbit_radius 3.5 \
    --orbit_speed 0.8

```

## Handling Transitions Between Segments

Transitions between adjacent segments occur in `build_camera_path` (lines 422-476). The system uses `_smoothstep` and `_lerp_camera` to blend camera poses across the transition frame range. If your mode requires a different blending strategy, modify the blend computation block (around line 546) or supply a custom `interpolation` value in the `CameraPath` constructor.

## Summary

Implementing **custom camera modes** in LingBot-Map requires understanding the separation between configuration and execution:

- **Configuration**: Extend `CameraSegment` in [`config.py`](https://github.com/Robbyant/lingbot-map/blob/main/config.py) to declare mode-specific parameters.
- **Factory**: Implement a builder function in [`camera.py`](https://github.com/Robbyant/lingbot-map/blob/main/camera.py) that generates `CameraPath` keyframes.
- **Registration**: Hook the factory into the `build_camera_path` orchestrator's mode selection cascade.
- **Usage**: Reference the mode in YAML configs or CLI arguments to trigger the new behavior.

## Frequently Asked Questions

### Where do I define the parameters for my custom camera mode?

Define parameters in the `CameraSegment` dataclass in [`demo_render/rgbd_render/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/config.py). Add typed fields with default values to ensure backward compatibility with existing configurations that may not specify your new parameters.

### How does the render loop access my custom camera poses?

The render loop in [`demo_render/rgbd_render/renderer.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/rgbd_render/renderer.py) consumes the final `CameraPath` object produced by the pipeline. It calls `CameraPath.get_camera(frame_idx)` for each frame, which returns the interpolated `Camera` pose containing the `eye`, `center`, `up`, and `fov` values your factory function computed.

### Can I combine multiple custom modes in a single render?

Yes. `build_camera_path` processes a list of `CameraSegment` objects sequentially. Define multiple segments with different modes in your YAML configuration, and the orchestrator automatically stitches them together using the smooth transition logic defined in the path builder.

### What interpolation methods are available for camera paths?

The `CameraPath` constructor accepts an `interpolation` parameter, typically set to `'linear'` in factory functions. The transition blending between segments uses `_smoothstep` and `_lerp_camera` as implemented in `build_camera_path` (lines 422-476), ensuring smooth handoffs between different camera motions.