# Common Use Cases for camera_head.py and dpt_head.py in Ling-Bot Map

> Explore common use cases for camera_head.py and dpt_head.py in Ling-Bot Map. Learn how these scripts refine 9-DoF camera poses for SLAM and generate dense depth maps for 3D perception.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-07

---

**Both files implement decoder-only prediction heads that transform transformer token outputs into actionable 3-D perception data: camera_head.py iteratively refines 9-DoF camera poses for SLAM and robotics, while dpt_head.py generates dense depth maps with confidence estimates for monocular reconstruction and scene understanding.**

The Ling-Bot Map repository provides a modular transformer backbone for 3-D perception tasks. The [`camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/camera_head.py) and [`dpt_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/dpt_head.py) modules serve as specialized heads that interpret high-level token representations—producing either camera trajectories or per-pixel depth—without modifying the underlying backbone weights.

## CameraHead: Visual Odometry and Pose Estimation

Located in [`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py), the **CameraHead** processes the camera token (the first token of each frame) to produce accurate 9-dimensional pose encodings. These comprise 3-D translation, 4-D quaternion rotation, and 2-D field-of-view parameters essential for real-time spatial tracking.

### Visual Odometry and SLAM

The head’s core innovation is **iterative pose refinement**. The `trunk_fn` method loops `num_iterations` times, feeding previous estimates back through transformer blocks alongside a learned `self.empty_pose_tokens` embedding. This architecture supports **visual odometry** and **SLAM** (Simultaneous Localization and Mapping) by progressively refining camera extrinsics across video sequences.

### Robotics Navigation and AR/VR Applications

CameraHead outputs feed directly into motion-planning modules for **robotics navigation** and rendering engines for **AR/VR** tracking. The adaptive layer-normalization system (`self.poseLN_modulation`) and optional 3-D rotary positional embeddings (`self.rope3d`, lines 106-120) enable precise temporal encoding critical for stable trajectory estimation in dynamic environments.

## DPTHead: Dense Depth Prediction

Defined in [`lingbot_map/heads/dpt_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/dpt_head.py), the **DPTHead** (Depth Prediction Transformer) fuses multi-scale transformer features to output dense maps. Unlike CameraHead’s sparse pose output, DPTHead generates per-pixel predictions suitable for 3-D scene reconstruction.

### Monocular Depth Estimation

The head implements multi-scale feature fusion using 1×1 convolutions (`self.projects`, lines 68-71) to project tokens from multiple transformer layers into 2-D feature maps. Progressive upsampling via `self.resize_layers` (lines 74-84) and RefineNet-style fusion blocks (`_make_fusion_block`) restore full spatial resolution, enabling **monocular depth estimation** from single RGB frames.

### Dense Reconstruction and Confidence-Aware Sensing

DPTHead supports **dense scene reconstruction** by generating depth maps that merge into 3-D models. The optional `conf_activation` parameter produces confidence channels for uncertain regions, while the configurable `output_dim` allows the same architecture to emit surface normals or segmentation masks for multi-task extensions.

## Architectural Implementation Details

Both heads follow a decoder-only design, meaning they attach to any Ling-Bot Map backbone that yields token sequences without requiring backbone modifications.

### CameraHead Technical Specifications

In [`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py), the implementation relies on:

- **`modulate()`** (lines 49-55): Applies learned scale and shift to tokens, similar to Diffusion Transformer (DiT) implementations.
- **`self.adaln_norm`** (lines 73-74): Adaptive layer normalization without affine parameters, gated by `gate_msa` to control modulation strength.
- **`activate_pose()`** (lines 140-144): Applies separate activations to translation (`trans_act`), quaternion (`quat_act`), and field-of-view (`fl_act`) components.
- **3-D RoPE**: Optional rotary positional embeddings from [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) when `enable_3d_rope=True`.

### DPTHead Technical Specifications

The `DPTHead` class in [`lingbot_map/heads/dpt_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/dpt_head.py) implements dense prediction through:

- **Multi-scale projection**: `self.projects` ModuleList converting tokens to feature maps.
- **RefineNet fusion**: `_make_scratch` and `_make_fusion_block` merge four scale-specific features (see `scratch_forward`, lines 64-78).
- **Positional embedding**: `_apply_pos_embed` (lines 252-262) adds UV-grid coordinate biases.
- **Output activation**: `activate_head()` (lines 445-447) in [`lingbot_map/heads/head_act.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/head_act.py) processes final depth and confidence tensors.

## Practical Code Examples

Below are minimal, runnable implementations showing how to instantiate each head with simulated backbone outputs.

### Instantiating CameraHead for Pose Estimation

```python
import torch
from lingbot_map.heads.camera_head import CameraHead

# Simulated final transformer tokens: (batch, seq_len, channels)

B, N, C = 2, 64, 2048
last_tokens = torch.randn(B, N, C)

# Initialize head with specific activation functions

camera_head = CameraHead(
    dim_in=C,
    trunk_depth=4,
    num_heads=16,
    trans_act="linear",
    quat_act="linear",
    fl_act="relu"
)

# Run iterative refinement (4 iterations)

pose_encodings = camera_head.forward(
    [last_tokens],  # aggregated_tokens_list expects list

    num_iterations=4
)

# Output: list of tensors, one per iteration, each shape (B, 1, 9)

for i, enc in enumerate(pose_encodings):
    print(f"Iteration {i+1}: {enc.shape}")  # 9-DOF: [tx, ty, tz, qx, qy, qz, qw, fx, fy]

```

### Instantiating DPTHead for Depth Prediction

```python
import torch
from lingbot_map.heads.dpt_head import DPTHead

B, N, C = 2, 64, 2048
num_layers = 4

# Simulate tokens from multiple transformer layers

tokens_per_layer = [torch.randn(B, N, C) for _ in range(num_layers)]
images = torch.rand(B, N, 3, 224, 224)  # Required for patch calculations

dpt_head = DPTHead(
    dim_in=C,
    patch_size=14,
    output_dim=1,              # Single channel depth

    activation="inv_log",      # Inverse log depth activation

    conf_activation="expp1",   # Confidence activation

    features=256,
    out_channels=[256, 512, 1024, 1024],
    intermediate_layer_idx=[0, 1, 2, 3],
    pos_embed=True,
    feature_only=False,
    down_ratio=1,
)

# Generate depth and confidence maps

depth, confidence = dpt_head(
    aggregated_tokens_list=tokens_per_layer,
    images=images,
    patch_start_idx=1,         # Skip camera token at index 0

    frames_chunk_size=8
)

print(f"Depth map: {depth.shape}")        # (B, S, 1, H, W)

print(f"Confidence map: {confidence.shape}")  # (B, S, 1, H, W)

```

## Summary

- **camera_head.py** implements `CameraHead` for iterative 9-DoF camera pose estimation, supporting SLAM, robotics navigation, and AR/VR through adaptive layer normalization (`self.poseLN_modulation`) and optional 3-D RoPE.
- **dpt_head.py** implements `DPTHead` for dense depth prediction using multi-scale feature fusion (`_make_fusion_block`) and RefineNet blocks, enabling monocular depth estimation and confidence-aware sensing.
- Both heads are decoder-only and interchangeable, allowing the same Ling-Bot Map backbone to switch between pose-only and dense-prediction tasks.
- Key functions include `modulate()` and `activate_pose()` in CameraHead, and `activate_head()` in DPTHead, defined in [`lingbot_map/heads/head_act.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/head_act.py).

## Frequently Asked Questions

### What is the primary difference between camera_head.py and dpt_head.py?

**camera_head.py** produces sparse 9-dimensional pose encodings (translation, quaternion, field-of-view) suitable for camera tracking, while **dpt_head.py** generates dense per-pixel depth maps with optional confidence channels for scene reconstruction. CameraHead processes primarily the first token (camera token) with iterative refinement via `trunk_fn`, whereas DPTHead fuses multi-scale features from multiple transformer layers using `self.projects` and RefineNet blocks.

### Can DPTHead predict outputs other than depth maps?

Yes. By modifying the `output_dim` parameter and adjusting the `activation` strategy in [`lingbot_map/heads/dpt_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/dpt_head.py), the head can emit surface normals, semantic segmentation masks, or other dense predictions. The architecture supports arbitrary output dimensions through the final `output_conv2` layer, making it adaptable for multi-task learning scenarios beyond monocular depth estimation.

### How does CameraHead handle temporal information in video sequences?

CameraHead supports temporal encoding through optional **3-D Rotary Positional Embeddings (RoPE)** when `enable_3d_rope=True`. The `CameraCausalHead` variant leverages `WanRotaryPosEmbed` from [`lingbot_map/layers/rope.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/rope.py) to maintain temporal consistency across frames, which is critical for visual odometry applications where smooth trajectory estimation depends on understanding frame-to-frame relationships.

### Where are the activation functions defined for both heads?

Output activations are centralized in [`lingbot_map/heads/head_act.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/head_act.py). CameraHead uses `activate_pose()` (lines 140-144) to apply separate activations to translation (`trans_act`), quaternion (`quat_act`), and field-of-view (`fl_act`) components. DPTHead uses `activate_head()` (lines 445-447) to process the final dense output, supporting various depth parameterizations like `"inv_log"` for metric depth prediction.