How rerun.io Visualization Enables Real-Time 3D Rendering in Peng

The rerun.io visualization enables real-time 3D rendering in Peng by spawning a background RecordingStream that streams incremental scene updates—transforms, geometry, and depth images—directly to the rerun viewer, which renders them instantly while synchronizing to the simulation clock.

The open-source quadrotor simulation framework Peng (makeecat/peng) leverages rerun.io to provide live, interactive 3D telemetry without blocking the physics loop. By conditionally initializing a RecordingStream based on a configuration flag, Peng streams high-frequency spatial data directly to the rerun viewer, enabling developers to visualize complex drone maneuvers, maze navigation, and camera feeds in real time.

Architecture of the rerun.io Integration in Peng

Peng’s integration follows a three-stage pipeline: conditional initialization, per-frame logging of simulation primitives, and timestamp-synchronized rendering. This architecture ensures that the visualization overhead remains minimal while providing sub-second latency for scene updates.

Conditional Stream Initialization

In src/main.rs, the simulation checks the use_rerun boolean defined in src/config.rs before spawning the visualization backend. When enabled, the code constructs a RecordingStreamBuilder, names the application "Peng", and spawns the background server. A Logger instance is then attached with the path prefix "logs" and the default filter to capture structured telemetry.

// src/main.rs – Conditional rerun initialization
let rec = if config.use_rerun {
    let _rec = rerun::RecordingStreamBuilder::new("Peng").spawn()?;
    rerun::Logger::new(_rec.clone())
        .with_path_prefix("logs")
        .with_filter(rerun::default_log_filter())
        .init()?;
    Some(_rec)
} else {
    None
};

The resulting Option<RecordingStream> is passed into the simulation loop, allowing the rest of the codebase to remain agnostic of whether visualization is active.

Per-Frame Data Logging

During each simulation step, Peng invokes helper functions from src/lib.rs to serialize quadrotor state, environment geometry, and sensor data into rerun primitives. The system logs Transforms for the drone pose, Points3D for desired positions, Scalars for telemetry values, Boxes3D for maze tubes, and Images for depth camera outputs. Each call targets a specific entity path, such as "world/quad/base_link" or "world/maze/obstacles", creating a hierarchical scene graph that the viewer renders instantly.

// src/lib.rs – Logging a point cloud with color and radius
rec.log(
    "world/quad/desired_position",
    &rerun::Points3D::new([(desired.x, desired.y, desired.z)])
        .with_radii([0.1])
        .with_colors([rerun::Color::from_rgb(255, 255, 255)]),
)?;

For depth perception, Peng computes pinhole camera models and depth buffers, then logs them using log_depth_image and log_pinhole_depth, enabling the viewer to display textured 3D point clouds derived from the simulated camera.

Real-Time Synchronization

To ensure the viewer’s timeline matches the simulation clock, Peng calls rec.set_duration_secs("timestamp", time) at the start of every frame. This method stamps each logged entity with the current simulation time, allowing rerun to interpolate and animate the scene smoothly. The viewer consumes these timestamped protobuf messages over a local TCP socket, updating the 3D viewport immediately without waiting for the entire frame buffer.

Key Source Files and Functions

The following files implement the rerun.io pipeline in Peng:

• src/config.rs: Defines the use_rerun flag that toggles visualization.
• src/main.rs: Contains the RecordingStreamBuilder setup and the main loop that calls logging functions conditionally.
• src/lib.rs: Houses the logging primitives, including log_data, log_trajectory, log_maze_obstacles, log_maze_tube, and log_depth_image.
• Cargo.toml: Declares the rerun crate dependency, enabling the RecordingStream API.

Specific functions of note:

• log_data: Serializes quadrotor pose, velocity, acceleration, and Euler angles.
• log_maze_obstacles: Streams obstacle positions as Points3D with radii.
• log_depth_image: Renders and transmits depth buffers as Image entities.

Practical Code Example: Minimal rerun Setup

The following standalone example demonstrates the same pattern used in Peng: initializing a stream, logging a static cube, and animating a moving point with timestamp synchronization.

use rerun::{RecordingStreamBuilder, Logger, Points3D, Boxes3D, Color};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize the recording stream
    let rec = RecordingStreamBuilder::new("PengDemo").spawn()?;
    Logger::new(rec.clone())
        .with_path_prefix("logs")
        .with_filter(rerun::default_log_filter())
        .init()?;

    // Log a static maze cube
    let center = rerun::external::glam::Vec3::new(0.0, 0.0, 0.0);
    let half = rerun::external::glam::Vec3::new(0.5, 0.5, 0.5);
    rec.log(
        "world/cube",
        &Boxes3D::from_centers_and_half_sizes([center], [half])
            .with_colors([Color::from_rgb(100, 150, 250)]),
    )?;

    // Animate a point orbiting the cube
    for i in 0..100 {
        let t = i as f32 * 0.1;
        rec.set_duration_secs("timestamp", t);
        rec.log(
            "world/point",
            &Points3D::new([(t.sin(), t.cos(), 0.0)])
                .with_radii([0.05])
                .with_colors([Color::from_rgb(255, 0, 0)]),
        )?;
        std::thread::sleep(std::time::Duration::from_millis(100));
    }
    Ok(())
}

Running this snippet and launching rerun displays a blue cube with a red point orbiting it in real time, mirroring the behavior seen in Peng’s quadrotor visualization.

Summary

• Conditional initialization via use_rerun in src/config.rs ensures zero overhead when visualization is disabled.
• RecordingStreamBuilder::new("Peng").spawn() in src/main.rs starts a background server that accepts streaming data without blocking the physics thread.
• Per-frame logging of Transforms, Points3D, Boxes3D, and Images via helpers in src/lib.rs builds a hierarchical scene graph incrementally.
• rec.set_duration_secs("timestamp", time) synchronizes the viewer timeline with the simulation clock, enabling smooth real-time playback.
• The rerun viewer renders protobuf streams instantly, providing interactive 3D inspection of drone trajectories, maze geometries, and depth camera feeds.

Frequently Asked Questions

How does Peng minimize performance impact when rerun visualization is enabled?

Peng encapsulates the RecordingStream in an Option and only initializes the Logger and stream when the use_rerun flag is true. By sending incremental updates rather than full frame buffers and using rerun’s compact binary encoding, the simulation loop retains its real-time constraints while the viewer consumes data asynchronously over a local socket.

What entity paths does Peng use to organize the 3D scene in rerun?

Peng follows a hierarchical naming convention: "world/quad/base_link" for the drone transform, "world/quad/desired_position" for target waypoints, "world/maze/obstacles" for collision spheres, and "world/camera/depth" for sensor images. This structure allows the rerun viewer to toggle visibility of individual components and maintain clean scene organization.

Can the rerun visualization handle high-frequency depth camera data in real time?

Yes. The log_depth_image function in src/lib.rs transmits depth buffers as Image entities, while log_pinhole_depth establishes the camera intrinsics. Rerun’s native Image primitive is optimized for GPU upload, enabling Peng to stream high-resolution depth maps at simulation frame rates without dropping telemetry.

How do I disable rerun visualization when running Peng headlessly?

Set use_rerun: false in your configuration file or runtime arguments. When this flag is false, src/main.rs skips the RecordingStreamBuilder initialization and the simulation runs without spawning the rerun server, ensuring no network sockets or rendering overhead are created.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}