How to Run a Single Scene in Lingbot-Map: Command-Line Guide

Invoke benchmark/run_worker.py with the --scene flag to evaluate a method on one specific scene instead of processing an entire dataset.

Lingbot-Map’s benchmark pipeline processes datasets that consist of many scenes, but you often need to test algorithms on a single scene for debugging or rapid iteration. According to the Robbyant/lingbot-map source code, you can bypass the full dataset orchestration by calling the lower-level worker script directly with scene-specific parameters.

Method 1: Direct Worker Invocation (Recommended)

The benchmark/run_worker.py script is the actual worker that runs a method on a single scene. It accepts a --scene argument (defined on lines 94‑96) that overrides the default behavior of processing all scenes. When provided, the script builds a single-item scene list on lines 14‑19 and executes only that scene.

Use this command template, replacing placeholders with your actual configuration values:

# <method>  = key from configs/base.yaml under methods

# <dataset> = key from configs/base.yaml under datasets

# <scene>   = scene identifier, e.g., "office/seq-01"

conda run -n <env> --no-capture-output \
    python benchmark/run_worker.py \
    --config configs/base.yaml \
    --method <method> \
    --dataset <dataset> \
    --scene <scene>

The --env value comes from the method’s env field in base.yaml. If the method has no environment constraint, omit the conda run wrapper and invoke the script directly. The worker handles loading the method class, reading ground-truth artifacts via benchmark/core/loader.py, calling method.process_scene(), and persisting outputs via benchmark/core/saver.py.

Method 2: Debug Mode for First Scene Only

If you want a quick sanity check without specifying a scene name, use the top-level driver benchmark/run.py with the --debug flag. This forces the driver to resolve only the first scene of each dataset (see the resolution logic on lines 96‑108) and run the method on that single scene.

python benchmark/run.py \
    --config configs/base.yaml \
    --debug

Debug mode is useful when you need to verify that your method loads correctly and produces output, but you do not need to target a specific scene identifier.

How Scene Selection Works Internally

Understanding the architecture helps you decide which entry point to use.

In-Process vs. Subprocess Paths

The main orchestrator benchmark/run.py builds the scene list (lines 31‑40) and selects between two execution strategies:

• In-process: Calls run_scene() directly within the Python process.
• Subprocess: Calls run_via_subprocess() (lines 91‑107), which constructs a shell command that includes --scene <scene> when a specific scene is targeted.

When you invoke run_worker.py directly, you are essentially entering the subprocess path manually, skipping the orchestration overhead.

Scene List Construction

In benchmark/run_worker.py, the logic branches on line 14:

• If --scene is None (default), it loads all scenes for the dataset.
• If --scene is provided, it creates a list containing only that scene identifier (lines 14‑19).

This single-scene list is then passed to the processing loop, ensuring only your specified scene is evaluated.

Supporting Files and Architecture

File	Role
benchmark/run.py	Main orchestrator; decides between in-process execution or spawning run_worker.py as a subprocess.
benchmark/run_worker.py	Worker script that executes a method on a single scene; implements the --scene CLI argument.
benchmark/core/loader.py	Loads ground-truth data for a specific scene; used by both the driver and worker.
benchmark/core/saver.py	Persists method outputs per-scene.
benchmark/core/registry.py	Dynamically loads dataset and method classes from the YAML configuration.

Summary

• To run a single specific scene: Use python benchmark/run_worker.py --scene <scene> with your method and dataset parameters.
• To run only the first scene: Use python benchmark/run.py --debug for rapid smoke tests.
• The --scene flag in run_worker.py (lines 94‑96) overrides the default all-scenes behavior by constructing a single-item processing list (lines 14‑19).
• The benchmark/run.py driver can spawn workers with scene-specific arguments via run_via_subprocess() (lines 91‑107), but direct invocation of the worker script is faster for targeted evaluation.

Frequently Asked Questions

Can I use run.py to target a specific scene by name?

The top-level run.py does not expose a --scene argument directly; it only supports --debug for the first scene. To target a specific scene by identifier, invoke benchmark/run_worker.py directly with the --scene flag as shown in Method 1. Alternatively, you could write a thin wrapper that calls run_via_subprocess() with a specific scene name, mimicking the internal behavior on lines 101‑107 of run.py.

What format should the scene name be in?

The scene identifier must match the keys used in your dataset configuration (typically configs/base.yaml). Common formats include directory-style paths like "office/seq-01" or simple identifiers like "scene_001". The benchmark/core/loader.py expects the scene string to resolve to a valid data path defined in the dataset class.

Does debug mode use the same code path as normal execution?

Yes. When you pass --debug to run.py, it still executes run_scene() or spawns run_worker.py, but it restricts the scene list to the first item per dataset (lines 96‑108). The actual method execution—loading, processing, and saving—follows the identical code path as a full run, ensuring your debug results are representative.

How do I handle conda environments when running single scenes?

Check the env field for your method in configs/base.yaml. If an environment is specified, prepend your command with conda run -n <env> --no-capture-output as shown in the Method 1 example. If the field is empty or omitted, you can call python benchmark/run_worker.py directly without the conda wrapper, assuming your current shell environment has the required dependencies installed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}