How to Use the `--force` Flag to Re-run Completed Scenes in LingBot-Map

Append --force (or -f) to any benchmark command to delete previous results and re-process scenes that would otherwise be skipped as complete.

The LingBot-Map benchmark suite applies a "skip-if-finished" policy by default, automatically bypassing scenes that have already been processed. When you need to overwrite existing results—whether due to code changes, parameter adjustments, or corrupted outputs—the --force flag provides a deterministic way to clear artifacts and start fresh. This guide explains the flag's implementation across the Robbyant/lingbot-map repository and provides practical usage examples.

Where the --force Flag Is Implemented

Core Worker Logic

In benchmark/run_worker.py, the argument parser defines the flag:

parser.add_argument('--force', '-f', action='store_true',
                    help='Force re-run even if scene is already complete')

Completion Check Bypass

The worker uses the flag to conditionally clear directories. In the scene processing logic of benchmark/run_worker.py:

if args.force:
    pred_artifact.clear_directory()          # delete previous results

    scenes_to_run.append((scene, gt_artifact, pred_artifact))
elif not pred_artifact.is_complete():
    scenes_to_run.append((scene, gt_artifact, pred_artifact))

The is_complete() method in benchmark/core/storage.py checks a completion flag in the artifact's metadata. When --force is active, the script invokes clear_directory() to remove prior predictions before re-adding the scene to the execution queue.

Supported Commands and Syntax

The flag propagates through the benchmark pipeline. Append it to any of these commands:

• benchmark/run.py – Re-runs inference on target scenes
• benchmark/prepare.py – Re-creates ground-truth BSS artifacts
• benchmark/evaluate.py – Re-computes evaluation metrics

Practical Usage Examples

Re-run a Full Dataset

python benchmark/run.py \
    --config configs/base.yaml \
    --method cut3r \
    --dataset 7scenes_s10 \
    --force

Re-run a Specific Scene

python benchmark/run.py \
    --config configs/base.yaml \
    --method cut3r \
    --dataset 7scenes_s10 \
    --scene chess/seq-01 \
    --force

Re-prepare Ground-Truth Data

python benchmark/prepare.py \
    --config configs/base.yaml \
    --dataset 7scenes_s10 \
    --force

Force Re-evaluation

python benchmark/evaluate.py \
    --config configs/base.yaml \
    --method cut3r \
    --dataset 7scenes_s10 \
    --force

When to Use the --force Flag

Use this flag when:

• Algorithmic changes require fresh predictions to ensure the new implementation is evaluated correctly
• Parameter sweeps demand clean result directories for fair hyper-parameter comparisons
• Corrupted outputs exist from interrupted runs (e.g., OOM errors), leaving partial files that is_complete() might misinterpret

Summary

• The --force flag is parsed in benchmark/run_worker.py and available in run.py, prepare.py, and evaluate.py
• It triggers pred_artifact.clear_directory() to delete previous results before processing
• Without the flag, scenes where BSSArtifact.is_complete() returns True are automatically skipped
• Both --force and -f syntax are accepted across all benchmark scripts

Frequently Asked Questions

What happens if I don't use --force on an already completed scene?

The benchmark suite checks pred_artifact.is_complete() in benchmark/core/storage.py. If the scene is marked complete, the worker skips processing to save computation time, leaving existing files unchanged.

Does --force affect ground-truth preparation or just predictions?

The flag works for both. When passed to benchmark/prepare.py, it forces re-creation of ground-truth BSS artifacts. When passed to benchmark/run.py or benchmark/evaluate.py, it clears prediction or evaluation results respectively.

Can I use --force with specific scenes only?

Yes. Combine --force with the --scene argument in benchmark/run.py to target individual sequences while leaving other completed scenes intact.

Is there a risk of data loss when using --force?

Yes. The flag calls clear_directory() on the artifact's output path, permanently deleting previous prediction files, metrics, or cached ground-truth data. Ensure you have backups of critical results before forcing a re-run.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}