# How Point Cloud Evaluation Computes Accuracy, Completeness, and Chamfer Distance

> Learn how point cloud evaluation computes accuracy, completeness, and Chamfer distance using scipy's cKDTree for efficient spatial indexing. Understand these key metrics for 3D data.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-02

---

**Point cloud evaluation in the lingbot-map repository calculates accuracy as the mean nearest-neighbor distance from predicted to ground-truth points, completeness as the reverse directional mean, and Chamfer distance as the average of both metrics, leveraging scipy's cKDTree for efficient spatial indexing.**

The lingbot-map benchmark suite provides robust metrics for comparing reconstructed point clouds against ground truth. Understanding how it calculates **accuracy**, **completeness**, and **Chamfer distance** reveals the underlying geometric similarity measures that drive 3D reconstruction evaluation according to the Robbyant/lingbot-map source code.

## The Three Core Metrics

The evaluation system in [`benchmark/benchmark/evaluation/points.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/evaluation/points.py) implements three distinct metrics that capture different aspects of geometric alignment between point clouds.

### What Accuracy Measures

**Accuracy** quantifies how close the predicted points are to the ground truth. The implementation extracts XYZ coordinates from the first three columns of the input arrays, builds a KD-tree from the target points, and queries the distance from each source point to its nearest neighbor in the target.

In `accuracy()` (lines 54-78), the function calls `distance(pred, gt)` to compute the mean of all source-to-target distances. A lower value indicates that the reconstructed points lie closer to the actual surface.

### What Completeness Measures

**Completeness** evaluates whether the prediction covers the entire ground-truth surface. Instead of measuring how far predictions are from truth, it measures how far ground-truth points are from the nearest predicted point.

The `completeness()` function (lines 80-102) achieves this by swapping the arguments—calling `distance(gt, pred)`—to compute the mean distance from target to source. High completeness indicates the reconstruction captures the full geometry without missing regions.

### Chamfer Distance as the Symmetric Metric

**Chamfer distance** provides a single scalar that treats both directions equally. The `chamfer_distance()` function (lines 105-110) computes the arithmetic mean of accuracy and completeness: `(accuracy + completeness) / 2.0`.

This symmetric metric penalizes both geometric inaccuracy (predicted points far from truth) and incomplete coverage (missing parts of the ground truth) in a single value.

## Implementation Details

The evaluation pipeline relies on efficient spatial data structures and directional distance calculations.

### Nearest-Neighbor Search with KD-Trees

The foundation of all metrics lies in the `distance()` function (lines 10-40). The implementation uses `scipy.spatial.cKDTree` to accelerate spatial queries:

1. Extract only the XYZ coordinates: `source_points[:, :3]` and `target_points[:, :3]`
2. Instantiate the tree: `tree = KDTree(target_xyz)`
3. Query with parallel processing: `tree.query(source_xyz, workers=-1)`

This returns the Euclidean distance from each source point to its closest neighbor in the target cloud.

### Directional Distance Calculation

The distinction between accuracy and completeness emerges solely from argument order:

- **Accuracy**: `distance(pred, gt)` returns `dist_s2t`, then `np.mean(dist_s2t)`
- **Completeness**: `distance(gt, pred)` returns `dist_t2s`, then `np.mean(dist_t2s)`

Both metrics use identical KD-tree logic but measure opposite directional flows.

## Using the Evaluation API

The repository provides multiple interfaces for computing these metrics, from low-level functions to high-level evaluators.

### Direct Metric Functions

For quick calculations on NumPy arrays, import the individual functions:

```python
import numpy as np
from benchmark.benchmark.evaluation.points import accuracy, completeness, chamfer_distance

# Dummy point clouds (Nx3)

gt   = np.random.rand(1000, 3)
pred = np.random.rand(1000, 3)

acc = accuracy(pred, gt)          # mean distance pred→gt

comp = completeness(pred, gt)     # mean distance gt→pred

cham = chamfer_distance(acc, comp)

print(f"Accuracy: {acc:.4f}")
print(f"Completeness: {comp:.4f}")
print(f"Chamfer: {cham:.4f}")

```

### High-Level Evaluation with evaluate_pointcloud

The `evaluate_pointcloud()` function (lines 250-390) handles reshaping, threshold-based precision/recall calculations, and returns a comprehensive metrics dictionary:

```python
import numpy as np
from benchmark.benchmark.evaluation.points import evaluate_pointcloud

gt   = np.random.rand(2000, 3)
pred = np.random.rand(2000, 3)

metrics = evaluate_pointcloud(pred, gt)   # thresholds default to [0.01,0.02,0.05,0.10]

print("Chamfer:", metrics["chamfer"])
print("Accuracy:", metrics["accuracy"])
print("Completeness:", metrics["completeness"])

# threshold-specific values available as metrics["precision_0.05"]

```

### The PointCloudEvaluator Class

For integration with the benchmark pipeline, the `PointCloudEvaluator` class (lines 425-470) wraps the evaluation logic:

```python
from benchmark.benchmark.evaluation.points import PointCloudEvaluator

evaluator = PointCloudEvaluator(my_dataset)
metrics, gt_pts, pred_pts, thresholds = evaluator.evaluate(gt_loader, pred_loader)

print(metrics)          # dict with chamfer, accuracy, completeness, etc.

```

This class forwards dataset-specific calls to `evaluate_pointcloud()` and returns colored point clouds for visualization alongside the scalar metrics.

## Summary

- **Accuracy** computes the mean nearest-neighbor distance from predicted points to ground truth, measuring geometric precision.
- **Completeness** reverses the direction, computing the mean distance from ground truth to predictions, measuring surface coverage.
- **Chamfer distance** averages accuracy and completeness to provide a symmetric metric that penalizes both deviation and missing regions.
- The implementation in [`benchmark/benchmark/evaluation/points.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/evaluation/points.py) uses `scipy.spatial.cKDTree` with `workers=-1` for parallel nearest-neighbor queries.
- Three API levels exist: direct functions (`accuracy()`, `completeness()`), the convenience wrapper `evaluate_pointcloud()`, and the pipeline-ready `PointCloudEvaluator` class.

## Frequently Asked Questions

### What is the difference between accuracy and completeness in point cloud evaluation?

**Accuracy** measures how well the predicted points align with the ground-truth surface by calculating the mean distance from each predicted point to its nearest neighbor in the ground truth. **Completeness** measures coverage by calculating the mean distance from each ground-truth point to its nearest neighbor in the prediction. High accuracy means predicted points are close to the true surface, while high completeness means the reconstruction captures the full extent of the ground truth without gaps.

### How does the Chamfer distance calculation differ from accuracy and completeness?

While accuracy and completeness are **directional** metrics (one-way distances), Chamfer distance is **symmetric**. The `chamfer_distance()` function computes the arithmetic mean of the accuracy and completeness values: `(accuracy + completeness) / 2.0`. This produces a single scalar that equally penalizes both geometric deviation (inaccuracy) and missing surface regions (incompleteness), making it suitable for overall reconstruction quality assessment.

### Why does the implementation use scipy's cKDTree for distance calculations?

The `distance()` function utilizes `scipy.spatial.cKDTree` because KD-trees provide **O(log n)** query complexity for nearest-neighbor searches in 3D space, dramatically outperforming brute-force approaches. The implementation further optimizes performance by setting `workers=-1` during the `tree.query()` call, enabling parallel processing across all available CPU cores. This allows efficient evaluation of point clouds containing thousands of points.

### How can I evaluate my own point clouds using the lingbot-map benchmark?

Import the evaluation functions from `benchmark.benchmark.evaluation.points` and ensure your point clouds are NumPy arrays with shape (N, 3) containing XYZ coordinates. For simple metrics, use `accuracy(pred, gt)` and `completeness(pred, gt)`. For comprehensive evaluation including threshold-based precision and recall, use `evaluate_pointcloud(pred, gt, thresholds=[0.01, 0.05])`. If integrating with the full benchmark pipeline, instantiate `PointCloudEvaluator` with a dataset object that implements the required loader interface.