# Why rdp_mask Returns a Boolean Mask Instead of Simplified Points

> Understand why rdp_mask returns a boolean mask not points. Discover how this preserves indices and enables identical filtering for parallel datasets in cubao/pybind11-rdp.

- Repository: [cubao/pybind11-rdp](https://github.com/cubao/pybind11-rdp)
- Tags: faq
- Published: 2026-02-28

---

**The `rdp_mask` function is deliberately designed to return a 1-D integer mask (Eigen::VectorXi) rather than coordinate points, allowing you to preserve original array indices for downstream processing or apply identical filtering to multiple parallel datasets.**

The `pybind11-rdp` library implements the Ramer-Douglas-Peucker (RDP) line simplification algorithm in C++ with Python bindings via pybind11. While the high-level `rdp()` function returns simplified coordinates, the companion `rdp_mask()` function exposes the underlying selection logic as a boolean mask—a design choice optimized for workflows requiring index preservation or batch processing of aligned data arrays.

## The Design Philosophy Behind rdp_mask

When simplifying polylines, retaining the **boolean mask** (integer flags) offers distinct advantages over receiving only the filtered coordinates. The mask preserves the original indexing, enabling you to:

- Synchronize simplification across multiple parallel data arrays (e.g., coordinates, timestamps, and sensor readings)
- Track which vertices were removed for debugging or analytic purposes
- Apply the same RDP reduction to auxiliary data without re-running the geometric algorithm

According to the `pybind11-rdp` source code, `rdp_mask` returns an `Eigen::VectorXi` where `1` indicates a point to keep and `0` indicates a point to discard. This implementation lives in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp) within the `douglas_simplify_mask` function (lines 28–40).

## C++ Implementation: Generating the Mask

The core mask generation logic resides in the C++ backend. The `douglas_simplify_mask` function initializes a zero-filled vector, then populates it via either recursive or iterative RDP algorithms depending on the `recursive` parameter.

```cpp
// src/main.cpp (lines 28-40)
Eigen::VectorXi
douglas_simplify_mask(const Eigen::Ref<const RowVectors> &coords,
                      double epsilon, bool recursive) {
    Eigen::VectorXi mask(coords.rows());
    mask.setZero();                     // start with all zeros
    if (recursive) {
        douglas_simplify(coords, mask, 0, mask.size() - 1, epsilon);
    } else {
        douglas_simplify_iter(coords, mask, epsilon);
    }
    return mask;                        // ← mask of points to keep
}

```

The Python binding exposes this directly as `rdp_mask` (lines 35–46 in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp)):

```cpp
// Python binding in src/main.cpp
m.def(
    "rdp_mask",
    [](const Eigen::Ref<const RowVectors> &coords, double epsilon,
       bool recursive) -> Eigen::VectorXi {
        return douglas_simplify_mask(coords, epsilon, recursive);
    },
    rdp_mask_doc, "coords"_a,
    py::kw_only(), "epsilon"_a = 0.0, "recursive"_a = true);

```

The high-level Python API in [`src/pybind11_rdp/__init__.py`](https://github.com/cubao/pybind11-rdp/blob/main/src/pybind11_rdp/__init__.py) simply re-exports this binding:

```python

# src/pybind11_rdp/__init__.py

from ._core import rdp_mask  # noqa

```

## Converting Masks to Simplified Points

If you need the actual coordinate array rather than the mask, you have two options: manual indexing in Python or using the `rdp()` function with `return_mask=True`.

### Manual Mask Application

Apply the mask to your original NumPy array using boolean indexing:

```python
import numpy as np
from pybind11_rdp import rdp_mask

points = np.array([[0.0, 0.0], [1.0, 0.1], [2.0, -0.1], [3.0, 0.0]])
mask = rdp_mask(points, epsilon=0.2)
simplified = points[mask.astype(bool)]  # Manual selection

```

### Using the rdp() Function

The `rdp()` function (and its aliases `rdp_iter`, `rdp_rec`) internally calls `select_by_mask` (lines 60–71 in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp)) to convert the mask to points automatically:

```cpp
// src/main.cpp – converting mask to points
RowVectors select_by_mask(const Eigen::Ref<const RowVectors> &coords,
                          const Eigen::Ref<const Eigen::VectorXi> &mask) {
    RowVectors ret(mask.sum(), coords.cols());
    for (int i = 0, k = 0; i < mask.size(); ++i) {
        if (mask[i]) {
            ret.row(k++) = coords.row(i);
        }
    }
    return ret;          // ← simplified points
}

```

You can also request the mask directly from `rdp()` by setting `return_mask=True`:

```python
from pybind11_rdp import rdp

# Returns mask instead of points

mask = rdp(points, epsilon=0.2, algo="iter", return_mask=True)

```

## Practical Usage Examples

The following examples demonstrate both mask retrieval and point simplification workflows:

```python
import numpy as np
from pybind11_rdp import rdp, rdp_mask

# Example polyline

points = np.array([
    [0.0, 0.0],
    [1.0, 0.1],
    [2.0, -0.1],
    [3.0, 0.0],
    [4.0, 0.2]
])

# 1️⃣ Get boolean mask directly from rdp_mask

mask = rdp_mask(points, epsilon=0.15, recursive=False)
print("Mask:", mask)  # array([1, 0, 0, 1, 1])

# 2️⃣ Synchronize filtering across multiple arrays

timestamps = np.array([0, 1, 2, 3, 4])
coords_simplified = points[mask.astype(bool)]
times_simplified = timestamps[mask.astype(bool)]

# 3️⃣ Get simplified points via high-level API

simplified = rdp(points, epsilon=0.15, algo="iter")
print("Simplified:\n", simplified)

# 4️⃣ Request mask from rdp function

mask_via_rdp = rdp(points, epsilon=0.15, return_mask=True)

```

## Summary

- **Mask preservation**: `rdp_mask` returns an `Eigen::VectorXi` (exposed as a NumPy array) to maintain original indices and enable parallel array filtering.
- **C++ core**: The mask is generated by `douglas_simplify_mask` in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp) (lines 28–40), supporting both recursive (`douglas_simplify`) and iterative (`douglas_simplify_iter`) algorithms.
- **Point conversion**: Use `select_by_mask` (lines 60–71) internally via the `rdp()` function, or apply boolean indexing manually in Python.
- **API flexibility**: Set `return_mask=True` in `rdp()` to access mask generation without calling `rdp_mask` directly, or use `rdp_mask` with `recursive=False` for memory-efficient iterative processing.

## Frequently Asked Questions

### How do I convert the rdp_mask output to actual coordinates?

Apply boolean indexing to your original array using the mask: `simplified = points[mask.astype(bool)]`. Alternatively, use `rdp(points, epsilon=value)` which returns the coordinates directly by internally calling `select_by_mask` as implemented in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp).

### What is the difference between rdp_mask and the return_mask parameter in rdp()?

`rdp_mask()` always returns the mask and exposes the underlying C++ `douglas_simplify_mask` directly with a `recursive` boolean parameter. The `rdp()` function with `return_mask=True` uses the same core logic but provides a unified API accepting an `algo` string parameter (`"iter"` or `"rec"`) where you can toggle between point output and mask output without changing functions.

### Can I use rdp_mask with the iterative algorithm instead of recursive?

Yes. By default, `rdp_mask` uses the recursive algorithm (`recursive=True`). Set `recursive=False` to use the iterative implementation (`douglas_simplify_iter`), which is more memory-efficient for large datasets: `rdp_mask(points, epsilon=0.5, recursive=False)`.

### Why does the mask return integers (0/1) instead of true booleans?

The mask is implemented as `Eigen::VectorXi` (integer vector) in the C++ core for performance and compatibility with Eigen operations. When exposed to Python via pybind11, it becomes a NumPy array of integers. These behave identically to booleans in indexing contexts, or you can convert explicitly with `.astype(bool)`.