How to Use the `return_mask` Parameter in pybind11-rdp's `rdp` Function

Set return_mask=True in the rdp function to receive a boolean array indicating which points survive the Ramer-Douglas-Peucker line simplification, enabling you to filter parallel data arrays or track original indices.

The rdp function in the cubao/pybind11-rdp repository exposes a high-performance C++ implementation of the Ramer-Douglas-Peucker (RDP) algorithm through pybind11. While the default behavior returns the simplified coordinate array, the return_mask parameter provides a powerful alternative for workflows requiring knowledge of which specific input points were retained.

Understanding the return_mask Parameter

The return_mask boolean argument changes the function's output type without altering the underlying simplification logic:

• return_mask=False (default): Returns an np.ndarray containing only the simplified coordinate points. Use this when you need the reduced polyline directly.

• return_mask=True: Returns a 1-D np.ndarray of dtype bool or int32 (depending on the internal implementation) where True (or 1) marks indices to keep. This mask aligns with the original input array's order, allowing you to filter accompanying data like timestamps, colors, or elevation values.

According to the source code in src/pybind11_rdp/__init__.py, this parameter determines whether the function calls the C++ rdp_mask helper or the standard _rdp implementation.

Implementation Details in the Source Code

In src/pybind11_rdp/__init__.py (lines 45–64), the Python façade handles the parameter routing:

def rdp(
    points,
    epsilon: float = 0.0,
    dist=None,
    algo="iter",
    return_mask=False,
):
    """
    Main interface for the Ramer-Douglas-Peucker algorithm.
    …
    """
    __notify_dist_fn(dist)
    points = np.asarray(points, dtype=np.float64)
    recursive = "iter" != algo
    if return_mask:
        return rdp_mask(points, epsilon=epsilon, recursive=recursive)
    return _rdp(points, epsilon=epsilon, recursive=recursive)

When return_mask=True, the function invokes rdp_mask (exposed from the _core module) instead of _rdp. The type stub in src/pybind11_rdp/__init__.pyi (lines 106–119) defines this helper's signature as:

def rdp_mask(
    coords: NDArray[np.float64], *, epsilon: float = 0.0, recursive: bool = True
) -> NDArray[np.int32]:
    ...

The rdp_mask function respects the algo parameter through the internal recursive flag. When algo="iter", it uses the iterative implementation; when algo="rec", it uses the recursive variant. Both return identical masks for the same input data.

Practical Code Examples

Retrieve Simplified Points (Default Behavior)

By default, rdp returns the filtered coordinate array. This is ideal for direct geometric processing:

import numpy as np
from pybind11_rdp import rdp

pts = np.array([[0, 0], [1, 0.1], [2, 0], [3, 5], [4, 0]])
simplified = rdp(pts, epsilon=0.5)
print(simplified)

# Output: [[0, 0], [2, 0], [3, 5], [4, 0]]

Obtain a Boolean Mask for Selective Indexing

Use return_mask=True when you need to filter parallel arrays or retain original indices:

import numpy as np
from pybind11_rdp import rdp

pts = np.array([[0, 0], [1, 0.1], [2, 0], [3, 5], [4, 0]])
mask = rdp(pts, epsilon=0.5, return_mask=True)

print(mask)

# Output: [ True False  True  True False]

# Filter accompanying metadata using the same mask

colors = np.array(['red', 'green', 'blue', 'orange', 'purple'])
kept_colors = colors[mask]
print(kept_colors)

# Output: ['red' 'blue' 'orange']

Combine Masks with Different Algorithms

Both iterative (algo="iter") and recursive (algo="rec") implementations support mask generation:

import numpy as np
from pybind11_rdp import rdp

pts = np.random.rand(1000, 2)

# Generate masks using both algorithms

mask_iter = rdp(pts, epsilon=0.01, algo="iter", return_mask=True)
mask_rec = rdp(pts, epsilon=0.01, algo="rec", return_mask=True)

# Verify identical results

assert np.array_equal(mask_iter, mask_rec)

Handle Extreme Epsilon Values

The mask behavior remains consistent at boundary conditions:

import numpy as np
from pybind11_rdp import rdp

pts = np.cumsum(np.random.rand(20, 2), axis=0)

# Epsilon=0: no simplification, all points kept

mask_zero = rdp(pts, epsilon=0.0, return_mask=True)
print(np.all(mask_zero))  # True

# Infinite epsilon: only endpoints retained

mask_inf = rdp(pts, epsilon=np.inf, return_mask=True)
print(np.count_nonzero(mask_inf))  # 2 (first and last points only)

Note that the dist parameter is currently ignored by the implementation; passing a custom distance function triggers a warning but does not affect the mask output.

Summary

• The return_mask parameter in pybind11_rdp's rdp function switches output from simplified coordinates to a boolean index array.
• When enabled, the function dispatches to the C++ helper rdp_mask defined in the pybind11 bindings.
• The returned mask maintains the original input order, allowing synchronized filtering of auxiliary data arrays.
• Both iterative (algo="iter") and recursive (algo="rec") algorithms support mask generation with identical results.
• Edge cases like epsilon=0.0 (full retention) and epsilon=np.inf (endpoint-only retention) behave predictably when using masks.

Frequently Asked Questions

What data type does the mask return?

The function returns a 1-D NumPy array of type int32 (containing 0s and 1s) or bool, depending on the C++ implementation details. Both types work for boolean indexing in NumPy, so you can use array[mask] directly without casting.

Can I use return_mask with custom distance functions?

Currently, no. The dist parameter is accepted in the function signature for API compatibility but triggers a warning and is ignored. The mask generation always uses the standard Euclidean distance metric as implemented in the C++ core.

Does the mask preserve the original input order?

Yes. The mask array aligns index-for-index with the input points array. A True value at index i indicates that points[i] survived the simplification and should be retained. This positional alignment enables filtering of parallel time-series or attribute arrays.

How does return_mask affect performance?

Generating a mask requires nearly identical computational cost to generating the simplified point array, as both operations execute the same RDP algorithm. The mask path avoids the final coordinate extraction step, making it marginally faster when you only need the boolean indices rather than the actual coordinates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}