# Handling Infinite Epsilon in RDP: Edge Case Behavior in pybind11-rdp > Discover how pybind11-rdp handles infinite epsilon in RDP algorithms. Learn about edge case behavior and achieve maximum simplification without errors. - Repository: [cubao/pybind11-rdp](https://github.com/cubao/pybind11-rdp) - Tags: deep-dive - Published: 2026-02-28 --- **Passing `float('inf')` or `np.inf` as the epsilon parameter to the RDP algorithm causes the library to return only the start and end points of the polyline, effectively applying maximum simplification without triggering infinite recursion or crashes.** The cubao/pybind11-rdp repository provides a high-performance Python wrapper for the Ramer-Douglas-Peucker (RDP) line simplification algorithm, implemented in C++ using pybind11. Understanding how the library handles extreme epsilon values, particularly infinite epsilon behavior in RDP, is essential for preventing unexpected recursion errors and achieving predictable polyline simplification results. ## How Epsilon Controls the RDP Algorithm The core implementation resides in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp), which defines two algorithm variants: `douglas_simplify` (recursive) and `douglas_simplify_iter` (iterative). Both implementations rely on the same geometric stopping condition to determine when to cease splitting a line segment. The algorithm calculates the squared perpendicular distance (`max_dist2`) from each intermediate point to the current line segment. The critical control flow occurs at this comparison: ```cpp if (max_dist2 <= epsilon * epsilon) { // Stop splitting - all points are within tolerance } ``` When the maximum squared distance falls within the squared epsilon threshold, the algorithm halts processing for that segment, preserving the endpoints and discarding intermediate points that deviate less than the tolerance. ## Infinite Epsilon Behavior in RDP When `epsilon` is set to infinity (`std::numeric_limits::infinity()` in C++, or `float('inf')`/`np.inf` in Python), the expression `epsilon * epsilon` evaluates to infinity. Since any finite squared distance (`max_dist2`) satisfies `max_dist2 ≤ ∞`, the stopping condition triggers immediately on the initial segment encompassing the entire polyline. ### Impact on the Recursive Implementation In `douglas_simplify`, the function evaluates the stopping condition before making any recursive calls. When epsilon is infinite, the condition is satisfied immediately, causing the function to return without recursing. This prevents stack overflow despite the extreme parameter value, safely returning a mask that marks only the first and last points as retained. ### Impact on the Iterative Implementation The `douglas_simplify_iter` function uses a queue to manage segments pending processing. With infinite epsilon, the initial segment is popped from the queue, the stopping condition is satisfied, and no new sub-segments are pushed back onto the queue. The queue becomes empty after the first iteration, naturally terminating the loop without infinite cycling. ## Other Extreme Epsilon Values Understanding the full spectrum of edge cases helps prevent misuse and unexpected behavior: - **Epsilon = 0**: Forces the algorithm to retain all points unless they are exactly collinear (distance exactly zero). This produces the most faithful representation of the original polyline, effectively disabling simplification. - **Very large finite epsilon**: Behaves similarly to infinite epsilon, causing aggressive simplification that typically retains only the endpoints. The squared value may risk floating-point overflow with extremely large numbers, though modern systems handle this robustly. - **NaN (Not a Number)**: Any comparison with `NaN` returns false, preventing the stopping condition from ever triggering. This causes infinite recursion in the recursive implementation or infinite looping in the iterative version, likely resulting in a crash or stack overflow. The library does not validate against `NaN` inputs. ## Practical Implementation Examples The Python wrapper in [`src/pybind11_rdp/__init__.py`](https://github.com/cubao/pybind11-rdp/blob/main/src/pybind11_rdp/__init__.py) forwards arguments directly to the C++ core, allowing seamless use of NumPy's infinity constants: ```python import numpy as np from pybind11_rdp import rdp, rdp_mask # Sample polyline with minor deviations points = np.array([[0, 0], [2, 0.1], [4, 0], [6, 0.2], [8, 0]]) # Normal simplification with epsilon=0.15 simplified = rdp(points, epsilon=0.15) # Maximum simplification using infinite epsilon endpoints_only = rdp(points, epsilon=np.inf) # Returning a boolean mask for downstream indexing mask = rdp_mask(points, epsilon=np.inf, algo="iter") # Result: [True, False, False, False, True] ``` When using `rdp_mask` with infinite epsilon, the function returns a boolean array where only the first and last indices are `True`, enabling efficient filtering of corresponding data arrays without copying point coordinates. ## Summary - **Infinite epsilon behavior in RDP** triggers immediate stopping conditions in both recursive and iterative implementations, returning only the start and end points of the polyline. - The comparison `max_dist2 <= epsilon * epsilon` evaluates to true for all finite distances when epsilon is infinity, preventing any recursive calls or queue operations. - **Zero epsilon** produces the most accurate simplification, retaining all non-collinear points. - **NaN values** are dangerous and cause infinite recursion or hanging; the library does not validate against them. - Both `douglas_simplify` and `douglas_simplify_iter` in [`src/main.cpp`](https://github.com/cubao/pybind11-rdp/blob/main/src/main.cpp) handle infinite epsilon safely without stack overflow risks. ## Frequently Asked Questions ### What happens if I pass infinity as the epsilon value to pybind11-rdp? Passing `float('inf')` or `np.inf` causes the algorithm to return only the first and last points of your polyline. Because any finite distance is less than infinity squared, the stopping condition triggers immediately on the initial segment, preventing any recursive splitting or iterative processing of sub-segments. ### Is it safe to use infinite epsilon with the recursive algorithm implementation? Yes, it is completely safe. The recursive function `douglas_simplify` evaluates the stopping condition before making any recursive calls. With infinite epsilon, this condition is satisfied immediately, so the function returns without recursing, eliminating any risk of stack overflow despite the extreme parameter value. ### How does infinite epsilon differ from a very large finite epsilon in practice? Mathematically, both produce the same result: only the endpoints are retained. However, using `np.inf` is more explicit about the intent to apply maximum simplification, and it avoids potential floating-point overflow issues that could theoretically occur when squaring extremely large finite numbers, though modern systems handle this robustly. ### Can infinite epsilon cause the algorithm to hang or enter an infinite loop? No. In the iterative implementation (`douglas_simplify_iter`), infinite epsilon causes the initial segment to be processed and immediately discarded without pushing new segments to the queue. The queue becomes empty after the first iteration, naturally terminating the loop. The recursive version exits immediately without further calls.