# Purpose of the Hampel Filter in ESPectre: Robust Outlier Detection for CSI Turbulence

> Discover the purpose of the Hampel filter in ESPectre. It effectively detects and removes outlier spikes from CSI turbulence data, ensuring accurate motion analysis and signal integrity.

- Repository: [Francesco Pace/espectre](https://github.com/francescopace/espectre)
- Tags: deep-dive
- Published: 2026-06-10

---

**The Hampel filter in ESPectre detects and removes outlier spikes from raw CSI turbulence measurements before movement-velocity-scaling (MVS) calculations, preventing false-positive motion detection while preserving the underlying signal shape.**

The ESPectre (micro‑ESPectre) signal‑processing pipeline relies on this filter to clean Wi‑Fi Channel State Information (CSI) data on ESP32‑based devices. Raw amplitude time‑series from CSI often contain large spikes caused by short‑term RF interference or hardware glitches that artificially inflate turbulence metrics. The Hampel filter provides a computationally efficient, **non‑smoothing** outlier removal mechanism optimized for MicroPython constraints.

## Why ESPectre Needs Outlier Filtering

Wi‑Fi CSI turbulence measurements serve as the foundation for movement‑velocity‑scaling (MVS) calculations in real‑time occupancy sensing. Occasional hardware glitches and RF interference introduce spikes that:

- Artificially inflate turbulence metrics
- Trigger false‑positive motion detections  
- Degrade reliability in low‑power, real‑time use cases

Traditional smoothing filters introduce latency and distort genuine motion signals. The Hampel filter addresses these issues by clipping only extreme deviations while leaving legitimate variation untouched.

## How the Hampel Filter Works in ESPectre

The implementation follows the classic Hampel identifier adapted for microcontroller constraints.

### Sliding Window Median and MAD Calculation

The filter maintains a sliding window of the last *W* samples. For each incoming value, it computes:

- The **median** of the current window
- The **median absolute deviation (MAD)** from that median

### Threshold‑Based Outlier Replacement

If the current sample deviates from the median by more than **T × MAD** (where *T* is a configurable threshold), the sample is replaced by the median. This clips extreme outliers while preserving the shape of genuine motion‑related variations.

## Implementation Details in micro‑ESPectre

The ESPectre Hampel filter is implemented in `micro‑espectre/src/filters.py` (lines 14‑38) within the `HampelFilter` class. The code is specifically optimized for MicroPython:

- **Pre‑allocated buffers** eliminate memory allocation during real‑time processing
- **Circular buffer** architecture stores the sliding window efficiently  
- **Insertion sort** routine handles small‑N median calculations

This design ensures the filter runs efficiently on ESP32 devices with limited RAM.

## Integration in the Signal Processing Pipeline

The filter operates at a specific stage in the ESPectre signal chain:

1. **Raw CSI → Turbulence calculation**: Each sub‑carrier’s amplitude series passes through a `HampelFilter` instance immediately after turbulence computation
2. **Non‑smoothing stage**: Unlike subsequent optional low‑pass filtering, the Hampel stage removes only spikes without attenuating motion signals
3. **MVS preprocessing**: Cleaned turbulence values feed into the movement‑velocity‑scaling detector to produce robust motion estimates

## Configuration and Usage Examples

Instantiate the filter with window size and threshold parameters:

```python
from micro_espectre.src.filters import HampelFilter

# Window = 7 samples, threshold = 4×MAD

hampel = HampelFilter(window_size=7, threshold=4.0)

# Apply to a stream of turbulence values

cleaned = []
for raw_value in raw_turbulence_series:
    cleaned.append(hampel.filter(raw_value))

```

In the ESPectre analysis tools, the filter is conditionally applied per sub‑carrier:

```python

# From micro-espectre/tools/5_analyze_filter_turbulence.py (lines 82-84)

hampel = HampelFilter(window_size=HAMPEL_WINDOW,
                      threshold=HAMPEL_THRESHOLD) if config.get('hampel', False) else None

```

## Testing and Validation

The implementation includes comprehensive validation utilities:

- **Unit testing**: [`test/test/test_hampel_filter/test_hampel_filter.cpp`](https://github.com/francescopace/espectre/blob/main/test/test/test_hampel_filter/test_hampel_filter.cpp) validates that the filter reduces maximum spike amplitude while keeping the baseline unchanged (line 48)
- **Parameter optimization**: `micro‑espectre/tools/6_optimize_filter_params.py` performs grid searches to determine optimal window sizes and thresholds for specific environments  
- **Integration testing**: `micro‑espectre/tests/test_segmentation.py` ensures the filter remains enabled by default in production pipelines

## Summary

- The Hampel filter in ESPectre removes RF interference and hardware glitch spikes from Wi‑Fi CSI turbulence data before MVS calculations
- It uses a sliding window median and MAD calculation with configurable threshold (*T*) and window (*W*) parameters
- Implementation in `micro‑espectre/src/filters.py` prioritizes MicroPython efficiency through pre‑allocated circular buffers and insertion sort
- The filter improves detection reliability without sacrificing temporal resolution, essential for real‑time ESP32 applications
- Configuration occurs via the `HampelFilter` class constructor and can be enabled via the `hampel` config flag in analysis tools

## Frequently Asked Questions

### What is the difference between the Hampel filter and a low-pass filter in ESPectre?

The Hampel filter is a **non‑smoothing** outlier detector that replaces only extreme spikes with the window median. A low‑pass filter attenuates high‑frequency components across the entire signal. In ESPectre, the Hampel stage runs before optional low‑pass smoothing to ensure motion‑related variations remain unaltered while only removing anomalous spikes.

### How do you configure the Hampel filter window size and threshold?

Configure the `HampelFilter` class via the `window_size` and `threshold` constructor arguments in `micro‑espectre/src/filters.py`. The `window_size` defines how many samples (*W*) are used for median calculation, while `threshold` sets the multiplier (*T*) for MAD‑based outlier detection. These values can be overridden in analysis scripts like [`5_analyze_filter_turbulence.py`](https://github.com/francescopace/espectre/blob/main/5_analyze_filter_turbulence.py).

### Why does ESPectre use MAD instead of standard deviation for outlier detection?

The **median absolute deviation (MAD)** is a robust statistic that is less sensitive to existing outliers in the window than standard deviation. Since the calculation itself must remain resistant to the very spikes it attempts to detect, MAD provides reliable outlier identification even when the window contains multiple anomalous values.

### Where is the Hampel filter applied in the ESPectre codebase?

The core implementation resides in `micro‑espectre/src/filters.py` (lines 14‑38) within the `HampelFilter` class. It is applied in `micro‑espectre/tools/5_analyze_filter_turbulence.py` during turbulence analysis and tested in [`test/test/test_hampel_filter/test_hampel_filter.cpp`](https://github.com/francescopace/espectre/blob/main/test/test/test_hampel_filter/test_hampel_filter.cpp). Parameter optimization utilities are available in [`6_optimize_filter_params.py`](https://github.com/francescopace/espectre/blob/main/6_optimize_filter_params.py).