Purpose of the Hampel Filter in ESPectre: Robust Outlier Detection for CSI Turbulence

The Hampel filter in ESPectre detects and removes outlier spikes from raw CSI turbulence measurements before movement-velocity-scaling (MVS) calculations, preventing false-positive motion detection while preserving the underlying signal shape.

The ESPectre (micro‑ESPectre) signal‑processing pipeline relies on this filter to clean Wi‑Fi Channel State Information (CSI) data on ESP32‑based devices. Raw amplitude time‑series from CSI often contain large spikes caused by short‑term RF interference or hardware glitches that artificially inflate turbulence metrics. The Hampel filter provides a computationally efficient, non‑smoothing outlier removal mechanism optimized for MicroPython constraints.

Why ESPectre Needs Outlier Filtering

Wi‑Fi CSI turbulence measurements serve as the foundation for movement‑velocity‑scaling (MVS) calculations in real‑time occupancy sensing. Occasional hardware glitches and RF interference introduce spikes that:

• Artificially inflate turbulence metrics
• Trigger false‑positive motion detections
• Degrade reliability in low‑power, real‑time use cases

Traditional smoothing filters introduce latency and distort genuine motion signals. The Hampel filter addresses these issues by clipping only extreme deviations while leaving legitimate variation untouched.

How the Hampel Filter Works in ESPectre

The implementation follows the classic Hampel identifier adapted for microcontroller constraints.

Sliding Window Median and MAD Calculation

The filter maintains a sliding window of the last W samples. For each incoming value, it computes:

• The median of the current window
• The median absolute deviation (MAD) from that median

Threshold‑Based Outlier Replacement

If the current sample deviates from the median by more than T × MAD (where T is a configurable threshold), the sample is replaced by the median. This clips extreme outliers while preserving the shape of genuine motion‑related variations.

Implementation Details in micro‑ESPectre

The ESPectre Hampel filter is implemented in micro‑espectre/src/filters.py (lines 14‑38) within the HampelFilter class. The code is specifically optimized for MicroPython:

• Pre‑allocated buffers eliminate memory allocation during real‑time processing
• Circular buffer architecture stores the sliding window efficiently
• Insertion sort routine handles small‑N median calculations

This design ensures the filter runs efficiently on ESP32 devices with limited RAM.

Integration in the Signal Processing Pipeline

The filter operates at a specific stage in the ESPectre signal chain:

1. Raw CSI → Turbulence calculation: Each sub‑carrier’s amplitude series passes through a HampelFilter instance immediately after turbulence computation
2. Non‑smoothing stage: Unlike subsequent optional low‑pass filtering, the Hampel stage removes only spikes without attenuating motion signals
3. MVS preprocessing: Cleaned turbulence values feed into the movement‑velocity‑scaling detector to produce robust motion estimates

Configuration and Usage Examples

Instantiate the filter with window size and threshold parameters:

from micro_espectre.src.filters import HampelFilter

# Window = 7 samples, threshold = 4×MAD

hampel = HampelFilter(window_size=7, threshold=4.0)

# Apply to a stream of turbulence values

cleaned = []
for raw_value in raw_turbulence_series:
    cleaned.append(hampel.filter(raw_value))

In the ESPectre analysis tools, the filter is conditionally applied per sub‑carrier:

# From micro-espectre/tools/5_analyze_filter_turbulence.py (lines 82-84)

hampel = HampelFilter(window_size=HAMPEL_WINDOW,
                      threshold=HAMPEL_THRESHOLD) if config.get('hampel', False) else None

Testing and Validation

The implementation includes comprehensive validation utilities:

• Unit testing: test/test/test_hampel_filter/test_hampel_filter.cpp validates that the filter reduces maximum spike amplitude while keeping the baseline unchanged (line 48)
• Parameter optimization: micro‑espectre/tools/6_optimize_filter_params.py performs grid searches to determine optimal window sizes and thresholds for specific environments
• Integration testing: micro‑espectre/tests/test_segmentation.py ensures the filter remains enabled by default in production pipelines

Summary

• The Hampel filter in ESPectre removes RF interference and hardware glitch spikes from Wi‑Fi CSI turbulence data before MVS calculations
• It uses a sliding window median and MAD calculation with configurable threshold (T) and window (W) parameters
• Implementation in micro‑espectre/src/filters.py prioritizes MicroPython efficiency through pre‑allocated circular buffers and insertion sort
• The filter improves detection reliability without sacrificing temporal resolution, essential for real‑time ESP32 applications
• Configuration occurs via the HampelFilter class constructor and can be enabled via the hampel config flag in analysis tools

Frequently Asked Questions

What is the difference between the Hampel filter and a low-pass filter in ESPectre?

The Hampel filter is a non‑smoothing outlier detector that replaces only extreme spikes with the window median. A low‑pass filter attenuates high‑frequency components across the entire signal. In ESPectre, the Hampel stage runs before optional low‑pass smoothing to ensure motion‑related variations remain unaltered while only removing anomalous spikes.

How do you configure the Hampel filter window size and threshold?

Configure the HampelFilter class via the window_size and threshold constructor arguments in micro‑espectre/src/filters.py. The window_size defines how many samples (W) are used for median calculation, while threshold sets the multiplier (T) for MAD‑based outlier detection. These values can be overridden in analysis scripts like 5_analyze_filter_turbulence.py.

Why does ESPectre use MAD instead of standard deviation for outlier detection?

The median absolute deviation (MAD) is a robust statistic that is less sensitive to existing outliers in the window than standard deviation. Since the calculation itself must remain resistant to the very spikes it attempts to detect, MAD provides reliable outlier identification even when the window contains multiple anomalous values.

Where is the Hampel filter applied in the ESPectre codebase?

The core implementation resides in micro‑espectre/src/filters.py (lines 14‑38) within the HampelFilter class. It is applied in micro‑espectre/tools/5_analyze_filter_turbulence.py during turbulence analysis and tested in test/test/test_hampel_filter/test_hampel_filter.cpp. Parameter optimization utilities are available in 6_optimize_filter_params.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}