Hampel Filter
Filter outliers using Hampel identifier
Libraries:
DSP System Toolbox /
Filtering /
Filter Designs
Description
The Hampel Filter block detects and removes the outliers of the input signal by using the Hampel identifier. The Hampel identifier is a variation of the threesigma rule of statistics, which is robust against outliers. For each sample of the input signal, the block computes the median of a window composed of the current sample and $$\frac{Len1}{2}$$ adjacent samples on each side of the current sample. Len is the window length you specify through the Window length parameter. The block also estimates the standard deviation of each sample about its window median by using the median absolute deviation. If a sample differs from the median by more than the threshold multiplied by the standard deviation, the filter replaces the sample with the median. For more information, see Algorithms.
Ports
Input
Output
Parameters
Block Characteristics
Data Types 

Direct Feedthrough 

Multidimensional Signals 

VariableSize Signals 

ZeroCrossing Detection 

More About
Algorithms
For a given sample of data, x_{s}, the algorithm:
Centers the window of odd length at the current sample.
Computes the local median, m_{i}, and standard deviation, σ_{i}, over the current window of data.
Compares the current sample with n_{σ} × σ_{i}, where n_{σ} is the threshold value. If $$\left{x}_{s}{m}_{i}\right>{n}_{\sigma}\times {\sigma}_{i}$$, the filter identifies the current sample, x_{s}, as an outlier and replaces it with the median value, m_{i}.
Consider a frame of data that is passed into the Hampel filter.
In this example, the Hampel filter slides a window of length 5 (Len) over the data. The filter has a threshold value of 2 (n_{σ}). To have a complete window at the beginning of the frame, the filter algorithm prepends the frame with Len – 1 zeros. To compute the first sample of the output, the window centers on the $${\left[\frac{Len1}{2}+1\right]}^{\text{th}}$$ sample in the appended frame, the third zero in this case. The filter computes the median, median absolute deviation, and the standard deviation over the data in the local window.
Current sample: x_{s} = 0.
Window of data: win = [0 0 0 0 1].
Local median: m_{i} = median([0 0 0 0 1]) = 0.
Median absolute deviation: $$ma{d}_{i}=\mathrm{median}\left(\left{x}_{ik}{m}_{i}\right,\dots ,\left{x}_{i+k}{m}_{i}\right\right)$$. For this window of data, $$mad=\mathrm{median}\left(\left00\right,\dots ,\left10\right\right)=0$$.
Standard deviation: σ_{i} = κ × mad_{i} = 0, where $$\kappa =\frac{1}{\sqrt{2}{\mathrm{erfc}}^{1}\left(1/2\right)}\approx 1.4826$$.
The current sample, x_{s} = 0, does not obey the relation for outlier detection.
$$\left[\left{x}_{s}{m}_{i}\right=0\right]>\left[\left({n}_{\sigma}\times {\sigma}_{i}\right)=0\right]$$
Therefore, the Hampel filter outputs the current input sample, x_{s} = 0.
Repeat this procedure for every succeeding sample until the
algorithm centers the window on the $${\left[End\frac{Len1}{2}\right]}^{\text{th}}$$ sample, marked
as End
. Because the window centered on the last $$\frac{Len1}{2}$$ samples cannot
be full, these samples are processed with the next frame of input
data.
Here is the first output frame the Hampel filter generates:
The seventh sample of the appended input frame, 23, is an outlier. The Hampel filter replaces this sample with the median over the local window [4 9 23 8 12].
References
[1] Bodenham, Dean. “Adaptive Filtering and Change Detection for Streaming Data.” Ph.D. Thesis. Imperial College, London, 2012.
[2] Liu, Hancong, Sirish Shah, and Wei Jiang. “Online outlier detection and data cleaning.” Computers and Chemical Engineering. Vol. 28, March 2004, pp. 1635–1647.
[3] Suomela, Jukka. Median Filtering Is Equivalent to Sorting, 2014.
Extended Capabilities
Version History
Introduced in R2017a