Is there a best way to identify anomalous peaks in signals and remove them?

Hi to everybody!

I need some advices regarding how to find big peaks in a signal (both positive or negative). My data are hydrograms where in the x axis there is the datetime (hourly measurements) and in the y axis there is "flowls". I looked for the outliers detection in matlab using the movmean but I didn't find any changes and replacements.

This is my code:

clear; close; clc;
load('data_prova')
 t= DateTime;
[B,TF,U,L,C] = filloutliers(flowls,'previous','movmean',5)
plot(t,flowls,t,B,'o')
legend('Original Data','Filled Data')

Alternatively, is there another way to do that?

Thanks in advance

[Walter Roberson]

"Outliers" is basically a matter of intention. In some other context, the exact same points of the exact same data might be information (for example about turbulence).

Therefore there cannot be a single "best" way.

[Image Analyst]

Can you attach your screenshot so we can get a peek before having to go to the trouble of downloading and running it?

[EM geo]

As you can see here I plotted t vs flowls and I have some big peaks (positive and negative). I would like to identify them as possibile outliers and remove from the plot in order to remove noise.

[Mathieu NOE]

hello

a few thougts ,

1. why not simply average the data with movmean ?
2. or use movmean to create a reference line and says we remove data that are 10% (to be checked) above and below this reference line ?

Answer 1

[KALYAN ACHARJYA]

"identify anomalous peaks in signals and remove them?"

You can do this in many ways, it depends on you and your exact needs. But you must have to decide some threshold %, based on it pre and sucessive neighbours.

1. Apply the diff, if the difference, if the different quite large as per threshold %, you can delete that indices datapoints.
2. It is not sure, whether those peaks because on single outliers data point or multiple, if moltiple case, you can do movesum to get the indices ranges.

Your job to get the indices, where these anomalous peaks appears, and remove those data as follows

data([index positions])=[]

The resulting data length will be shorter than the original data point. Here you have to contact locally. Other methods are also available, please answer the same question as the related MATLAB.