How to remove or identify spikes in a random signal.

I have signal from different RFID tags. I just want to remove the spikes occurs in the signal.

Data file and signal images is attached. I have to remove the highlighted spikes.

Thanks.

[Walter Roberson]

rmoutliers() ?

[Subhash Sagar]

Thanks Walter.

In my case, rmoutlier() is not working. it just return almost same signal.

[Image Analyst]

"remove" means to clip/clamp/saturate, or to completely delete those elements from the array, essentially shortening it (it's less wide than before)?

Can you count on the spike always being below a threshold, and good values being above some threshold?

[Subhash Sagar]

Thanks.

It can be any, either completely remove it or to saturate the values based on the previous trend. Both or either of them will work for me.

As its a real time data, we cannot set a threshold as in the current signal, outlier has lower as compared with previous value. But it is also possible that outlier value will be higher than the previous one.

Answer 1

[Akira Agata]

Looking at your data, negative spikes you mentioned is always less than 1000. So, if this condition is applicable to other data, you can simply remove them by:

idx = t1(:,3) < 1000;
t1(idx,:) = [];

If you have to detect negative peaks with more complex conditions, I believe findpeaks function will be some help.

[Subhash Sagar]

Thanks Akira,

This data is just a reading of some tag, not always the spike is less 1000. it is also possible that spikes will be the value greater than 1000 or 1500 or more. I have such data as well. detecting outlier with value greater than previous value is relativily easy to find. For greater values, we can use rmoutlier.

[Akira Agata]

Thank you for your clarification.

Actually, rmoutlier might be one possible solution. But this function removes both higher- and lower-side outliers.

To remove lower-side outliers only, the following is one solution.

load('tag1.mat');

% Identify outliers of lower side
[~,lo] = isoutlier(double(t1(:,3)));
idx = t1(:,3) < int64(lo);

% Remove identified outliers
t1(idx,:) = [];

[Subhash Sagar]

Thank you.

It somewhat worked for me.