How to detect variations in data?

Hello everyone,

I would like to determine the location at which there is data variation (like the highlighted points in the attached image).

Kindly find the attached code + .mat file.

Each sub-plot represents a variable on y-axis and the time on x-axis. Usually, the variation appear in several variables (plots) like this one.

Could you suggest the best method to do so?

and How can I determine the starting and ending points of that variation?

I appreciate your help.

Answer 1

[Image Analyst]

You might try movstd() to do a sliding window standard deviation. Then threshold it to find elements where the standard deviation is what you deem "high". See attached code where I did it for your "B" signal. Adapt it to do any other signals you want.

[Mohamed Nedal]

Thank you very much!

Answer 2

[Ridwan Alam]

Depends on how complex of a solution you are looking for. There are algorithms for change point detection that you can try.

https://www.mathworks.com/help/signal/ref/findchangepts.html

Or, you can have a sliding window (your defined size) over the signal and look for specific features (e.g mean crossing rate, variance, etc. depending your signal type) to find the windows that contain a "change".

Hope this helps.

[Mohamed Nedal]

Can you show me an example plz?

It would make it much easier to understand.

[Ridwan Alam]

Please check the link above, there are some examples. Try on your own data, and if there is any difficulty, please feel free to ask about that.

[Mohamed Nedal]

Yes, it gives what I need for now. Thanks!

But I need to plot several findchangepts plots as subplots in a single figure. I found it can't be done unfortunately.

Do you know how to do that?

[Ridwan Alam]

Why can’t it be done? Please show your code and I will try to debug.

[Mohamed Nedal]

Here's what I was trying.

% define 3 column vectors 
x1 = rand(30,1); 
x2 = rand(30,1); 
x3 = rand(30,1); 
% first subplot 
subplot(3,1,1) 
findchangepts(x1,'MaxNumChanges',5); 
% second subplot 
subplot(3,1,2) 
findchangepts(x2,'MaxNumChanges',5); 
% third subplot 
subplot(3,1,3) 
findchangepts(x3,'MaxNumChanges',5); 

[Image Analyst]

Did you also give my code a try?

[Mohamed Nedal]

Yes, I tried it and it works. Thanks!

But it doesn't depend on findchangepts function, which is fine, I just would like to compare your code's result with that of findchangepts results.

[Image Analyst]

I think findchangepts() locates points where the data goes from smooth to something drastically different, or vice versa. So it basically finds corners or elbows in the data.

movstd() on the other hand gives a high signal wherever the data has high variability regardless if the signal looks like a corner or elbow there.

[Mohamed Nedal]

Okay, thanks a lot.