Calculate standard deviation of 5-minute for different time interval

Hi, everyone.

Here I attached my observations data on 1-31 of January 2014 for 24 hours from different transmitter (PRN) . By this excel, the 1-minute average for VTEC, S4 and Sigma for each 31 days already coded as below.

My problem now is, how can I find shortcut to calculate the standard deviation with the time interval of 5 min from the dataTT1min ? Should I reshape the column and row by separating it daily? Or how can I do by using for loop function?

Example:

TimeStamp VTEC S4 Sigma

'01-01-14 00:05' 15.0339395135249 0.0532669682730302 0.0447520000000000

'01-01-14 00:06' 10.1911139214755 0.0471003695720050 0.0373840000000000

'01-01-14 00:07' 12.1389652170293 0.0716977380449952 0.0451266666666667

'01-01-14 00:08' NaN NaN NaN

'01-01-14 00:09' 15.6912781124977 0.0547043336042455 0.0438950000000000

'01-01-14 00:10' 3.85456568208492 0.0395215802440296 0.0354220000000000

'01-01-14 00:11' 3.0019381831895 0.0687632803996655 0.0408106666666667

'01-01-14 00:12' 15.6748996657200 0.0903593103110256 0.0454846000000000

'01-01-14 00:13' NaN NaN NaN

'01-01-14 00:14' 10.5174985112160 0.0448554406539539 0.0380092500000000

Then, expected to get std VTEC from 00:05-00:09 and 00:10-00:14.Later will resulting only 288 std for one day (approx 8,928 std for January).

Any assistance would be highly appreciated.

data = readtable("Book_Jan.xlsx");
%_______________Read data_______________%

%___Convert DAY and TIME into durations___%
data.DAY = days(data.DAY);
data.TIME = days(data.TIME);

%___Create a datetime___%
data.TimeStamp = datetime(2014,01,01) + data.DAY-1 + data.TIME;
data.TimeStamp.Format = "MM-dd-yy HH:mm";

%___Convert the table to a timetable___%
dataTT = table2timetable(data,"RowTimes","TimeStamp");

%___Use retime to average the data into 1 minute increments___%
dataTT1min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),"minutely","mean");

Answer 1

[Mathieu NOE]

hello Ann

I slightly modified your code so the data are resampled with time increment of 5 minutes (or whatever value you're looking for)

hope it helps !

data = readtable("Book_Jan.xlsx");

%_______________Read data_______________%
%___Convert DAY and TIME into durations___%
data.DAY = days(data.DAY);
data.TIME = days(data.TIME);

%___Create a datetime___%
data.TimeStamp = datetime(2014,01,01) + data.DAY-1 + data.TIME;
data.TimeStamp.Format = "MM-dd-yy HH:mm";

%___Convert the table to a timetable___%
dataTT = table2timetable(data,"RowTimes","TimeStamp");

%% MN : % round time to nearest 5 min and take unique values only
tt = data.TimeStamp;
tt.Minute = 5 * floor(tt.Minute/5); % here ! 5 means 5 mins resampling
tt.Second = 0;
NEWTIMESTEP = unique(tt);

%___Use retime to average the data into 5 minute increments___%
dataTT5min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),NEWTIMESTEP);

[Eric Sofen]

First, instead of specifying a whole new time vector in retime, you can specify the timestep:

retime(dataTT,minutes(5),...)

To get the standard deviation: The retime method can be a function handle when you're doing a retime operation that's an aggregation, so similar to the last line in Mathieu's code...

dataTT5min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),NEWTIMESTEP,@std);

[Mathieu NOE]

hello Eric

thank you for the info ! I'll keep that in the back of my head !

regards

[Ann]

Much obliged for the responses from you both.

Mathieu's code perfectly fine with additional standard deviation (stdev) from Eric.

While I'm trying to do adjustment from what Eric shared regarding using the retime and below is what perfectly worked for average 5-min. Unfortunately, I couldn't figure out to find the stdev. Tried to use retime to find stdev but still resulting the 5-min average value.

dataTT1min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),"regular","mean","TimeStep",minutes(5));

[Mathieu NOE]

@ Eric

sorry , I couldn't make it to work

tried both options but got errors. How am I suppose to write it ?

tx

dataTT5min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),minutes(5));
dataTT5min = retime(dataTT,minutes(5));

[Mathieu NOE]

on my side still have problem :

>> dataTT5min = retime(dataTT(:,["VTEC" "S4" "Sigma"]),"TimeStep",minutes(5));
Error using timetable/retime (line 140)
'TimeStep' is not a valid specification of target time vector for synchronization.

[Ann]

Maybe this explanation can help you for a better understanding.

https://www.mathworks.com/matlabcentral/answers/522204-how-can-i-retime-my-timetable-with-a-10days-timestep#answer_429566

[Mathieu NOE]

Hi

tx for the link

It was simply lacking the ,"regular","mean" portion of the code !