Create unique matrices from existing matrix using datenum

Hello,

I have an ever expanding dataset with each new row being a unique timestamp. Data is continuously collected at 30 second intervals while in operation. At times, there is no operation and so the rows can be quite disconnected.

I would like to extract all columns from the example matrix below if the datenum value is within a certain specification. In this example lets use 30 seconds. If two adjacent rows are 30 seconds or less I want to create a new unique matrix containing all the rows. The data would be split if the two adjacent rows are greater than 30 seconds.

Here is some example data. This is a double matrix.

A = [datenum, data1, data2, data3, data4, data5]
737497.231944445	1408	0	0	0	0
737497.232291667	1409	0	0	0	0
737497.232638889	1410	0	0	0	0
737497.232986111	1411	0	0	0	0
737497.233333333	1412	0	0	0	0
737497.233680556	1413	0	0	0	0
737497.234027778	1414	0	0	0	0
737497.234375000	1415	0	0	0	0
737497.234722222	1416	0	0	0	0
737497.235069444	1417	0	0	0	0
737497.235416667	1418	0	0	0	0
737497.235763889	1419	0	0	0	0
737497.236111111	1420	0	0	0	0
737497.236458333	1421	0	0	0	0
737497.236805556	1422	0	0	0	0
737497.237152778	1423	0	0	0	0
737497.237500000	1424	0	0	0	0
737497.238888889	1425	0	0	0	0
737497.239236111	1426	0	0	0	0
737497.240277778	1427	0	0	0	0
737497.240972222	1428	0	0	0	0
737497.241319445	1429	0	0	0	0
737497.241666667	1430	0	0	0	0
737497.242013889	1431	0	0	0	0

I want to separate A into unique smaller matrices. Each of these unique matices will have the same number of columns, but a different number of rows based on if they are within 30 seconds of each other.

In this case we have 24 total rows. Based on the datenum and 30 seconds = 3.4722e-4 (in datenum format), the data should be separated into 5 new matrices as such

Rows: 1-17, 18-19, 20, 21, 22-24.

I've started to tackle this using a for loop but am unsure of how to proceed.

seconds = 3.4722e-4;
    for i=1:length(A)
        if A(i+1) - A(i) > seconds

How do I assign unique names to the newly created matrix. It would be grand if I could use the timestamp from the datenum of the first row in the newly creted matrices.

Is this possible?

[dpb]

Your example has rounding issues...

>> find([0; diff(A(:,1))]>(30/(24*3600)))
ans =
     6
    11
    15
    18
    20
    21
    22
>> 

Maybe converting to the datetime class will help...

>> find([seconds(0); diff(datetime(A(:,1),'ConvertFrom','datenum'))]>seconds(30))
ans =
     6
    11
    15
    18
    20
    21
    22
>>

Nope...what's the precision of the original datestamp? Having it to work with would undoubtedly make coding simpler than trying to deal with the above...

[Stephen23]

"How do I assign unique names to the newly created matrix."

Do NOT do this. Dynamically accessing variable names is one way that beginners force thmselvevs into writing slow, complex code that is hard to debug. Read this to know why:

https://www.mathworks.com/matlabcentral/answers/304528-tutorial-why-variables-should-not-be-named-dynamically-eval

Just use indexing. Indexing is simple, neat, and very efficient. Unlike what you want to do.

[Stephen23]

Two efficient ways to bin data with fixed bin edges:

• use a histogram function, or
• even better you should use a timetable:

https://www.mathworks.com/help/matlab/timetables.html

[dpb]

He's still going to have issues with the precision problems...

[Eric Escoto]

Hi dpb,

I'm not sure what you mean by the precision problems. There is a precision problem in MATLAB as it converts my datetime to datenum. instead of 30 seconds it goes to 29.9998 I believe.

However, there is never less than that datenum value of 29.9998 between readings if the sensor is in operation. The output from the datalogger is simply 30 seconds. So I'm not following the precision problem you speak of.

So far A. Sawas has provided a fine method that seems to work. The round function takes the 29.9998 datenumvalue to a whole 30 and then the rows are split.

[dpb]

How did you generate the datenums? What was the input? If it is really 30 sec differential, then the datenum diff() will be to machine precision of a double not on order of 10E-4.

Answer 1

[A. Sawas]

I am not sure what you are planning to do eventually. However, the following code does what you have explained.

% smallest time step
step = 30;
% convert timestamps into seconds, divide by the time step then do round (ignore time resolutions less than step)
A1 = round(A(:,1)*24*3600/step);
% get the (difference between each two consecutive elements) -1
D = diff(A1)-1;
% cut locations
cutLoc = [0; find(D); size(A1,1)];
% assign the submatrices into cell array
for k=1:numel(cutLoc)-1
    subA{k} = A(cutLoc(k)+1:cutLoc(k+1),:);
end