Bad time format, import to datetime

Hi everyone,

because of my sensor, I have trouble importing the data. The output data is suboptimal, but I hope there is an easy solution. So this is the time format the sensor gave me, it is stored in a table column (195500x1 table) as strings

%Example strings in the table
"2021-07-18T17:41:22.1692422+02:00"
"2021-07-18T17:41:22.1732451+02:00"
"2021-07-18T17:41:22.1752471+02:00"
"2021-07-18T17:41:22.1792497+02:00"
"2021-07-18T17:41:22.1812511+02:00"

How can I safely read these strings with a complete precision (22.1792497), and store them in a variable (195500x1)? I would also like them to have the datetime format to match them with another datetime variable. later, I would like to convert them to duration.

Thanks!

Answer 1

[Stephen23]

"Bad time format"

Good time format: it looks like a completely standard ISO 8601 timestamp

https://en.wikipedia.org/wiki/ISO_8601

txt = [...
    "2021-07-18T17:41:22.1692422+02:00"
    "2021-07-18T17:41:22.1732451+02:00"
    "2021-07-18T17:41:22.1752471+02:00"
    "2021-07-18T17:41:22.1792497+02:00"
    "2021-07-18T17:41:22.1812511+02:00"];
dtm = datetime(txt,'InputFormat','y-M-d''T''H:m:s.SSSSSSSZZZZZ','TimeZone','UTC')

dtm = 5×1 datetime array
   18-Jul-2021 15:41:22
   18-Jul-2021 15:41:22
   18-Jul-2021 15:41:22
   18-Jul-2021 15:41:22
   18-Jul-2021 15:41:22

dtm.TimeZone = '+2' % if you prefer

dtm = 5×1 datetime array
   18-Jul-2021 17:41:22
   18-Jul-2021 17:41:22
   18-Jul-2021 17:41:22
   18-Jul-2021 17:41:22
   18-Jul-2021 17:41:22

drn = timeofday(dtm);
drn.Format = 'hh:mm:ss.SSSSSSS'

drn = 5×1 duration array
   17:41:22.1692422
   17:41:22.1732450
   17:41:22.1752471
   17:41:22.1792497
   17:41:22.1812511

So not the complete precision, but pretty close.

[Lukas]

It needs to be completely precise, because i need to match the timepoints. Is that not possible? drn.Format = 'hh:mm:ss.SSSSSSS' seems to put out the exact number?

[Stephen23]

"It needs to be completely precise"

DATETIME objects (and presumably DURATION too) ultimately use binary floating point numbers to store their data, which has inherent limited precision. If you want "completely" precise then those will be the wrong tools. However it is very unlilkely that the original data source had infinite precision time, so the quest is likely to be trying to get more precision out of data which simply does not exist...

Anyway, lets increase the displayed number of digits and see what happens

txt = [...
    "2021-07-18T17:41:22.1692422+02:00"
    "2021-07-18T17:41:22.1732451+02:00"
    "2021-07-18T17:41:22.1752471+02:00"
    "2021-07-18T17:41:22.1792497+02:00"
    "2021-07-18T17:41:22.1812511+02:00"];
dtm = datetime(txt,'InputFormat','y-M-d''T''H:m:s.SSSSSSSZZZZZ','TimeZone','UTC');
dtm.TimeZone = '+2';
drn = timeofday(dtm);
drn.Format = 'hh:mm:ss.SSSSSSSSS'

drn = 5×1 duration array
   17:41:22.169242200
   17:41:22.173245099
   17:41:22.175247100
   17:41:22.179249700
   17:41:22.181251100

"because i need to match the timepoints. Is that not possible?"

The recommended approach with floating point numbers is to compare the absolute difference against some acceptable tolerance. I don't see any reason why we can't do something similar with DATETIME/DURATION objects:

tol = duration(0,0,0.000001);
new = duration(17,41,[22.173244,22.173245]);
new.Format = 'hh:mm:ss.SSSSSSSSS'

new = 1×2 duration array
   17:41:22.173244000   17:41:22.173244999

abs(drn-new)<tol

ans = 5×2 logical array
   0   0
   0   1
   0   0
   0   0
   0   0

[Lukas]

Yes, thank you. The sensor creates a datapoint every 3 µs, so to reliably determine the value, it would need to be a precision of +/- 2 µs. I think that your example shows that this works. Thank you!