Getting UTC time from Posix time - leap seconds

Posix time is almost based on UTC, but doesn't include leap seconds that get thrown in at unpredictable (at least to me) intervals every few years.

I can convert posix time to calendar time in the date time format using datetime(xxx,'ConvertFrom','posixtime') but I need time in UTC. Does the

conversion add back in the missing leap seconds? There have been 26 leap seconds since 1970, and I suspect we're due for another soon. I

don't want to have to modify my code everytime there's a leap second.

[Yair Altman]

For the benefit of potential users who just need plain POSIX-to-UTC datetime conversion and don't care about the leap seconds, an easy and super-fast method is to use the vectorized input format of the datenum function. i.e.

datenum([1970 1 1 0 0 posixValue])

For example:

>> datestr(datenum([1970 1 1 0 0 129450789]))
ans =
    '07-Feb-1974 06:33:09'

I am aware that this does not directly answer the poster's question (where leap secs are indeed needed), but in most cases for readers coming across this post, these leap seconds are not very important and then they could use datenum, on all Matlab releases (even R2019b or earlier).

Answer 1

[Swatantra Mahato]

Hi,

I am assuming you want to convert posixtime to UTC and take into account the leap seconds.

This can be done using the "datetime" function by also passing the 'TimeZone' Property as 'UTCLeapSeconds'. As an example to show the difference, consider the script:

d1=datetime(129450789,'ConvertFrom','posixtime','TimeZone','UTC'); %07-Feb-1974 06:33:09
d2=datetime(129450789,'ConvertFrom','posixtime','TimeZone','UTCLeapSeconds'); 
d3=datetime(14383421,'ConvertFrom','posixtime','TimeZone','UTC'); %16-Jun-1970 11:23:41
d4=datetime(14383421,'ConvertFrom','posixtime','TimeZone','UTCLeapSeconds');
a=d1-d3;
b=d2-d4;

On executing "a-b" on the MATLAB command line we get:

>> a-b

ans =

duration

-00:00:03

which is in line with there being 3 leap seconds in the duration considered.

You can refer to the documentation for the "datetime" function for more information:

https://www.mathworks.com/help/matlab/ref/datetime.html

Hope this helps

[Bruce McGuffin]

I'm still not sure. If I execute:

pt = 1.603458785041000e+09;

dt1 = datetime(pt,'ConvertFrom','posixtime','Format','uuuu-MM-dd''T''HH:mm:ss:SSS''Z''')

dt2 = datetime(pt,'ConvertFrom','posixtime','TimeZone','UTC','Format','uuuu-MM-dd''T''HH:mm:ss:SSS''Z''')

dt3 = datetime(pt,'ConvertFrom','posixtime','TimeZone','UTCLeapSeconds')

I get the same answer for all 3: 2020-10-23T13:13:05.041Z

All 'UTCLeapSeconds' appears to do is format the output. So my question remains, in converting from posix time to UTC, does matlab add in the right number of leap seconds?

The documetation at https://www.mathworks.com/help/matlab/ref/datetime.html says using 'TimeZone', 'UTCLeapSeconds' "accounts for leap seconds". That's kind of vague. What does it mean?

Incidentally, when I execute d1 and d2 I get the same time, just formatted differently. Same for d3 and d4. It's true that d1-d3 and d2-d4 give different results, so there must be something about datetime I don't understand. But that doesn't really matter because I'm converting the output to a character string for insertion into an XML document.

[Swatantra Mahato]

From my understanding of the "datetime" function, using the "UTCLeapSeconds" TimeZone does not modify the value of the output, but takes leap seconds into account when calculating the duration between two dates in this timezone.

In the example 'd1-d3' and 'd2-d4' have a difference of 3 seconds, which is because there are 3 leap seconds in the time between 16-Jun-1970 and 07-Feb-1974, the dates considered.

Hope this helps

[Bruce McGuffin]

So just to be clear, converting posix time to datetime format using the datetime command with arguement pair 'TimeZone', 'UTCleapseconds' does not add missing leap seconds to the result, but the missing leap seconds may be hidden away in some undocumented part of the datetime structure.

It looks like I could find the number of leap seconds since June 30, 1972 using the 'leapseconds' command, but that doesn't appear to be available in the version of matlab I am currently using. When was it added?

[Steven Lord]

According to the entry at the end of the leapseconds documentation page it was Introduced in R2020a.

Answer 2

[Peter Perkins]

Bruce, you are getting tripped up by the difference between clockface time, and the elapsed time since 1970. Here's the deal:

As you already know, the "real actual" UTC time line includes leap seconds that are inserted at somewhat unpredictable intervals. So in the "real actual" UTC time line, if you count the number of seconds since 1970, you get something that's 27s longer than what you might have expected. In MATLAB, you can demonstrate that using datetime, and MATLAB calls that timeline 'UTCLeapSeconds':

>> t = datetime(2020,11,18,'TimeZone','UTCLeapSeconds')
t = 
  datetime
   2020-11-18T00:00:00.000Z
>> t - datetime(1970,1,1,'TimeZone','UTCLeapSeconds')
ans = 
  duration
   446016:00:27

But here's the thing: almost noone wants to hear that "truth". So the POSIX time line, and datetime's default "unzoned" behavior, and datetime's behavior for 'UTC' tell a white lie

>> t = datetime(2020,11,18,'TimeZone','UTC','Format','dd-MMM-uuuu HH:mm:ss.SSS')
t = 
  datetime
   18-Nov-2020 00:00:00.000
>> t - datetime(1970,1,1,'TimeZone','UTC')
ans = 
  duration
   446016:00:00

because most people don't want their calculations involving elapsed times to "randomly" be off by several seconds. I can't tell which group of people you are in, but if all you are ultimately doing is converting to a text clockface timestamp, it won't matter unless one of your timestamps falls on a leapsecond. Generally speaking, people who need to care about leaps seconds are often doing things with satellites or similar, and they can opt in.

More details:

When you say, "converting posix time to datetime format using the datetime command with arguement pair 'TimeZone', 'UTCleapseconds' does not add missing leap seconds to the result", that's not actually true. You have to consider that POSIX times are not on the "leap seconds" time line, so converting to the latter has to account for the leap seconds. For a POSIX time around now, that means adding in 27s, but what's tripping you up is that it does not change the clockface time. In other words, the POSIX time for midnight 18-Nov-2020 may claim to be 1605657600s since 1970, but midnight 18-Nov-2020 in the "real actual" UTC timeline that 'UTCLeapSeconds' represents is 1605657627s since 1970. These two things

>> t1 = datetime(2020,11,18,'TimeZone','UTCLeapSeconds')
t1 = 
  datetime
   2020-11-18T00:00:00.000Z
>> t1 = datetime(2020,11,18,'TimeZone','UTC','Format','dd-MMM-uuuu HH:mm:ss.SSS')
t1 = 
  datetime
   18-Nov-2020 00:00:00.000

look the same, and refer to the same instant in time, but calculating elapsed times with them is not the same.

In short: converting POSIX to/from (MATLAB's) 'UTC' is straight-forward because they both tell the same white lie. Converting to/from 'UTCLeapSeconds' needs, internally, to add in leap seconds in one direction, and remove them in the other. And of course, what is the POSIX time for 31-Dec-2016 23:59:60Z? Uhhhh ... it doesn't exist. It's an ill-defined question. So datetime maps that back to 31-Dec-2016 23:59:59Z.