Timestamp Precision Loss

This was a question that popped into my head when I was looking at timestamps in Python.

In Python, time.time will give you the current time as a number of seconds (like 1722903780.0) since January 1st, 1970; the Unix Epoch. There’s various other reference points of time for computers if you’re interested – Windows for instance, counts the number of 100 nanosecond increments from January 1st, 1601. The time that time.time is, is a double floating point value, which only has some 15 digits of precision, meaning that the current sub-second precision will eventually be lost – but when?

The way that precision is lost in such floating points is via powers of 2. It stores a fixed number of digits and an exponent, similar to scientific notation. For example, if I had the number 9.997 x 10^17 and added the number 0.004 x 10 ^ 17 to it and could only ever keep four digits and the exponent value, the resulting value I get would be 10.00 x 10 ^ 17, not 10.001 x 10 ^ 17. I’ve lost the precision of knowing the value to the 0.001 x 10 ^ 17-ths place. This is what happens with floating points, but in base 2 (along with a whole bunch of other weirdness).

So I wrote a search function that prints out the precision lost after going up by powers of 2:

from datetime import datetime, timedelta
import numpy as np
import numtxt
import sys

def search():
    "unix time precision loss search"
    ts = sys.float_info.epsilon
    
    for i in range(128):
        prec = np.nextafter(ts, np.inf) - ts
        sipc = numtxt.si(prec, 's', fmt='{:9.05f}')
        
        dt   = datetime.utcfromtimestamp(ts)
        line = f'Precision at {dt:%a %b %e %Y, %H:%M:%S.%f} is {sipc}'

        #yr = 1970 + ts / (365.24225 * 24 * 60 *60)
        #line = f'Precision at year {yr:.6f} is {sipc}'

        #line += f' (timestamp: {ts})'
        print(line)
        ts *= 2

And here’s its output:

Precision at Thu Jan  1 1970, 00:00:00.000000 is   0.04930 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   0.09861 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   0.19722 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   0.39443 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   0.78886 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   1.57772 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   3.15544 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   6.31089 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  12.62177 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  25.24355 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  50.48710 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 100.97420 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 201.94839 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 403.89678 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 807.79357 qs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   1.61559 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   3.23117 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   6.46235 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  12.92470 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  25.84939 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is  51.69879 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 103.39758 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 206.79515 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 413.59031 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is 827.18061 rs
Precision at Thu Jan  1 1970, 00:00:00.000000 is   1.65436 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is   3.30872 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is   6.61744 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is  13.23489 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is  26.46978 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is  52.93956 ys
Precision at Thu Jan  1 1970, 00:00:00.000000 is 105.87912 ys
Precision at Thu Jan  1 1970, 00:00:00.000001 is 211.75824 ys
Precision at Thu Jan  1 1970, 00:00:00.000002 is 423.51647 ys
Precision at Thu Jan  1 1970, 00:00:00.000004 is 847.03295 ys
Precision at Thu Jan  1 1970, 00:00:00.000008 is   1.69407 zs
Precision at Thu Jan  1 1970, 00:00:00.000015 is   3.38813 zs
Precision at Thu Jan  1 1970, 00:00:00.000031 is   6.77626 zs
Precision at Thu Jan  1 1970, 00:00:00.000061 is  13.55253 zs
Precision at Thu Jan  1 1970, 00:00:00.000122 is  27.10505 zs
Precision at Thu Jan  1 1970, 00:00:00.000244 is  54.21011 zs
Precision at Thu Jan  1 1970, 00:00:00.000488 is 108.42022 zs
Precision at Thu Jan  1 1970, 00:00:00.000977 is 216.84043 zs
Precision at Thu Jan  1 1970, 00:00:00.001953 is 433.68087 zs
Precision at Thu Jan  1 1970, 00:00:00.003906 is 867.36174 zs
Precision at Thu Jan  1 1970, 00:00:00.007812 is   1.73472 as
Precision at Thu Jan  1 1970, 00:00:00.015625 is   3.46945 as
Precision at Thu Jan  1 1970, 00:00:00.031250 is   6.93889 as
Precision at Thu Jan  1 1970, 00:00:00.062500 is  13.87779 as
Precision at Thu Jan  1 1970, 00:00:00.125000 is  27.75558 as
Precision at Thu Jan  1 1970, 00:00:00.250000 is  55.51115 as
Precision at Thu Jan  1 1970, 00:00:00.500000 is 111.02230 as
Precision at Thu Jan  1 1970, 00:00:01.000000 is 222.04460 as
Precision at Thu Jan  1 1970, 00:00:02.000000 is 444.08921 as
Precision at Thu Jan  1 1970, 00:00:04.000000 is 888.17842 as
Precision at Thu Jan  1 1970, 00:00:08.000000 is   1.77636 fs
Precision at Thu Jan  1 1970, 00:00:16.000000 is   3.55271 fs
Precision at Thu Jan  1 1970, 00:00:32.000000 is   7.10543 fs
Precision at Thu Jan  1 1970, 00:01:04.000000 is  14.21085 fs
Precision at Thu Jan  1 1970, 00:02:08.000000 is  28.42171 fs
Precision at Thu Jan  1 1970, 00:04:16.000000 is  56.84342 fs
Precision at Thu Jan  1 1970, 00:08:32.000000 is 113.68684 fs
Precision at Thu Jan  1 1970, 00:17:04.000000 is 227.37368 fs
Precision at Thu Jan  1 1970, 00:34:08.000000 is 454.74735 fs
Precision at Thu Jan  1 1970, 01:08:16.000000 is 909.49470 fs
Precision at Thu Jan  1 1970, 02:16:32.000000 is   1.81899 ps
Precision at Thu Jan  1 1970, 04:33:04.000000 is   3.63798 ps
Precision at Thu Jan  1 1970, 09:06:08.000000 is   7.27596 ps
Precision at Thu Jan  1 1970, 18:12:16.000000 is  14.55192 ps
Precision at Fri Jan  2 1970, 12:24:32.000000 is  29.10383 ps
Precision at Sun Jan  4 1970, 00:49:04.000000 is  58.20766 ps
Precision at Wed Jan  7 1970, 01:38:08.000000 is 116.41532 ps
Precision at Tue Jan 13 1970, 03:16:16.000000 is 232.83064 ps
Precision at Sun Jan 25 1970, 06:32:32.000000 is 465.66129 ps
Precision at Wed Feb 18 1970, 13:05:04.000000 is 931.32257 ps
Precision at Wed Apr  8 1970, 02:10:08.000000 is   1.86265 ns
Precision at Tue Jul 14 1970, 04:20:16.000000 is   3.72529 ns
Precision at Sun Jan 24 1971, 08:40:32.000000 is   7.45058 ns
Precision at Wed Feb 16 1972, 17:21:04.000000 is  14.90116 ns
Precision at Wed Apr  3 1974, 10:42:08.000000 is  29.80232 ns
Precision at Tue Jul  4 1978, 21:24:16.000000 is  59.60464 ns
Precision at Mon Jan  5 1987, 18:48:32.000000 is 119.20929 ns
Precision at Sat Jan 10 2004, 13:37:04.000000 is 238.41858 ns
Precision at Tue Jan 19 2038, 03:14:08.000000 is 476.83716 ns
Precision at Sun Feb  7 2106, 06:28:16.000000 is 953.67432 ns
Precision at Wed Mar 16 2242, 12:56:32.000000 is   1.90735 μs
Precision at Wed May 30 2514, 01:53:04.000000 is   3.81470 μs
...
OSError: [Errno 22] Invalid argument

So as of this writing, time stamps as floating point have a precision of 238.42 nanoseconds. This means, at best, you could have a resolution of 200 nanoseconds written in your time stamps:

1722903780.0000000
1722903780.0000002
1722903780.0000005
1722903780.0000007
1722903780.0000010
1722903780.0000012
1722903780.0000014
1722903780.0000017
1722903780.0000019
1722903780.0000021

Whether you can actually get that kind of resolution or not is an entirely separate question of how good your clock is.

This precision will drop to 476.8 nanosecond increments on Tuesday, January 19th 2038 at 3:14:08 UTC time, the same time as the Year 2038 problem. Microsecond precision loss won’t happen until Wednesday March 16th 2242 at 12:56:32 UTC time which is quite a while away – though at that point we’d probably want to be storing time in pico or femtoseconds and likely be using a different storage method for time. Complete sub-second precision (thus, only have 1 second precision) will be lost some time around the year 142715360. Can’t say exactly what year as it’s dependent on how long of a year is and how much the length of a year has changed between now and 142 million years from now.

Or you could not use a floating point data type to store time; Windows storage method stores time in set increments of 100 nanoseconds as two integers and will be good until the year 30828. Python’s datetime module stores single microsecond resolution until the end of the year 9999.

But if you want to store single nanosecond resolution in a float, you could use a different epoch as the reference point but you’ll always lose that precision after 8388608 seconds (≈ 97 days). If you’re really insistent at storing time in this kind of data type but want nanosecond resolution for years to come, you could use quadruple precision! That won’t lose nanosecond precision until the year 306474867611246098.

program quad_resolution
    ! fortran code using quad precision
    use iso_fortran_env
    implicit none

    real(real128) ts, yr_len
    yr_len = 365.24225 * 24 * 60 * 60
    ts = 1

    do while (spacing(ts) <= 2.0)
        print *, 'Precision at year', 1970 + (ts / yr_len), 'is', spacing(ts)
        ts = ts * 2
    end do

end program quad_resolution

(P.S. that’s only 22.229 million times the age of the universe!)

Tagged with: ,
Posted in Uncategorized

Leave a comment

In Archive
Design a site like this with WordPress.com
Get started