In my experience, the simplest way to program with times and dates is nearly always to convert to Unix epoch as early as you can in the process. Do all processing in Epoch, then convert back to something readable by humans at the last possible moment. This approach won't work in all circumstances, but it should be the standard approach unless there's a compelling reason not to.
Disagree. Use a date/time library and datatypes. Don't use integers, unix timestamps, or any other naive representation of time.
As with cryptography, don't attempt to write your own date/time code. You will get it wrong. Use the datatypes and library functions in your language or database.
Hi, I'm the author of the blog post and the library used in the examples.
For many operations the Kalends library actually converts to UTC (and UTC is almost the same as epoch). For instance to calculate the amount of seconds between two datetimes.
Your advice might work in a lot of cases, but if you have a good datetime library you should not need to IMO. But I am sorry to say that I do not think most datetime libraries are very good when it comes to timezones.
You don't have to manually say: oh this datetime is not UTC, I better convert it as soon as possible. If you need to save a future datatime which is not UTC, it is actually error prone to convert to UTC.
What Kalends does instead is to have a struct with an unambiguous, validated datetime and the timezone along with it.
That way, if for instance you are dealing with a datetime entered by a user, you have the actual datetime entered, but you still have all information you need to convert to UTC.
>If you need to save a future datatime which is not UTC, [...] a struct with an unambiguous, validated datetime and the timezone along with it.
It's less ambiguous but it's not 100% unambiguous because future policies of Daylight Saving Time and Time Zone / Time offset boundaries can be changed by governments.
Even if the struct had extra fields for flags such as "obey future DST changes" or "ignore present TZ if different from historic TZ at time of data entry", it's still ambiguous if geographic coordinates are not embedded within the struct. Then again, there's probably some more edge cases even with latitude/longitude coordinates. (E.g. should the future time be interpreted at lat/long of where he data-entered the datetime, or the lat/long of where he later experiences the datetime?!)
Dates specifying future events are very tricky++. Unless, it's a future date for something celestial such as a solar eclipse; in such cases, a future date as UTC is unambiguous.
++Because we all conflate concepts of "socially-constructed future datetime as shown on a wall clock" and "scientific future datetime". We stuff both concepts into the same data structures. Nevertheless, we write software that makes educated guesses about the user intentions for "future dates" and it works for 99% of the typical use cases. (e.g. "send smartphone notification for 12:30 lunch with Bob next week.)
Right, it is unambiguous for the current rules and zone. Usually if the rules change, you want to do the calculation again. I made a library for persisting future DateTimes to a database. Whenever a DateTime is loaded from the database, the calculation is done again. A blog post about it can be found at http://www.creativedeletion.com/2015/03/19/persisting_future...
If the "borders of the timezones" change, that does not make it unambiguous, because the struct is defined as belong to a timezone name, not a specific geographical location.
What I find missing from most, if not all, date/time libraries is a three-arg timezone conversion function (i.e. one that is pure and doesn't rely on the current time or locale). Most timezone conversions take a zoned date and time and convert it to a different timezone. But it matters when/where you're doing the conversion for the reasons you're discussing. If you also supply a UTC timestamp indicating when the conversion should take place (note, not a zoned date and time, since the conversion to the universal timeline can be affected by DST changes), that conversion can be consistent no matter when/where the function runs.
I find this approach to be cleaner than saving wall time since it allows you to keep uniform times on the server and simplify server-side logic that acts on those times in bulk.
If the "borders of the timezones" change, that does not make it unambiguous, because the struct is defined as belong to a timezone name, not a specific geographical location.
The "TZ name" designation can change for a location so even if you store it in the struct, it's still ambiguous. It's not a common occurrence but it has happened before. It is impossible to store a 100% unambiguous socially-constructed future datetime that works for all edge cases.
To clarify, I'm not saying your datetime library is "incorrect" or needs to change. I'm just pointing out that your guarantee about it recording an "unambiguous future date" is not possible to achieve for all edge cases.
To use an Epoch you have to convert it into a timezone aware date (as you stated, as late as possible). Now that could be UTC but more likley it will be a timeZone that observes a daylight savings time.
Consider the epoch of 0, 1st of Jan 1970 or what ever it is. Well that's correct in GMT but if you are in NYC what time is it, how about Singapore, Australia, Huston etc...
I think it's best to make an explicit, type-level distinction between these things. JodaTime does this: you can have an Instant (which represents a point in time) or a DateTime (a human-readable date/time with a timezone), and if you try to pass one where the other is needed then you get a compile error.
I agree on the point of always be converting, but I'll settle with UTC. IMHO if you have a non-utc time one if two things should happen:
A.) Convert it to Utc.
B.) Show it to someone.
Got some 'splain'n to do if I catch non-utc manipulation or comparison going on in my code bases. And heaven help you should you stick a non-utc time into the database.
These are all good points and I have thought about it a lot.. It seems that all these issues arise from people wanting to schedule or track a future date that is itself tentative and ambiguous; at the mercy of policy changes.
Ignoring the case where you want to allow a user to track some tentative future date that can resolve to different absolute times there is another truth there: You recorded a future absolute time which now maps to a different local time due to your government mucking about. What about the people in Hamburg who were supposed to join the meeting but missed it because the absolute time shifted?
For usability sake I do absolutely see where you are coming from and the library is surely nice. But I'll stick with storing and mathing UTC unless I absolutely need to start taking these scheduling issues into account :)
I'm not sure I understand the point of saving the offset:
"To avoid ambigiuty we also save a third piece of data: the UTC offset. Note that the UTC offset is only there for the rare cases when the timezone does not change, but when there is an ambiguity we know about at scheduling time - that will usually be when the timezone goes off of DST and a range of time (usually an hour) happens twice in autumn when clocks are set back. In this example though this third piece of data is not used for anything."
This doesn't make sense to me? Great read otherwise. I hadn't considered this edge case!
The reason the offset is changed is ambigous datetimes that occur when going off of DST.
Let us say a user enters a datetime in the autumn. It happens to be a few minutes after the clocks are set back an hour. So the time is ambiguous. Which datetime exactly does the user want? The one before or after the clocks are set back? We show him a dialog and he chooses one. To differentiate we use the standard offset which is usally 0 or 3600 seconds.
Great, now we save it to the database as just wall time and timezone name and load it again. How do we know if it is 02:05 before the clocks are set back or 02:05 after the clocks are set back? We don't, unless we saved the UTC offset. But if we save the offset we know exactly which one it is.
But we don't want to use the offset we calculated from the beginning. That would get us the same error as if saving just the UTC time. So we only use the offset if we absolutely have to. This is what Kalecto does when loading a saved datetime from the library: we do the calculation based on the saved "wall time" and the timezone name. If that wall time is ambiguous we use the UTC offset. If there were no changes to the rules for that timezone - no problems.
Now, if we have a combination of the timezone rules changing and the saved datetime being ambiguous, the entered choice is no longer valid. We have to ask the user again or report an error.
Why do you prefer UTC? I feel storing plain unix times is simplest since there's no time zone information at all. And there's no possibility of forgetting a conversion, since (with most languages/libraries) different types are used for plain unix times vs. local times.
Just to be make sure we're talking about the same thing, by "plain unix time" I mean something like java.time.Instant, and by "UTC time" I mean something like java.time.ZonedDateTime with zone == UTC.
With ZonedDateTime, there's obviously a time zone field. We can have a convention of always setting it to UTC, but that introduces a possible source of bugs. We might call ZonedDateTime.now() and forget to pass an explicit zone. Especially if the author is a new contributor who hasn't been told about our convention.
Or we might use a library which produces ZonedDateTimes with the system's default zone. Then we have to canonicalize them at some point, and be very careful not to rely on the local date/time info prior to canonicalization.
I'd rather just use a type like Instant which has no time zone field, so that there's no need for any time zone convention.
Further, we might need to serialize a timestamp using a non-UTC timezone, say CST which is common in payments systems. If we're using UTC times, we might forget to convert from UTC to CST. If we're using a plain unix time, we're forced to specify a time zone when serializing, so it's harder to make a mistake.
This works brilliantly until you discover that the Unix epoch is ambiguous around leap seconds. Good luck sorting epoch-stamped events around leap seconds!
Comments
In my experience, the simplest way to program with times and dates is nearly always to convert to Unix epoch as early as you can in the process. Do all processing in Epoch, then convert back to something readable by humans at the last possible moment. This approach won't work in all circumstances, but it should be the standard approach unless there's a compelling reason not to.
Disagree. Use a date/time library and datatypes. Don't use integers, unix timestamps, or any other naive representation of time.
As with cryptography, don't attempt to write your own date/time code. You will get it wrong. Use the datatypes and library functions in your language or database.
Hi, I'm the author of the blog post and the library used in the examples.
For many operations the Kalends library actually converts to UTC (and UTC is almost the same as epoch). For instance to calculate the amount of seconds between two datetimes.
Your advice might work in a lot of cases, but if you have a good datetime library you should not need to IMO. But I am sorry to say that I do not think most datetime libraries are very good when it comes to timezones.
You don't have to manually say: oh this datetime is not UTC, I better convert it as soon as possible. If you need to save a future datatime which is not UTC, it is actually error prone to convert to UTC.
What Kalends does instead is to have a struct with an unambiguous, validated datetime and the timezone along with it.
That way, if for instance you are dealing with a datetime entered by a user, you have the actual datetime entered, but you still have all information you need to convert to UTC.
>If you need to save a future datatime which is not UTC, [...] a struct with an unambiguous, validated datetime and the timezone along with it.
It's less ambiguous but it's not 100% unambiguous because future policies of Daylight Saving Time and Time Zone / Time offset boundaries can be changed by governments.
Even if the struct had extra fields for flags such as "obey future DST changes" or "ignore present TZ if different from historic TZ at time of data entry", it's still ambiguous if geographic coordinates are not embedded within the struct. Then again, there's probably some more edge cases even with latitude/longitude coordinates. (E.g. should the future time be interpreted at lat/long of where he data-entered the datetime, or the lat/long of where he later experiences the datetime?!)
Dates specifying future events are very tricky++. Unless, it's a future date for something celestial such as a solar eclipse; in such cases, a future date as UTC is unambiguous.
++Because we all conflate concepts of "socially-constructed future datetime as shown on a wall clock" and "scientific future datetime". We stuff both concepts into the same data structures. Nevertheless, we write software that makes educated guesses about the user intentions for "future dates" and it works for 99% of the typical use cases. (e.g. "send smartphone notification for 12:30 lunch with Bob next week.)
Right, it is unambiguous for the current rules and zone. Usually if the rules change, you want to do the calculation again. I made a library for persisting future DateTimes to a database. Whenever a DateTime is loaded from the database, the calculation is done again. A blog post about it can be found at http://www.creativedeletion.com/2015/03/19/persisting_future...
If the "borders of the timezones" change, that does not make it unambiguous, because the struct is defined as belong to a timezone name, not a specific geographical location.
What I find missing from most, if not all, date/time libraries is a three-arg timezone conversion function (i.e. one that is pure and doesn't rely on the current time or locale). Most timezone conversions take a zoned date and time and convert it to a different timezone. But it matters when/where you're doing the conversion for the reasons you're discussing. If you also supply a UTC timestamp indicating when the conversion should take place (note, not a zoned date and time, since the conversion to the universal timeline can be affected by DST changes), that conversion can be consistent no matter when/where the function runs.
I find this approach to be cleaner than saving wall time since it allows you to keep uniform times on the server and simplify server-side logic that acts on those times in bulk.
The "TZ name" designation can change for a location so even if you store it in the struct, it's still ambiguous. It's not a common occurrence but it has happened before. It is impossible to store a 100% unambiguous socially-constructed future datetime that works for all edge cases.
To clarify, I'm not saying your datetime library is "incorrect" or needs to change. I'm just pointing out that your guarantee about it recording an "unambiguous future date" is not possible to achieve for all edge cases.
Thanks for replying. I fully agree that having a good library is essential, so good on you for writing one for Elixir.
I agree but this is only half the battle.
To use an Epoch you have to convert it into a timezone aware date (as you stated, as late as possible). Now that could be UTC but more likley it will be a timeZone that observes a daylight savings time.
Consider the epoch of 0, 1st of Jan 1970 or what ever it is. Well that's correct in GMT but if you are in NYC what time is it, how about Singapore, Australia, Huston etc...
I think it's best to make an explicit, type-level distinction between these things. JodaTime does this: you can have an Instant (which represents a point in time) or a DateTime (a human-readable date/time with a timezone), and if you try to pass one where the other is needed then you get a compile error.
I agree on the point of always be converting, but I'll settle with UTC. IMHO if you have a non-utc time one if two things should happen:
A.) Convert it to Utc.
B.) Show it to someone.
Got some 'splain'n to do if I catch non-utc manipulation or comparison going on in my code bases. And heaven help you should you stick a non-utc time into the database.
Here is an example where the "convert it to UTC" rule would fail: http://www.creativedeletion.com/2015/03/19/persisting_future...
I will stick non-UTC time into the database and I made a library to do it.
These are all good points and I have thought about it a lot.. It seems that all these issues arise from people wanting to schedule or track a future date that is itself tentative and ambiguous; at the mercy of policy changes.
Ignoring the case where you want to allow a user to track some tentative future date that can resolve to different absolute times there is another truth there: You recorded a future absolute time which now maps to a different local time due to your government mucking about. What about the people in Hamburg who were supposed to join the meeting but missed it because the absolute time shifted?
For usability sake I do absolutely see where you are coming from and the library is surely nice. But I'll stick with storing and mathing UTC unless I absolutely need to start taking these scheduling issues into account :)
I'm not sure I understand the point of saving the offset:
"To avoid ambigiuty we also save a third piece of data: the UTC offset. Note that the UTC offset is only there for the rare cases when the timezone does not change, but when there is an ambiguity we know about at scheduling time - that will usually be when the timezone goes off of DST and a range of time (usually an hour) happens twice in autumn when clocks are set back. In this example though this third piece of data is not used for anything."
This doesn't make sense to me? Great read otherwise. I hadn't considered this edge case!
The reason the offset is changed is ambigous datetimes that occur when going off of DST.
Let us say a user enters a datetime in the autumn. It happens to be a few minutes after the clocks are set back an hour. So the time is ambiguous. Which datetime exactly does the user want? The one before or after the clocks are set back? We show him a dialog and he chooses one. To differentiate we use the standard offset which is usally 0 or 3600 seconds.
Great, now we save it to the database as just wall time and timezone name and load it again. How do we know if it is 02:05 before the clocks are set back or 02:05 after the clocks are set back? We don't, unless we saved the UTC offset. But if we save the offset we know exactly which one it is.
But we don't want to use the offset we calculated from the beginning. That would get us the same error as if saving just the UTC time. So we only use the offset if we absolutely have to. This is what Kalecto does when loading a saved datetime from the library: we do the calculation based on the saved "wall time" and the timezone name. If that wall time is ambiguous we use the UTC offset. If there were no changes to the rules for that timezone - no problems.
Now, if we have a combination of the timezone rules changing and the saved datetime being ambiguous, the entered choice is no longer valid. We have to ask the user again or report an error.
I hope this explains it better.
Yes much better thanks!
Why do you prefer UTC? I feel storing plain unix times is simplest since there's no time zone information at all. And there's no possibility of forgetting a conversion, since (with most languages/libraries) different types are used for plain unix times vs. local times.
There's no time zone information in UTC either. People often mistake UTC for a time zone, but it isn't actually one.
Just to be make sure we're talking about the same thing, by "plain unix time" I mean something like java.time.Instant, and by "UTC time" I mean something like java.time.ZonedDateTime with zone == UTC.
With ZonedDateTime, there's obviously a time zone field. We can have a convention of always setting it to UTC, but that introduces a possible source of bugs. We might call ZonedDateTime.now() and forget to pass an explicit zone. Especially if the author is a new contributor who hasn't been told about our convention.
Or we might use a library which produces ZonedDateTimes with the system's default zone. Then we have to canonicalize them at some point, and be very careful not to rely on the local date/time info prior to canonicalization.
I'd rather just use a type like Instant which has no time zone field, so that there's no need for any time zone convention.
Further, we might need to serialize a timestamp using a non-UTC timezone, say CST which is common in payments systems. If we're using UTC times, we might forget to convert from UTC to CST. If we're using a plain unix time, we're forced to specify a time zone when serializing, so it's harder to make a mistake.
Unless your users are happy to view UTC you will have to deal with it when you display it to them.
This works brilliantly until you discover that the Unix epoch is ambiguous around leap seconds. Good luck sorting epoch-stamped events around leap seconds!