www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Article Review: Migrating from std.date to std.datetime

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
It recently came to my attention that an article on converting code from using 
std.date to using std.datetime would be of value, so I wrote one up. Since 
it's an article, and it's within the time period set by Walter for the article 
contest, I guess that it's in the article contest, but I wrote it because it 
needed writing rather than for the contest. It might as well be part of the 
contest though.

In any case, I put it in my d-programming-language.org repository on github: 
https://github.com/jmdavis/d-programming-language.org/tree/article_datetime

Or if you just want to download it without building d-programming-language.org 
yourself, go here: http://is.gd/jFAmDy

So, feel free to read it and give feeback on it. Hopefully it does its job 
fairly well. Writing it did give me some insight into a couple of tweaks that 
need to be made to std.datetime though. In particular, I really should come up 
with a better way to save time zones for those that care about restoring the 
_exact_ time zone that they had in the SysTime that they saved in a database 
or to disk or whatever. ISO is great for storing the exact time - including 
what the offset from UTC of the original time zone was - but it's not 
generally enough for restoring the exact time zone that you were dealing with 
in the first place. Most code won't care, but I really should find a way to do 
it. I knew about the problem, but it's definitely a pain to solve, so I left 
it alone. I should work on fixing that.

In any case, the article is now available for you to read and review.

- Jonathan M Davis
May 14 2011
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
Thanks a lot for writing the article. I was just about to get rid of 
std.date and migrate to std.datetime, so it's perfect timing. ;-)

A few nitpicks:

- a short motivation for using hecto-nano-seconds would be nice, it's 
not really the most obvious choice.

- Reading this code:

 d_time dTime = "myfile.txt".lastModified();
 SysTime sysTime = "myfile.txt".timeLastModified();

Using UFCS distracted me quite a bit from the actual point, better use standard call syntax. When migrating, I hit a few problems: - there is an alias for std.string.indexOf in std.datetime, that causes ambiguities when imported with other modules. (AFAICT it is used to disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad by themselves, but it'd be better to use explicite std.string.indexOf in the few places in std.datetime). - the documentation for std.datetime.WindowsTimeZone is missing on the website - the documentation for std.file.getTimesWin/Posix is not found on the website (why the different function names?) - my current uses of datetime are comparing file times and displaying the file time. Much better than std.date, the times displayed are now the same as shown by Explorer/dir most of the time, but some are off by one hour. It seems this happens for times with a different daylight saving. Is this a bug? Or do I need to call something else than writeln(timeLastModified(file).toSimpleString()); The ISO versions or conversion to DateTime produce the same output. My local time is UTC+1+DST. - running the unittests for std.datetime takes more than 10 minutes until it assert here: Failed time zone: America/Los_Angeles core.exception.AssertError std\datetime.d(27696): _assertPred!"==" failed: [Pazifik Normalzeit] is not == [Pacific Standard Time]. It seems the test strings need to be localized aswell. Do the tests also run this long under linux? My impression from stopping execution inside a debugger is that constructing SysTime is rather slow (most time seems to be spent in SystemTimeToTzSpecificLocalTime). Rainer Jonathan M Davis wrote:
 It recently came to my attention that an article on converting code from using 
 std.date to using std.datetime would be of value, so I wrote one up. Since 
 it's an article, and it's within the time period set by Walter for the article 
 contest, I guess that it's in the article contest, but I wrote it because it 
 needed writing rather than for the contest. It might as well be part of the 
 contest though.
 
 In any case, I put it in my d-programming-language.org repository on github: 
 https://github.com/jmdavis/d-programming-language.org/tree/article_datetime
 
 Or if you just want to download it without building d-programming-language.org 
 yourself, go here: http://is.gd/jFAmDy
 
 So, feel free to read it and give feeback on it. Hopefully it does its job 
 fairly well. Writing it did give me some insight into a couple of tweaks that 
 need to be made to std.datetime though. In particular, I really should come up 
 with a better way to save time zones for those that care about restoring the 
 _exact_ time zone that they had in the SysTime that they saved in a database 
 or to disk or whatever. ISO is great for storing the exact time - including 
 what the offset from UTC of the original time zone was - but it's not 
 generally enough for restoring the exact time zone that you were dealing with 
 in the first place. Most code won't care, but I really should find a way to do 
 it. I knew about the problem, but it's definitely a pain to solve, so I left 
 it alone. I should work on fixing that.
 
 In any case, the article is now available for you to read and review.
 
 - Jonathan M Davis

May 15 2011
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
Jonathan M Davis wrote:
 On 2011-05-15 03:34, Rainer Schuetze wrote:
 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)

 A few nitpicks:

 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.

It's the highest precision that you can have and still have a decent range of values in a 64-bit integer. Going by 10 ns units would result in the range being too small, but 100 ns gives you from around 29,000 B.C. to 29,000 A.D. It's also what C# uses.

Thanks for the clarification (I know it's also the resolution from the Windows API). I suggest to put it somewhere in the article or the documentation.
 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).

That's a bug. I shouldn't have done that. I didn't think about the alias being public when I did it. It has now been fixed on github.

Unfortunately, for some obscure reason, private module declarations are still placed into the overload set, so your fix does not help. You should either alias it to something unlikely to clash with something else or use fully qualified names (though this does not allow UFCS).
 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website

I don't know why it would be. It's probably a bug relating to the new StdDdoc versioning that we switched to (instead of using D_Ddoc which screwed up people who build the documentation at the same time as their code - a bad practice IMHO, but we decided to not set it up so that it didn't work when using Phobos). As WindowsTimeZone is only on Windows, it definitely had to be versioned with StdDdoc.

I just noticed that it shows up on http://www.d-programming-language.org/phobos/std_datetime.html, but not on http://www.digitalmars.com/d/2.0/phobos/std_datetime.html
 
 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)

Probably a StdDdoc problem as well.

also on d-programming-lanuage.org.
May 15 2011
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
Jonathan M Davis wrote:
 On 2011-05-15 03:34, Rainer Schuetze wrote:
 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than

 writeln(timeLastModified(file).toSimpleString());

 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.

I'd need more data to have any idea what's going on, but it sounds like a bug.

I have investigated it a bit, and I'm currently thinking that Explorer, dir and Total Commander are wrong, and std.datetime is right. What I did to test: - create a new file, verify the dates printed are correct. fine in all programs - set the system time back to February - create the file again, verify the dates printed are correct. fine in all programs - set the system time back to May - now the displayed file time is the same when using std.datetime, but an hour later with all other programs. Maybe it's an XP or NTFS issue, but I would expect the file time not to change with the date it is displayed.
May 16 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 03:34, Rainer Schuetze wrote:
 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)
 
 A few nitpicks:
 
 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.

It's the highest precision that you can have and still have a decent range of values in a 64-bit integer. Going by 10 ns units would result in the range being too small, but 100 ns gives you from around 29,000 B.C. to 29,000 A.D. It's also what C# uses.
 - Reading this code:
  > d_time dTime = "myfile.txt".lastModified();
  > SysTime sysTime = "myfile.txt".timeLastModified();
 
 Using UFCS distracted me quite a bit from the actual point, better use
 standard call syntax.
 
 When migrating, I hit a few problems:
 
 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).

That's a bug. I shouldn't have done that. I didn't think about the alias being public when I did it. It has now been fixed on github.
 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website

I don't know why it would be. It's probably a bug relating to the new StdDdoc versioning that we switched to (instead of using D_Ddoc which screwed up people who build the documentation at the same time as their code - a bad practice IMHO, but we decided to not set it up so that it didn't work when using Phobos). As WindowsTimeZone is only on Windows, it definitely had to be versioned with StdDdoc.
 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)

Probably a StdDdoc problem as well. As for the different names, the that's because of the fact that they give you different values. Windows has file creation times. Posix does not. Instead, it gives you the last time that the file status was changed (which historically, std.file has mistakenly been returning for the creation time of files on Posix).
 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than
 
 writeln(timeLastModified(file).toSimpleString());
 
 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.

I'd need more data to have any idea what's going on, but it sounds like a bug.
 - running the unittests for std.datetime takes more than 10 minutes
 until it assert here:
 
 Failed time zone: America/Los_Angeles
 core.exception.AssertError std\datetime.d(27696): _assertPred!"=="
 failed: [Pazifik Normalzeit] is not == [Pacific Standard Time].
 
 It seems the test strings need to be localized aswell.

I didn't even know that time zone strings were ever localized, and there's no way that the unit tests are going to be able to deal with that other than not testing stuff that would ideally be being tested. I'll have to figure out what to about that.
 Do the tests also run this long under linux? My impression from stopping
 execution inside a debugger is that constructing SysTime is rather slow
 (most time seems to be spent in SystemTimeToTzSpecificLocalTime).

The unit tests have never taken anywhere near that long for me. I'm in the midst of rewriting them though (some of that rewrite was in the last release), so their performance characteristics could change. It tests a lot though, so I'd expect it to take longer than the other Phobos modules. - Jonathan M Davis
May 15 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 05:26, Rainer Schuetze wrote:
 Jonathan M Davis wrote:
 On 2011-05-15 03:34, Rainer Schuetze wrote:
 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)
 
 A few nitpicks:
 
 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.

It's the highest precision that you can have and still have a decent range of values in a 64-bit integer. Going by 10 ns units would result in the range being too small, but 100 ns gives you from around 29,000 B.C. to 29,000 A.D. It's also what C# uses.

Thanks for the clarification (I know it's also the resolution from the Windows API). I suggest to put it somewhere in the article or the documentation.
 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).

That's a bug. I shouldn't have done that. I didn't think about the alias being public when I did it. It has now been fixed on github.

Unfortunately, for some obscure reason, private module declarations are still placed into the overload set, so your fix does not help. You should either alias it to something unlikely to clash with something else or use fully qualified names (though this does not allow UFCS).
 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website

I don't know why it would be. It's probably a bug relating to the new StdDdoc versioning that we switched to (instead of using D_Ddoc which screwed up people who build the documentation at the same time as their code - a bad practice IMHO, but we decided to not set it up so that it didn't work when using Phobos). As WindowsTimeZone is only on Windows, it definitely had to be versioned with StdDdoc.

I just noticed that it shows up on http://www.d-programming-language.org/phobos/std_datetime.html, but not on http://www.digitalmars.com/d/2.0/phobos/std_datetime.html
 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)

Probably a StdDdoc problem as well.

also on d-programming-lanuage.org.

Well, Walter obviously isn't using the d-programming-language.org makefile to generate the documentation for www.digitalmars.com, since the style is completely different. I would guess that he didn't adjust what he was doing to use -version=StdDdoc. - Jonathan M Davis
May 15 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 05:26, Rainer Schuetze wrote:
 Jonathan M Davis wrote:
 On 2011-05-15 03:34, Rainer Schuetze wrote:
 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).

That's a bug. I shouldn't have done that. I didn't think about the alias being public when I did it. It has now been fixed on github.

Unfortunately, for some obscure reason, private module declarations are still placed into the overload set, so your fix does not help. You should either alias it to something unlikely to clash with something else or use fully qualified names (though this does not allow UFCS).

Bug reported: http://d.puremagic.com/issues/show_bug.cgi?id=6013 I also altered the private alias for indexOf so that it's using a name which won't conflict, so hopefully it won't cause any further problems, though I'd definitely prefer it if the bug were fixed and private alias worked properly. - Jonathan M Davis
May 15 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Fantastic work! I was lost in that std.datetime jungle for a while
now. It's a great read (your English is pretty fluent!).

Btw, I think there's a missing word in this sentence: "If what you're
doesn't need that extra boost of efficiency. (doing?).
May 16 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
 Fantastic work! I was lost in that std.datetime jungle for a while
 now. It's a great read (your English is pretty fluent!).

I would hope so. I'm a native of California. English is my native language. I'd be more worried about my French fluency (which is good but likely deteriorating since I haven't been to Europe recently) than my English fluency. And, of course, I'm not fluent in any other language, so I'd be complete rubbish at anything else.
 Btw, I think there's a missing word in this sentence: "If what you're
 doesn't need that extra boost of efficiency. (doing?).

Yes. It sounds like it's the word doing that's missing. I'll fix it this evening. - Jonathan M Daviss
May 16 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
 Jonathan M Davis wrote:
 On 2011-05-15 03:34, Rainer Schuetze wrote:
 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than
 
 writeln(timeLastModified(file).toSimpleString());
 
 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.

I'd need more data to have any idea what's going on, but it sounds like a bug.

I have investigated it a bit, and I'm currently thinking that Explorer, dir and Total Commander are wrong, and std.datetime is right. What I did to test: - create a new file, verify the dates printed are correct. fine in all programs - set the system time back to February - create the file again, verify the dates printed are correct. fine in all programs - set the system time back to May - now the displayed file time is the same when using std.datetime, but an hour later with all other programs. Maybe it's an XP or NTFS issue, but I would expect the file time not to change with the date it is displayed.

Well, Microsoft made the mistake of making it so that Windows boxes keep their time in local time (which they probably can't fix and maintain backwards compatability, so we're likely stuck), and that can have some interesting consequences, since that means that the system clock actually gets changed when DST begins or ends. I would have thought that the OS and file system would handle that well enough for Explorer to display the time correctly, but I don't know. Given how hard it can be to catch DST-related bugs and the fact that people don't generally examine the times on their files closely enough to verify that their times aren't 1 hour off 6 months later, it wouldn't entirely surprise me if there were bugs with regards to that on XP. But I still would have thought that they would have caught it. - Jonathan M Davis
May 16 2011