www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - datetime review part 2

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
As some of you know, I've been working on a datetime module for possible 
inclusion in Phobos, and I posted the initial code for review on the Phobos
list 
about a week ago. I have updated the code per the suggestions in that thread. 
Andrei requested that I move the review to the main D list, so I'm posting my 
code here this time around. The code and html documentation can be found in
this 
compressed tar file here: http://is.gd/g6em1

It is intended to replace std.date and the nascent std.datetime. The code 
currently in std.datetime has already been integrated into my code.

My datetime module is modeled after Boost's date/time API but does not use any 
of its code. I've attempted to D-ify it in various ways (including making it
use 
ranges instead of iterators), but the API is still fairly close to Boost's. One 
of the big differences from Boost is that I removed the special time points 
(posifitive infinity, negative infinity, and invalid), since they complicated
the 
code considerably and didn't help at all with infinite ranges, since infinite 
ranges must be known to be infinite at compile time whereas Boost determines 
infinity based on the end points (which are only known at runtime). I also made 
it so the the interval and duration types are generic, so you don't have to
deal 
with different types of intervals and durations depending on the type of the
time 
point.

The major time point types are Date, TimeOfDay, DateTime, and SysTime. Date, 
TimeOfDay, and DateTime are optimized for calendar-based operations and have no 
concept of time zone, whereas SysTime is used for the system time and does have 
a time zone but is not optimized for calendar-based operations. SysTime holds 
the time in hecto-nanoseconds (100 ns) from midnight, January 1st 1 A.D. UTC 
which avoids all conversion errors due to DST (conversions to the time zone for 
the SysTime is done in the getters). All date types use the Proleptic Gregorian 
Calendar (so they use the Gregorian leap year calculations for their whole 
length) and use year 0 for 1 B.C. as per ISO 8601. Unlike Boost, they cover
from 
tens-of-thousands of years B.C. to tens-of-thousands of years A.D. (the exact 
range depending on the type) instead of just some subset of A.D. dates.

When using SysTime, you can ignore time zones if you wish (in most places, 
LocalTime is used if no TimeZone object is given), but you can also specificy 
which time zone to use. LocalTime implements local time, UTC implements UTC, 
SimpleTimeZone allows you to construct a time zone with an offset from UTC but
no 
DST, and PosixTimeZone and WindowsTimeZone allow you to select arbitrary time 
zones on posix and windows systems respectively (you can also use PosixTimeZone 
on Windows if you provide the time zone database files). Unlike my first 
submission, PosixTimeZone is now complete, but WindowsTimeZone is still just 
stubbed out.

Durations have been simplified since my previous submission. There are now only 
two duration types: Duration and TickDuration (TickDuration being used by
SHOO's 
stopwatch code and internally for getting the time, whereas Duration is used 
pretty much everywhere else). You can no longer have durations of months or 
years due to the issue of converting between months and smaller units (because 
months and years have a variable number of days). Rather, functions have been 
provided on the various types for handling months and years directly.

There are interval types (Interval, PosInfInterval, and NegInfInterval) for 
dealing with intervals (including positively and negatively infinite intervals) 
as well as corresponding range types. Ranges are constructed from an interval 
and range-generating function which generates each successive time point in a 
range (IRange and Interval's fwdRange() are good places to look for more
details 
an that), so they are extremely flexible.

I have included a text file, changes.txt, which contains a list of the majority 
of changes that I've made since I last posted my code, so if you looked at the 
first round of code, it should give you a good idea of what has been changed.

The source files are in Src, and the html doc files are in Doc. I attempted to 
make the doc files look more like Phobos this time around, so I used a slight 
variation on std.ddoc (so that it looked for the css files in the same 
directory), and have included the css files with them, so they should be much 
easier to read (though honestly, the link list at the top of the page is 
horrible for this module, since it makes no attempt to split out module 
functions from struct or class functions, and many of the struct and class 
functions have the same name and yet only one of them gets listed).

Hopefully the documentation is enough to properly understand the module, and of 
course, the source is there to examine as well. So, review away.

- Jonathan M Davis
Oct 17 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:

 Most recent code: http://is.gd/hO85C
 
 I added code for Mac OS X which should use its monotonic clock functions.
 Unfortunately, as I don't own a Mac, it's completed untested, but in theory,
 what's there should work, and someone with a Mac can fix it later if need be.

Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the definitions: extern(C) { struct mach_timebase_info_data_t { uint numer; uint denom; } alias mach_timebase_info_data_t* mach_timebase_info_t; kern_return_t mach_timebase_info(mach_timebase_info_t); ulong mach_absolute_time(); } I tested it with this small program: extern(C) void sleep(uint sec); void main() { auto start = TickDuration.currSystemTick; sleep(20); auto end = TickDuration.currSystemTick; writeln(to!string(end-start)); } And this is the result (more or less a few thousands on each run): TickDuration(20000046269) So it seems all good. One thing I wonder about... what is the expected behaviour if you put the computer to sleep in the middle of the above program? With mach_absolute_time, the clock stops counting while the computer is put to sleep. Does TickDuration.currSystemTick works like that on all systems? If there's a difference in behaviour perhaps it should be documented. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 26 2010
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
Jonathan M Davis Wrote:

 On Friday 26 November 2010 04:38:47 Michel Fortin wrote:

 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping. Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can.

You may have misunderstood. It's when the *computer* is put to sleep, not the thread. It's still an issue, but not as bad I think. -Steve
Nov 26 2010
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:

 On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
 
 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.

No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function. I understand my example using the sleep() function was misleading because the term sleep had two meanings. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 26 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-26 20:50:16 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:

 By the way, did you run the full unit tests on Mac OS X, or did you just test
 and fix the Mac-specific changes that I made? I assume that the unit 
 tests pass on
 Mac OS X, but I don't know.

The unit tests in unittests.d and time.d pass fine, but datetime.d does not compile: datetime.d(26977): Error: undefined identifier tzset datetime.d(27035): Error: template unittests.assertEqual(alias pred = "a == b",T,U) if (__traits(compiles,binaryFun!(pred)(actual,expected)) && isPrintable!(T) && isPrintable!(U)) does not match any function template declaration ... and the last error repeated a couple of time -- Michel Fortin michel.fortin michelf.com http://michelf.com/
Nov 27 2010
prev sibling parent Kagamin <spam here.lot> writes:
Jonathan M Davis Wrote:

 totally fries Clock.currAppTick(), which tells you how long the application
has 
 been running. The documentation for QueryPermanceCounter() and clock_gettime() 
 give no indication that the clock stops counting when a thread is sleeping.

I think, QueryPermanceCounter just queries processor's builtin counter (HPET). So it's hardware implementation defined. HPET is not present on older hardware, and future hardware can implement something different.
Nov 26 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
 On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:
 Most recent code: http://is.gd/hO85C
 
 I added code for Mac OS X which should use its monotonic clock functions.
 Unfortunately, as I don't own a Mac, it's completed untested, but in
 theory, what's there should work, and someone with a Mac can fix it
 later if need be.

Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the definitions: extern(C) { struct mach_timebase_info_data_t { uint numer; uint denom; } alias mach_timebase_info_data_t* mach_timebase_info_t; kern_return_t mach_timebase_info(mach_timebase_info_t); ulong mach_absolute_time(); } I tested it with this small program: extern(C) void sleep(uint sec); void main() { auto start = TickDuration.currSystemTick; sleep(20); auto end = TickDuration.currSystemTick; writeln(to!string(end-start)); } And this is the result (more or less a few thousands on each run): TickDuration(20000046269) So it seems all good.

Thanks! It's hard enough to test for Windows, but doing any Mac testing is essentially impossible for me.
 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping. Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can. - Jonathan M Davis
Nov 26 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 26 November 2010 05:59:58 Jonathan M Davis wrote:
 On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
 On 2010-11-26 03:03:15 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:
 Most recent code: http://is.gd/hO85C
 
 I added code for Mac OS X which should use its monotonic clock
 functions. Unfortunately, as I don't own a Mac, it's completed
 untested, but in theory, what's there should work, and someone with a
 Mac can fix it later if need be.

Just tested it. There's an error in the definition of mach_timebase_info (takes a mach_timebase_info_t* argument, but it should be mach_timebase_info_t, no pointer) and the mach_timebase_info_t alias is missing. Here's a fixed version of the definitions: extern(C) { struct mach_timebase_info_data_t { uint numer; uint denom; } alias mach_timebase_info_data_t* mach_timebase_info_t; kern_return_t mach_timebase_info(mach_timebase_info_t); ulong mach_absolute_time(); } I tested it with this small program: extern(C) void sleep(uint sec); void main() { auto start = TickDuration.currSystemTick; sleep(20); auto end = TickDuration.currSystemTick; writeln(to!string(end-start)); } And this is the result (more or less a few thousands on each run): TickDuration(20000046269) So it seems all good.

Thanks! It's hard enough to test for Windows, but doing any Mac testing is essentially impossible for me.
 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping. Do you know if there's another way to get a monotonic clock on the Mac - one which isn't affected by sleep? Much as I'd hate to do it, it would be better to use gettimeofday() and forgo the monotonic clock then have a monotonic clock which is affected by sleep. But I'd _really_ like to have a monotonic clock if I can.

This page ( http://is.gd/hP6Zr ) claims that mach_absolute_time() is not affected by sleeping. - Jonathan M Davis
Nov 26 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 26 November 2010 06:14:29 Michel Fortin wrote:
 On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:
 On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.

No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function. I understand my example using the sleep() function was misleading because the term sleep had two meanings.

Oh. Well that isn't the same magnitude of problem at all. I wouldn't have thought that that the computer going to sleep/hibernating to RAM would cause that sort of problem, but I guess that it could. I have no idea if Windows or Linux would be affected in the same way. They might be. I don't have any way to test it. Regardless, it's still a bit problematic, but acceptable I think. If it could be verified whether Linux or Windows have the same problem, that would be good. It probably should be put in the documentation. It really never would have occurred to me that putting the computer to sleep/hibernating to RAM would affect the clock. Of course, since I don't have a working laptop (and I've hated having it go to sleep when I've had one), I certainly wouldn't run into it personally. - Jonathan M Davis
Nov 26 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday 26 November 2010 06:14:29 Michel Fortin wrote:
 On 2010-11-26 08:59:58 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:
 On Friday 26 November 2010 04:38:47 Michel Fortin wrote:
 One thing I wonder about... what is the expected behaviour if you put
 the computer to sleep in the middle of the above program? With
 mach_absolute_time, the clock stops counting while the computer is put
 to sleep. Does TickDuration.currSystemTick works like that on all
 systems? If there's a difference in behaviour perhaps it should be
 documented.

Really? That is... not good. Unacceptable in fact. Even if it were acceptable when dealing with stop watch and benchmarking code (which is questionable), it totally fries Clock.currAppTick(), which tells you how long the application has been running. The documentation for QueryPermanceCounter() and clock_gettime() give no indication that the clock stops counting when a thread is sleeping.

No no no... not when a *thread* is sleeping. When the *computer* is sleeping; when the OS is suspended while you close the lid on a laptop for instance. So I start my program, it enters the sleep() function, I close the lid, and one minute later I reopen it and when the sleep() function returns the tick duration is as if the clock was stopped while the lid was closed, despite that the real elapsed time was far much. Closing the lid puts the *computer* to sleep and stops the clock, not the sleap() function. I understand my example using the sleep() function was misleading because the term sleep had two meanings.

By the way, did you run the full unit tests on Mac OS X, or did you just test and fix the Mac-specific changes that I made? I assume that the unit tests pass on Mac OS X, but I don't know. - Jonathan M Davis
Nov 26 2010
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday 27 November 2010 04:35:58 Michel Fortin wrote:
 On 2010-11-26 20:50:16 -0500, Jonathan M Davis <jmdavisProg gmx.com> said:
 By the way, did you run the full unit tests on Mac OS X, or did you just
 test and fix the Mac-specific changes that I made? I assume that the
 unit tests pass on
 Mac OS X, but I don't know.

The unit tests in unittests.d and time.d pass fine, but datetime.d does not compile: datetime.d(26977): Error: undefined identifier tzset datetime.d(27035): Error: template unittests.assertEqual(alias pred = "a == b",T,U) if (__traits(compiles,binaryFun!(pred)(actual,expected)) && isPrintable!(T) && isPrintable!(U)) does not match any function template declaration ... and the last error repeated a couple of time

It looks like that's a bug in druntime. For some reason, it doesn't list tzset() for OSX in core.stdc.time. But according to this man page ( http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/10 5/man3/tzset.3.html ), it does exist on Mac OS X. - Jonathan M Davis
Nov 27 2010