www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 2917] New: std.date fails for all years before 1970

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917

           Summary: std.date fails for all years before 1970
           Product: D
           Version: 2.029
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: ghaecker idworld.net


For years prior to 1970, conversion from string to d_time returns d_time_nan. 
Strings are parsed correctly, but conversion to d_time fails the test at line
440 of date.d in the makeDay() function.

Commenting out lines 440 through 445 allows a d_time to be returned, but the
value is skewed.  Line 447 works for years >= 1970:

    return day(t) + date - 1;

but for years < 1970, this adjustment:

    return day(t) + date;

returns the expected value.

There is also another problem: attempting to convert a negative d_time back to
a string requires adding ticksPerSecond to d_time to return the original value
from secFromTime(), which, of course, trashes the ms value.  Adding the value
of  ticksPerSecond appears to work for all negative d_time values, except for
-1000.  

NOTE: I was only working with a resolution of seconds, so the effect on
milliseconds/ticks was not taken into account.

I'm new to D, so please don't be too critical of my code.  I'm including it so
you can see the kludges I added to work around these issues.  BTW, an
addMonths() function (more elegant than mine) would be a welcome addition to
std.date.  My attempt at writing one along with a unittest section was what
revealed this bug.  Unittest is a brilliant addition to the language.

import std.stdio,
    std.date,
    std.conv,
    std.random;

int main(char[][] args)
{
    bool pass;
    int adj;
    string tscmp;
    d_time tt, rtt;

    string[] testdates = [
        "1969-12-31 23:59:58",
        "1969-12-31 23:59:59",
        "1970-01-01 00:00:00",
        "1970-01-01 00:00:01",
        "1970-01-01 00:00:02"
        ];
    writefln("Crossing the 1970 epoch...\n");
    foreach(int i, string s; testdates) {
        tt = dbgParse(s ~ " GMT");
        // KLUDGE
        adj = (i < 2) ? ticksPerSecond - 5: 0;
        rtt = tt + adj;
        tscmp = toDbDateTime(rtt);
        pass = (tscmp == s);
        writefln("%s -> %5d", s, tt);
        writefln("%s <- %5d   %s\n", tscmp, rtt, pass);
    }

    // Random tests
    auto gen = Mt19937(unpredictableSeed);
    int yr, mh, dy, hr, me, sd, days;
    int yr2, mh2, dy2, hr2, me2, sd2;
    int tested, passed, before1970, show = 7;
    string ts;
    bool first;
    for (int i; i < 10000; i++) {
        tested++;
        yr = uniform(1601, 2400, gen);
        if (yr < 1970) before1970++;
        mh = uniform(1, 12, gen);
        days = daysInMonth(yr, mh);
        dy = uniform(1, days, gen);
        hr = uniform(0, 23, gen);
        me = uniform(0, 59, gen);
        sd = uniform(0, 59, gen);
        ts = std.string.format("%d-%02d-%02d %02d:%02d:%02d", yr, mh, dy, hr,
me, sd);
        tt = dbgParse(ts ~ " GMT");
        // KLUDGE
        adj = (yr < 1970) ? ticksPerSecond - 5: 0;
        rtt = tt + adj;
        tscmp = toDbDateTime(rtt);
        pass = (ts == tscmp);
        if (! pass || show > 0) {
            if (! first) {
                first = true;
                writefln("Showing first %d random samples...\n", show);
            }
            show--;
            writefln("%s -> %16d", ts, tt);
            writefln("%s <- %16d   %s\n", tscmp, rtt, pass);
        }
        if (pass) passed++;
    }

    writefln("");
    writefln("Tested: %7d", tested);
    writefln("< 1970: %7d", before1970);
    writefln("Passed: %7d", passed);

    return 0;
}

string toDbDateTime(d_time t) {
    auto yr = yearFromTime(t);
    auto mh = monthFromTime(t) + 1;
    auto dy = dateFromTime(t);
    auto hr = hourFromTime(t);
    auto me = minFromTime(t);
    auto sd = secFromTime(t);
    return std.string.format("%d-%02d-%02d %02d:%02d:%02d", yr, mh, dy, hr, me,
sd);
}

d_time dbgParse(string s)
{
    try
    {
        Date dp;
        dp.parse(s);
        auto time = makeTime(dp.hour, dp.minute, dp.second, dp.ms);
        if (dp.tzcorrection == int.min)
            time -= localTZA;
        else
        {
            time += cast(d_time)(dp.tzcorrection / 100) * msPerHour +
                    cast(d_time)(dp.tzcorrection % 100) * msPerMinute;
        }
        auto day = dbgMakeDay(dp.year, dp.month - 1, dp.day);
        d_time result = makeDate(day,time);
        return timeClip(result);
    }
    catch
    {
        return d_time_nan;                // erroneous date string
    }
}

d_time dbgMakeDay(d_time year, d_time month, d_time date)
{

    int[12] mdays =
        [ 0,31,59,90,120,151,181,212,243,273,304,334 ];
    const y = cast(int)(year + floor(month, 12));
    const m = dmod(month, 12);

    const leap = leapYear(y);
    auto t = timeFromYear(y) + cast(d_time) mdays[m] * msPerDay;
    if (leap && month >= 2)
        t += msPerDay;

    /* DISABLED: this test failes for y < 1970

    if (yearFromTime(t) != y ||
        monthFromTime(t) != m ||
        dateFromTime(t) != 1)
    {
        writefln("Bad match");
        writefln("%d : %d, %d : %d, %d : %d", y, yearFromTime(t), m,
monthFromTime(t), 1, dateFromTime(t));

        return  d_time_nan;
    }

    */

    // KLUDGE
    int adj = (y < 1970) ? 0 : 1;

    return day(t) + date - adj;

}

d_time addMonths(d_time dt, int months) {
    int days = 0;
    if (months <> 0) {
        int mon = monthFromTime(dt); // zero-based month
        int yr = yearFromTime(dt);
        while (months > 0) {
            days += daysInMonth(yr, mon + 1); // one-based month
            mon++;
            if (mon > 11) {
                mon = 0;
                yr++;
            }
            months--;
        }
        while (months < 0) {
            mon--;
            if (mon < 0) {
                mon = 11;
                yr--;
            }
            days -= daysInMonth(yr, mon + 1);  // one-based month
            months++;
        }
    }
    return dt += days * cast(d_time) ticksPerDay;
}

unittest {

    d_time dt1, dt2, dt3, dt4;
    dt1 = dbgParse("1979-10-31 13:14:15 GMT");
    dt2 = dbgParse("1981-05-31 13:14:15 GMT");
    dt3 = addMonths(dt1, 19);
    assert(dt3 == dt2, text(dt3, " != ", dt2));
    dt4 = addMonths(dt3, -19);
    assert(dt4 == dt1, text(dt4, " != ", dt1));

    dt1 = dbgParse("1969-10-31 13:14:15 GMT");
    dt2 = dbgParse("1971-05-31 13:14:15 GMT");
    dt3 = addMonths(dt1, 19);
    assert(dt3 == dt2, text(dt3, " != ", dt2));
    dt4 = addMonths(dt3, -19);
    assert(dt4 == dt1, text(dt4, " != ", dt1));

    dt1 = dbgParse("1599-10-31 13:14:15 GMT");
    dt2 = dbgParse("1601-05-31 13:14:15 GMT");
    dt3 = addMonths(dt1, 19);
    assert(dt3 == dt2, text(dt3, " != ", dt2));
    dt4 = addMonths(dt3, -19);
    assert(dt4 == dt1, text(dt4, " != ", dt1));

    dt1 = dbgParse("2399-10-31 13:14:15 GMT");
    dt2 = dbgParse("2401-05-31 13:14:15 GMT");
    dt3 = addMonths(dt1, 19);
    assert(dt3 == dt2, text(dt3, " != ", dt2));
    dt4 = addMonths(dt3, -19);
    assert(dt4 == dt1, text(dt4, " != ", dt1));

}


-- 
Apr 30 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917





------- Comment #1 from ghaecker idworld.net  2009-05-04 05:39 -------
Created an attachment (id=348)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=348&action=view)
patch for std.date.d v. 2.029 fixes issues with negative time values

Rather than whine about the date issues, I decided to lend a hand and address
the issues myself.  If you find the modifications useful you are welcome to use
the code in any way you see fit.  I only added my name as a modifier so that
Walter won't get blamed for any mistakes I made.  You may remove my name or
leave it, as you see fit.  The changes I made aren't anything special; just
some grunt work that needed doing.

I tinkered with some of the more vital functions in this module.  In
particular, I modified floor(), dmod(), day(), and dayFromYear().  The latter
got a little messy with year value 0 and lower.  Of course, pushing the
Gregorian calendar back that far back is purely hypothetical.  I added unittest
sections to day() and dayFromYear, as the proper working of these functions is
vital to the entire module.

I threw in a bit of trivial tidying up as well.  The calls to toInteger() and
timeClip() were removed, since they don't do anything.  I left the functions in
place, in case some existing code calls them.  I changed the name of the 'day'
variable that called the day() function, as it seems like a bad idea to
duplicate the name in the same scope.

I added an overload for dateFromTime(d_time t, int months).  In most cases,
where it's used in the module, the value for month is already known, so there's
no need for dateFromTime() to recalculate it.

My addition of toDbDateTime() is trivial, but I found it useful in testing, and
it has practical value.  It's a fairly common format, and it doesn't add much
weight to the module.

I also added two other functions: addMonths() and addYears().  These are both
common date/time-manipulation tasks, which aren't as straightforward as adding
fixed values like days or weeks.  addMonths() is rather clunky, but it gets the
job done.  It never cranks through more than 11 months on any call.  Beyond 11,
it calls addYears() so that adding 60 months is almost as quick as adding 5
years.

I've rebuilt the 2.029 library with this patch to std.date on linux.  All
appears to be working well.  Someone needs to test the DosDate functions to
make sure my changes didn't break them.  There were no unit tests for them.  I
don't have D installed on a Windows box.

Thanks in advance for considering these changes.


-- 
May 04 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917





--- Comment #2 from Glenn Haecker <ghaecker idworld.net>  2009-05-14 16:20:06
PDT ---
Created an attachment (id=366)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=366)
patch for std.date.d v. 2.030 fixes issues with negative time values

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
May 14 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917





--- Comment #3 from Glenn Haecker <ghaecker idworld.net>  2009-05-14 17:06:15
PDT ---
This bit of code shows the bug.  There are larger effects as well.

-------------------------------------------
    string[] testdt = [
        "1969-12-31 23:59:58 UTC",
        "1969-12-31 23:59:59 UTC",
        "1970-01-01 00:00:00 UTC", // epoch
        "1970-01-01 00:00:01 UTC",
        "1970-01-01 00:00:02 UTC"];

    foreach(int i, string s; testdt)
        writefln("%s = %5d", s, parse(s));

    d_time[] times = [-2000,-1000,0,1000,2000];
    foreach(tm; times)
        writefln("%5d = %s", tm, toUTCString(tm));
-------------------------------------------

Unpatched results:

1969-12-31 23:59:58 UTC = -9223372036854775808 (d_time_nan)
1969-12-31 23:59:59 UTC = -9223372036854775808 (d_time_nan)
1970-01-01 00:00:00 UTC =     0
1970-01-01 00:00:01 UTC =  1000
1970-01-01 00:00:02 UTC =  2000
-2000 = Wed, 31 Dec 1969 23:59:57 UTC  (off by 1 sec)
-1000 = Wed, 31 Dec 1969 23:59:58 UTC  (off by 1 sec)
    0 = Thu, 01 Jan 1970 00:00:00 UTC
 1000 = Thu, 01 Jan 1970 00:00:01 UTC
 2000 = Thu, 01 Jan 1970 00:00:02 UTC

Patched results:

1969-12-31 23:59:58 UTC = -2000
1969-12-31 23:59:59 UTC = -1000
1970-01-01 00:00:00 UTC =     0
1970-01-01 00:00:01 UTC =  1000
1970-01-01 00:00:02 UTC =  2000
-2000 = Wed, 31 Dec 1969 23:59:58 UTC
-1000 = Wed, 31 Dec 1969 23:59:59 UTC
    0 = Thu, 01 Jan 1970 00:00:00 UTC
 1000 = Thu, 01 Jan 1970 00:00:01 UTC
 2000 = Thu, 01 Jan 1970 00:00:02 UTC

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
May 14 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917



--- Comment #4 from Glenn Haecker <ghaecker idworld.net> 2010-01-25 02:59:44
PST ---
Created an attachment (id=555)
patch for std.date.d v.2.039 fixes issues with negative time values

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 25 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917



--- Comment #5 from Glenn Haecker <ghaecker idworld.net> 2010-01-25 03:12:00
PST ---
Created an attachment (id=556)
date_assert.d contains assertions for date.d v2.039 patch

date_assert.d contains test cases, both to demonstrate what's wrong with
std.date and to validate modifications by the supplied patch.

Please note that this file and the provided patch were created on Linux and
contain UNIX-style line endings.

In order to compile date_assert.d with v2.039 phobos before the patch is
applied, you'll need to comment out lines 68 through 84.  These lines contain
tests that call addMonths(), which is not available before the patch is
applied.

You can search for "fails w/o patch" to identify the assert() tests that fail
with out the patch.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 25 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917


downs <default_357-line yahoo.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |default_357-line yahoo.de


--- Comment #6 from downs <default_357-line yahoo.de> 2010-05-22 11:07:48 PDT
---
This bug report is symptomatic for what's wrong with the D development process,
in my opinion. There's a bug, there's a fix, there's a patch, there's a
distinct lack of feedback from Walter/Andrei. No wonder so few people work on
D; a situation like this would drive any contributor away.

My apologies to Mr. Haecker.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
May 22 2010
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=2917


Lars T. Kyllingstad <bugzilla kyllingen.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla kyllingen.net
         Resolution|                            |WONTFIX


--- Comment #7 from Lars T. Kyllingstad <bugzilla kyllingen.net> 2011-02-03
00:13:24 PST ---
std.date has now been superseded by std.datetime.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 03 2011