www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 4653] New: More unit test functions should be added - like assertEqual() and assertNotEqual()

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653

           Summary: More unit test functions should be added - like
                    assertEqual() and assertNotEqual()
           Product: D
           Version: D2
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: Phobos
        AssignedTo: nobody puremagic.com
        ReportedBy: jmdavisProg gmail.com


--- Comment #0 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-16
01:10:00 PDT ---
Most unit test frameworks seem to have more functions for test verification
than just assert. assert is nice and all, but it doesn't do a very good job of
telling you what went wrong. A good example would be if you're checking for
equality between two objects. Ideally, the test failure would indicate what the
expected result and actual result were, so that you know _what_ was wrong and
can start tracking it down. Otherwise, you have to go edit the test to tell
you, recompile, and run it again. Having a function like assertEqual() would
make the tests clearer and be much more useful when a test fails. There are a
whole host of such functions which could be added - assertNull(),
assertNotNull(), assertLessThan(), assertGreaterThan(), etc. I don't know how
many we really want to add to Phobos, but it seems to me that adding a few such
functions would help make clearer tests. Here are possible implementations for
assertEqual() and assertNotEqual():

void assertEqual(T, U)(in T actual, U expected, string msg = null, string file
= __FILE__, size_t line = __LINE__)
    if(__traits(compiles, actual != expected) && __traits(compiles,
format("%s", actual)))
{
    if(actual != expected)
    {
        if(msg.empty)
            throw new AssertError(format("assertEquals() failed: actual <%s>,
expected <%s>", actual, expected), file, line);
        else
            throw new AssertError(format("assertEquals() failed: actual <%s>,
expected <%s> : %s", actual, expected, msg), file, line);
    }
}

void assertNotEqual(T, U)(in T actual, U expected, string msg = null, string
file = __FILE__, size_t line = __LINE__)
    if(__traits(compiles, actual == expected) && __traits(compiles,
format("%s", actual)))
{
    if(actual == expected)
    {
        if(msg.empty)
            throw new AssertError(format("assertNotEquals() failed: value
<%s>", actual), file, line);
        else
            throw new AssertError(format("assertNotEquals() failed: value <%s>
: %s", actual, msg), file, line);
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 16 2010
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #1 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-16
02:09:45 PDT ---
Hmm. I forgot the in on the "expected" parameter. Oh well, it should be added.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 16 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653


Andrej Mitrovic <andrej.mitrovich gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrej.mitrovich gmail.com


--- Comment #2 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2010-08-16
07:00:16 PDT ---
AFAIK you could make a template that accepts a string literal (I think alias is
used for that), e.g. "x < y" or "x == y", which would maybe be a better idea
than to have duplicated code and a dozen "assertEqual, assertNotEqual" special
purpose functions.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 16 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #3 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-16
14:39:44 PDT ---
It's not necessarily a bad idea, but it doesn't work as well with the error
messages - and the improved error messages is the main reason for having these
sort of functions over normal assert. With a generic predicate, you'd have to
print out both the actual and expected every time when some functions would do
just fine with just the actual, while other functions might work better as not
being treated entirely as a boolean result. For instance, for assertLessThan()
could choose to use opCmp if available rather than < and then print out the
actual result of opCmp if it weren't < 0 rather that just printing the two
values and saying it failed. If you're assuming a generic predicate which takes
two values and returns a bool, then you can't do that.

So, I do think that having a unit testing function which takes a generic
predicate which takes two values and returns bool could be useful, I don't
think that it's ultimately as useful because the error messages aren't as good.
And honestly, the error messages are the reason that I think that these sort of
functions are worth adding to Phobos instead of just using normal assert.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 16 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #4 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2010-08-16
14:52:07 PDT ---
Oops, I didn't realize that you've posted some actual code up there. I'll have
a look to understand this better.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 16 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #5 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-17
23:08:49 PDT ---
Created an attachment (id=723)
Some useful unit testing functions.

Okay. I'm attaching some functions that I've already done which seem quite
useful - namely assertEqual(), assertNotEqual(), and assertOpCmp(). The first
two should be obvious, but assertOpCmp() takes a template parameter of "==",
"<", or ">" and tests that operation using opCmp(). The resulting assertion on
failure not only prints out the two values that you gave it to test, but it
also prints out whether the first was actually ==, <, or > the second. The
attached file also includes assertExcThrown() and assertExcNotThrown() from
http://d.puremagic.com/issues/show_bug.cgi?id=4644 , since I used them in the
unit tests for assertEqual(), assertNotEqual(), and assertOpCmp().

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 17 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653


Jonathan M Davis <jmdavisProg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #723 is|0                           |1
           obsolete|                            |


--- Comment #6 from Jonathan M Davis <jmdavisProg gmail.com> 2010-08-28
17:20:03 PDT ---
Created an attachment (id=742)
Some useful unit testing functions.

I've been using these functions heavily in my unit tests, and have them to be
_very_ useful. However, I found that using <> to enclose the toString() result
of the object in an error message causes confusion with opCmp. e.g.

assertOpCmp!("==")() failed: <1> < <2>


is too hard to read. So, I've changed it to use []. e.g.

assertOpCmp!("==")() failed: [1] < [2]


So, I'm posting the updated code.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 28 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653


Jonathan M Davis <jmdavisProg gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX


--- Comment #7 from Jonathan M Davis <jmdavisProg gmx.com> 2012-01-20 23:10:55
PST ---
For better or worse, this has been subsumed by issue# 5547.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 20 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #8 from Andrej Mitrovic <andrej.mitrovich gmail.com> 2013-02-25
14:48:41 PST ---
I don't know whether assertEqual and other others will ever get in
(assertThrown/assertNotThrown did), but the implementation can be seriously
simplified.

Here's an example of some assert functions (without a custom user message, but
that could be easily added):

import core.exception;
import std.string;

template assertOp(string op)
{
    void assertOp(T1, T2)(T1 lhs, T2 rhs, 
                          string file = __FILE__, 
                          size_t line = __LINE__)
    {
        string msg = format("(%s %s %s) failed.", lhs, op, rhs);

        mixin(format(q{
            if (!(lhs %s rhs)) throw new AssertError(msg, file, line);
        }, op));
    }
}

alias assertOp!"==" assertEqual;
alias assertOp!"!=" assertNotEqual;
alias assertOp!">" assertGreaterThan;
alias assertOp!">=" assertGreaterThanOrEqual;

unittest
{
    assertEqual(1, 1);
    assertNotEqual(1, 2);
    assertGreaterThan(2, 1);
    assertGreaterThanOrEqual(2, 2);
}

That's a huge save of line numbers. Admittedly both of us have learned a few
tricks since 2010. :)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=4653



--- Comment #9 from Jonathan M Davis <jmdavisProg gmx.com> 2013-02-25 15:16:30
PST ---
Cute. assertPred was the evolution of this, and it did a lot of nice stuff
(though it wouldn't surprise me it all if it could be improved), but it got
shot down as far as Phobos goes. std.datetime still uses a version of it
internally, but I'm slowing phasing it out. The problem is the template
overhead. I'm pretty sure that it's one of reasons (possibly _the_ reason) why
Walter can't build std.datetime's unit tests (due to running out of memory) on
his older FreeBSD machine (which he still complains about from time to time).

I still like it, but it _does_ instantiate a lot of templates. Ultimately, I
think that most people just use assert at this point and deal with it.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2013