www.digitalmars.com         C & C++   DMDScript  

D - Assertions, unit testing

reply Mike Swieton <mike swieton.net> writes:
I really like the built in unit testing in D, but I think that there are
points that could be improved upon.

In no particular order:

If assert were a method, then I could override it and extend it. This would
allow us to add, easily, things like assertEquals and the usual crew there,
and also custom ones, such as (for example) assertConnected, or better error
reporting or improved messages.

I see no way to do this now: I could do things like wrap assert with another
method, but then the wrong file/line number will be reported.

Anotehr, and rather minor, gripe is this: there is no way to really name
tests. I know I can just have a name in a comment before the block, but that
really isn't the same thing. Nitpicking, really, but I feel it ought to be
there.

Basically, I think D's unit tests are not powerful enough. If it were in a
library, I'd say, OK, write a new library, but as they are part of the
standard, and one of the defining features of the language, I think it's
important to do it well. Things like more asserts, with messages are
significant. Another large omission is the setup/teardown methods. Also, there
is no way at all, that I can see, to hook this into a reporting framework.

I think its important that these things be there, even though I think it will
necessitate some language changes. Thoughts on this?

Mike Swieton
__
...Unix, MS-DOS, and Windows NT (also known as the Good, the Bad, and the
Ugly)
	- Matt Welsh
Mar 23 2004
next sibling parent reply Manfred Nowak <svv1999 hotmail.com> writes:
Mike Swieton wrote:

[...]
 I could do things like wrap assert with another
 method, but then the wrong file/line number will be reported.

To obtain filename and line number la myassert( 0, std.compiler.currLine) would suffice. So long!
Mar 23 2004
parent reply Mike Swieton <mike swieton.net> writes:
On Wed, 24 Mar 2004 08:53:23 +0100, Manfred Nowak wrote:

 Mike Swieton wrote:

 To obtain filename and line number  la
 
   myassert( 0, std.compiler.currLine)
 
 would suffice.

Did not know that, that's helpful. But it still isn't quite there. Here's the thing: I don't want to type std.compiler.currLine each time. And D doesn't have macros, does it? In C I'd just macro-out the real assert. Mike Swieton __ I've developed a new philosophy. I only dread one day at a time. - Charlie Brown
Mar 24 2004
parent Manfred Nowak <svv1999 hotmail.com> writes:
Mike Swieton wrote:

[...]
 But it still isn't quite there.

Right. It is a suggestion. [...]
 I don't want to type

Use a macro editor.
 And D doesn't have macros, does it?

Right. No macros for good reasons, but aliases and debug statements. [...] So long!
Mar 24 2004
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle4 juno.com> writes:
"Mike Swieton" <mike swieton.net> wrote in message
news:pan.2004.03.24.06.10.37.841395 swieton.net...
 I really like the built in unit testing in D, but I think that there are
 points that could be improved upon.

 In no particular order:

 If assert were a method, then I could override it and extend it. This

 allow us to add, easily, things like assertEquals and the usual crew

 and also custom ones, such as (for example) assertConnected, or better

 reporting or improved messages.

 I see no way to do this now: I could do things like wrap assert with

 method, but then the wrong file/line number will be reported.

It's possible to suplement the built-in unittesting/assert with custom testing tools. For instance make a subclass of object that all your classes subclass that have the methods you want and then call those instead of assert. If it is generic enough and robust enough and useful enough it could make it into phobos or something like that.
 Anotehr, and rather minor, gripe is this: there is no way to really name
 tests. I know I can just have a name in a comment before the block, but

 really isn't the same thing. Nitpicking, really, but I feel it ought to be
 there.

I haven't tried it and maybe it already exists somewhere but is it possible to write an "assertStr" that throws a custom object? See the file std.asserterror.d for how the built-in assert is implemented.
 Basically, I think D's unit tests are not powerful enough. If it were in a
 library, I'd say, OK, write a new library, but as they are part of the
 standard, and one of the defining features of the language, I think it's
 important to do it well. Things like more asserts, with messages are
 significant. Another large omission is the setup/teardown methods. Also,

 is no way at all, that I can see, to hook this into a reporting framework.

I assume Walter wanted to build in basic unit testing and leave "full fledged" unit testing to a library (which doesn't exist yet). Since there are endless variations and features to add to testing infrastructure it would be impossible to get it totally "right" in the core language.
 I think its important that these things be there, even though I think it

 necessitate some language changes. Thoughts on this?

 Mike Swieton
 __
 ...Unix, MS-DOS, and Windows NT (also known as the Good, the Bad, and the
 Ugly)
 - Matt Welsh

Mar 24 2004
parent reply Mike Swieton <mike swieton.net> writes:
On Wed, 24 Mar 2004 14:57:44 -0500, Ben Hinkle wrote:
 I assume Walter wanted to build in basic unit testing and leave "full
 fledged" unit testing to a library (which doesn't exist yet). Since there
 are endless variations and features to add to testing infrastructure it
 would be impossible to get it totally "right" in the core language.

I think that that is the right way to do it. What I am trying to say here is that I don't think that what is exposed is very extendable, for example what I mentioned about assert not being a real function (generated at compile-time as far as I can tell). What I'm asking is this: is it possible to add features to the unit testing framework, such as reporting, suites, messages on asserts, other kinds of asserts, and so on, and still use the built-in unit test framework? These other features could be a seperate library (and that's where they belong, in fact). But: can it be done on top of the built-in one? It's been suggested that I use a macro editor (external tool; the toolkit/language/framework/whatever should do that for me), or that I write a class with custom asserts and inherit from it (ditto), and so on. But this is making the user implement the test framework, when a framework should already exist. I'm not trying to argue that the language be changed; I'd be delighted to find out that everything I am talking about can be built on top of the existing facilities. But it can't really be said that D has unit testing, if I have to throw out everything it provides if I want to get half the features that every other framework for any other language has. Please, correct me if I'm wrong, point me to the features that could be used to write such a framework. But if the above is correct, it ought to be addressed. Mike Swieton __ Computer Science is the only discipline in which we view adding a new wing to a building as being maintenance. - Jim Horning
Mar 24 2004
parent Ben Hinkle <bhinkle4 juno.com> writes:
 But it can't really be said that D has unit testing, if I have to throw out
 everything it provides if I want to get half the features that every other
 framework for any other language has.

I agree the assert expression is weak if you want to get line numbers *and* user messages/data. But saying the whole thing needs to be tossed is stating the case too strongly. I just played around for about 15 minutes or so and cooked up the following "test infrastructure" on top of the existing functionality. The ugly part of the user code is having to wrap the asserts in try catch blocks and rethrow. But is that ugly enough to throw out the whole thing? That's a style question that each person can choose for themselves. I certainly wouldn't mind seeing a more flexible assert and AssertError. One way you can help your plight is to propose something. What would make D extensible enough? Anyway, the code (compile with -unittest flag): import std.asserterror; // put this in some shared module struct MyTestHarness { char [] fMsg; // current test message, if any bit myTest(int a, int b, char[] msg) { // perform logging, report generation, etc. fMsg = msg; return a==b; } } class MyError { this(MyTestHarness harness, AssertError err) { fErr = err; fHarness = harness; } MyTestHarness fHarness; AssertError fErr; void print() { // perform logging, report generation, etc. printf("%.*s\n", fHarness.fMsg); fErr.print(); } } // this is user code unittest { MyTestHarness harness; try { assert(harness.myTest(1,1,"just checking if 1==1")); assert(harness.myTest(1,2,"just checking if 1==2")); } catch (AssertError e) { throw new MyError(harness,e); } } int main() { return 0; } Save it to test.d, run it and it prints Error: just checking if 1==2 AssertError Failure test(37)
Mar 24 2004
prev sibling parent J C Calvarese <jcc7 cox.net> writes:
Mike Swieton wrote:
 I really like the built in unit testing in D, but I think that there are
 points that could be improved upon.
 

 Anotehr, and rather minor, gripe is this: there is no way to really name
 tests. I know I can just have a name in a comment before the block, but that
 really isn't the same thing. Nitpicking, really, but I feel it ought to be
 there.

This probably won't satisfy you, but you can use nested functions to add some structure to unittests: void main() { printf("The actual program.\n"); } unittest { void strings() { assert("a" ~ "b" == "ab"); assert("a" < "b"); } void numbers() { assert(1 + 1 == 2); assert(10 * 10 == 100); } printf("Start unittests...\n"); strings(); numbers(); printf("Unittests successful.\n"); } Compile with: echo off dmd nested.d -unittest nested.exe pause -- Justin http://jcc_7.tripod.com/d/
Mar 24 2004