www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - unittest vs exceptions?

reply "Maxime Chevalier" <maximechevalierb gmail.com> writes:
I'd like to write some unit tests to check my software. Writing 
unittest blocks and putting assert statements in there seems easy 
enough, but I noticed that if the code I run in there throws an 
exception, I don't get the unit test failed message.

How does unittest check for success/failure? Does assert throw a 
special kind of error class when it fails? How would you check 
that some code didn't throw an exception, or did throw an 
exception?
Aug 04 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:
 I'd like to write some unit tests to check my software. Writing
 unittest blocks and putting assert statements in there seems easy
 enough, but I noticed that if the code I run in there throws an
 exception, I don't get the unit test failed message.
 
 How does unittest check for success/failure? Does assert throw a
 special kind of error class when it fails? How would you check
 that some code didn't throw an exception, or did throw an
 exception?

assert throws an AssertError. The unittest failed message is its message, so if it's never thrown, then you won't get that message. And if an exception is thrown, then the assertion won't fail, so no AssertError will be thrown, just the exception that your function threw. - Jonathan M Davis
Aug 04 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 22:09:03 UTC, Jonathan M Davis 
wrote:
 On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:
 How does unittest check for success/failure? Does assert throw 
 a
 special kind of error class when it fails? How would you check
 that some code didn't throw an exception, or did throw an
 exception?

assert throws an AssertError. The unittest failed message is its message, so if it's never thrown, then you won't get that message. And if an exception is thrown, then the assertion won't fail, so no AssertError will be thrown, just the exception that your function threw.

A failed unit test really just means that a unit test block is left via an exception. Currently, it just bubbles up to the druntime main(), where it is printed to console and causes a non-zero exit code, but future unit test frameworks could handle this in a more advanced way. David
Aug 04 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 00:15:49 David Nadlinger wrote:
 A failed unit test really just means that a unit test block is
 left via an exception. Currently, it just bubbles up to the
 druntime main(), where it is printed to console and causes a
 non-zero exit code, but future unit test frameworks could handle
 this in a more advanced way.

They could, but I think that it's pretty normal to just treat any thrown exception as a test failure. The only difference here and what I've seen in other test frameworks, is that most test frameworks would actually tell you what test failed, which really doesn't work in D, since unittest blocks are unnamed (whereas most unit test frameworks used named tests). That's one of the reasons why I'd love it if we could start naming unittest blocks, but in spite of some interest in such a feature, no one has implemented it yet. - Jonathan M Davis
Aug 04 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-08-05 01:43, Maxime Chevalier wrote:
 These improvements would be very nice. The unit test framework, as it
 is, is rather underpowered.

 Exceptions could also use more documentation on the D website. I heard
 there was some exception chaining mechanism, but I can't even seem to
 find any info on the Error class.

I have a simple, minimal unit test framework as part of my serialization library: https://github.com/jacob-carlborg/orange/blob/master/orange/test/UnitTester.d -- /Jacob Carlborg
Aug 05 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 22:15:53 UTC, David Nadlinger wrote:
 On Saturday, 4 August 2012 at 22:09:03 UTC, Jonathan M Davis 
 wrote:
 On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:
 How does unittest check for success/failure? Does assert 
 throw a special kind of error class when it fails? How would 
 you check that some code didn't throw an exception, or did 
 throw an exception?



 assert throws an AssertError. The unittest failed message is 
 its message, so if it's never thrown, then you won't get that 
 message. And if an exception is thrown, then the assertion 
 won't fail, so no AssertError will be thrown, just the 
 exception that your function threw.


 A failed unit test really just means that a unit test block is 
 left via an exception. Currently, it just bubbles up to the 
 druntime main(), where it is printed to console and causes a 
 non-zero exit code, but future unit test frameworks could 
 handle this in a more advanced way.

Not if it's static asserts :) Unless I'm wrong, I've recently used a few static asserts to ensure certain combinations of code don't compile (on purpose). This is important for proving the logic works the way it is suppose to. Sorta following TDPL example of is(typof() == bool) (pg 141): const int a = 42; static assert(is(typeof(a = 24) == bool) == false, "constness from 'a' not honored."); Although it seems like a small silly example, but when you include something generated for a mixin or template functions it becomes a very different story. Plus you'd get your warnings and errors during compiling and not runtime (find bugs faster?)
Aug 04 2012
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 4 August 2012 at 22:26:26 UTC, Jonathan M Davis 
wrote:
 On Sunday, August 05, 2012 00:15:49 David Nadlinger wrote:
 A failed unit test really just means that a unit test block is
 left via an exception. Currently, it just bubbles up to the
 druntime main(), where it is printed to console and causes a
 non-zero exit code, but future unit test frameworks could 
 handle
 this in a more advanced way.

They could, but I think that it's pretty normal to just treat any thrown exception as a test failure. The only difference here and what I've seen in other test frameworks, is that most test frameworks would actually tell you what test failed, which really doesn't work in D, since unittest blocks are unnamed (whereas most unit test frameworks used named tests). That's one of the reasons why I'd love it if we could start naming unittest blocks, but in spite of some interest in such a feature, no one has implemented it yet. - Jonathan M Davis

Aug 04 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 01:01:03 Tobias Pankrath wrote:
 you can do it as a library solution just fine.

You shouldn't have to use a library to get named unit tests, since unit testing is built into D, and the lack of name unit tests is a real problem for stack traces and the like. It was suggested at one point that at minimum, unittest block functions have their line number in their name so that you can figure out which unittest block you're dealing with even if you can't name it yourself ( http://d.puremagic.com/issues/show_bug.cgi?id=5587 ), and not even that's been done yet. Though now that I read through that enhancement request again, it looks like theres an old pull request with the necessary changes in it ( https://github.com/D-Programming-Language/dmd/pull/264 ), but it's one of those that's been sitting around for a long time, probably because it's an enhancement rather than a bug fix, and Walter just hasn't gotten around to it. - Jonathan M Davis
Aug 04 2012
prev sibling next sibling parent "Maxime Chevalier" <maximechevalierb gmail.com> writes:
These improvements would be very nice. The unit test framework, 
as it is, is rather underpowered.

Exceptions could also use more documentation on the D website. I 
heard there was some exception chaining mechanism, but I can't 
even seem to find any info on the Error class.

 You shouldn't have to use a library to get named unit tests, 
 since unit
 testing is built into D, and the lack of name unit tests is a 
 real problem for
 stack traces and the like. It was suggested at one point that 
 at minimum,
 unittest block functions have their line number in their name 
 so that you can
 figure out which unittest block you're dealing with even if you 
 can't name it
 yourself  ( http://d.puremagic.com/issues/show_bug.cgi?id=5587 
 ), and not even
 that's been done yet. Though now that I read through that 
 enhancement request
 again, it looks like theres an old pull request with the 
 necessary changes in
 it ( https://github.com/D-Programming-Language/dmd/pull/264 ), 
 but it's one of
 those that's been sitting around for a long time, probably 
 because it's an
 enhancement rather than a bug fix, and Walter just hasn't 
 gotten around to it.

 - Jonathan M Davis

Aug 04 2012
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 01:43:55 Maxime Chevalier wrote:
 These improvements would be very nice. The unit test framework,
 as it is, is rather underpowered.
 
 Exceptions could also use more documentation on the D website. I
 heard there was some exception chaining mechanism, but I can't
 even seem to find any info on the Error class.

It's in object_.d in druntime (which has the definition for Object in it), but like much of druntime, it's not documented in the API docs. However, there is some information about them here: http://dlang.org/errors.html TDPL (The D Programming Language by Andrei Alexandrescu) tends to cover a lot of those sort of things better than the online docs do. The online docs definitely could use some improvements. - Jonathan M Davis
Aug 04 2012