www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - unittest vs exceptions?

reply "Maxime Chevalier" <maximechevalierb gmail.com> writes:
I'd like to write some unit tests to check my software. Writing 
unittest blocks and putting assert statements in there seems easy 
enough, but I noticed that if the code I run in there throws an 
exception, I don't get the unit test failed message.

How does unittest check for success/failure? Does assert throw a 
special kind of error class when it fails? How would you check 
that some code didn't throw an exception, or did throw an 
exception?
Aug 04 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:

 I'd like to write some unit tests to check my software. Writing
 unittest blocks and putting assert statements in there seems easy
 enough, but I noticed that if the code I run in there throws an
 exception, I don't get the unit test failed message.
 
 How does unittest check for success/failure? Does assert throw a
 special kind of error class when it fails? How would you check
 that some code didn't throw an exception, or did throw an
 exception?


assert throws an AssertError. The unittest failed message is its message, so 
if it's never thrown, then you won't get that message. And if an exception is 
thrown, then the assertion won't fail, so no AssertError will be thrown, just 
the exception that your function threw.

- Jonathan M Davis
Aug 04 2012
parent reply "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 22:09:03 UTC, Jonathan M Davis 
wrote:

 On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:

 How does unittest check for success/failure? Does assert throw 
 a
 special kind of error class when it fails? How would you check
 that some code didn't throw an exception, or did throw an
 exception?


 assert throws an AssertError. The unittest failed message is 
 its message, so
 if it's never thrown, then you won't get that message. And if 
 an exception is
 thrown, then the assertion won't fail, so no AssertError will 
 be thrown, just
 the exception that your function threw.


A failed unit test really just means that a unit test block is 
left via an exception. Currently, it just bubbles up to the 
druntime main(), where it is printed to console and causes a 
non-zero exit code, but future unit test frameworks could handle 
this in a more advanced way.

David
Aug 04 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 00:15:49 David Nadlinger wrote:

 A failed unit test really just means that a unit test block is
 left via an exception. Currently, it just bubbles up to the
 druntime main(), where it is printed to console and causes a
 non-zero exit code, but future unit test frameworks could handle
 this in a more advanced way.


They could, but I think that it's pretty normal to just treat any thrown 
exception as a test failure. The only difference here and what I've seen in 
other test frameworks, is that most test frameworks would actually tell you 
what test failed, which really doesn't work in D, since unittest blocks are 
unnamed (whereas most unit test frameworks used named tests). That's one of 
the reasons why I'd love it if we could start naming unittest blocks, but in 
spite of some interest in such a feature, no one has implemented it yet.

- Jonathan M Davis
Aug 04 2012
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 4 August 2012 at 22:26:26 UTC, Jonathan M Davis 
wrote:

 On Sunday, August 05, 2012 00:15:49 David Nadlinger wrote:

 A failed unit test really just means that a unit test block is
 left via an exception. Currently, it just bubbles up to the
 druntime main(), where it is printed to console and causes a
 non-zero exit code, but future unit test frameworks could 
 handle
 this in a more advanced way.


 They could, but I think that it's pretty normal to just treat 
 any thrown
 exception as a test failure. The only difference here and what 
 I've seen in
 other test frameworks, is that most test frameworks would 
 actually tell you
 what test failed, which really doesn't work in D, since 
 unittest blocks are
 unnamed (whereas most unit test frameworks used named tests). 
 That's one of
 the reasons why I'd love it if we could start naming unittest 
 blocks, but in
 spite of some interest in such a feature, no one has 
 implemented it yet.

 - Jonathan M Davis


you can do it as a library solution just fine.
Aug 04 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 01:01:03 Tobias Pankrath wrote:

 you can do it as a library solution just fine.


You shouldn't have to use a library to get named unit tests, since unit 
testing is built into D, and the lack of name unit tests is a real problem for 
stack traces and the like. It was suggested at one point that at minimum, 
unittest block functions have their line number in their name so that you can 
figure out which unittest block you're dealing with even if you can't name it 
yourself  ( http://d.puremagic.com/issues/show_bug.cgi?id=5587 ), and not even 
that's been done yet. Though now that I read through that enhancement request 
again, it looks like theres an old pull request with the necessary changes in 
it ( https://github.com/D-Programming-Language/dmd/pull/264 ), but it's one of 
those that's been sitting around for a long time, probably because it's an 
enhancement rather than a bug fix, and Walter just hasn't gotten around to it.

- Jonathan M Davis
Aug 04 2012
parent reply "Maxime Chevalier" <maximechevalierb gmail.com> writes:
These improvements would be very nice. The unit test framework, 
as it is, is rather underpowered.

Exceptions could also use more documentation on the D website. I 
heard there was some exception chaining mechanism, but I can't 
even seem to find any info on the Error class.


 You shouldn't have to use a library to get named unit tests, 
 since unit
 testing is built into D, and the lack of name unit tests is a 
 real problem for
 stack traces and the like. It was suggested at one point that 
 at minimum,
 unittest block functions have their line number in their name 
 so that you can
 figure out which unittest block you're dealing with even if you 
 can't name it
 yourself  ( http://d.puremagic.com/issues/show_bug.cgi?id=5587 
 ), and not even
 that's been done yet. Though now that I read through that 
 enhancement request
 again, it looks like theres an old pull request with the 
 necessary changes in
 it ( https://github.com/D-Programming-Language/dmd/pull/264 ), 
 but it's one of
 those that's been sitting around for a long time, probably 
 because it's an
 enhancement rather than a bug fix, and Walter just hasn't 
 gotten around to it.

 - Jonathan M Davis
Aug 04 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, August 05, 2012 01:43:55 Maxime Chevalier wrote:

 These improvements would be very nice. The unit test framework,
 as it is, is rather underpowered.
 
 Exceptions could also use more documentation on the D website. I
 heard there was some exception chaining mechanism, but I can't
 even seem to find any info on the Error class.


It's in object_.d in druntime (which has the definition for Object in it), but 
like much of druntime, it's not documented in the API docs. However, there is 
some information about them here:

http://dlang.org/errors.html

TDPL (The D Programming Language by Andrei Alexandrescu) tends to cover a lot 
of those sort of things better than the online docs do. The online docs 
definitely could use some improvements.

- Jonathan M Davis
Aug 04 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-08-05 01:43, Maxime Chevalier wrote:

 These improvements would be very nice. The unit test framework, as it
 is, is rather underpowered.

 Exceptions could also use more documentation on the D website. I heard
 there was some exception chaining mechanism, but I can't even seem to
 find any info on the Error class.


I have a simple, minimal unit test framework as part of my serialization 
library:

https://github.com/jacob-carlborg/orange/blob/master/orange/test/UnitTester.d

-- 
/Jacob Carlborg
Aug 05 2012
prev sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 22:15:53 UTC, David Nadlinger wrote:

 On Saturday, 4 August 2012 at 22:09:03 UTC, Jonathan M Davis 
 wrote:

 On Sunday, August 05, 2012 00:00:24 Maxime Chevalier wrote:

 How does unittest check for success/failure? Does assert 
 throw a special kind of error class when it fails? How would 
 you check that some code didn't throw an exception, or did 
 throw an exception?







 assert throws an AssertError. The unittest failed message is 
 its message, so if it's never thrown, then you won't get that 
 message. And if an exception is thrown, then the assertion 
 won't fail, so no AssertError will be thrown, just the 
 exception that your function threw.





 A failed unit test really just means that a unit test block is 
 left via an exception. Currently, it just bubbles up to the 
 druntime main(), where it is printed to console and causes a 
 non-zero exit code, but future unit test frameworks could 
 handle this in a more advanced way.


  Not if it's static asserts :) Unless I'm wrong, I've recently 
used a few static asserts to ensure certain combinations of code 
don't compile (on purpose). This is important for proving the 
logic works the way it is suppose to. Sorta following TDPL 
example of is(typof() == bool) (pg 141):

   const int a = 42;

   static assert(is(typeof(a = 24) == bool) == false,
     "constness from 'a' not honored.");

  Although it seems like a small silly example, but when you 
include something generated for a mixin or template functions it 
becomes a very different story. Plus you'd get your warnings and 
errors during compiling and not runtime (find bugs faster?)
Aug 04 2012