www.digitalmars.com         C & C++   DMDScript  

D - Unittests

reply Dario <Dario_member pathlink.com> writes:
I would be nice that unittests were automatically launched by the compiler as
soon as it has finished the code-generation phase and that unittests weren't
really included in the object file at last.
This way you'll know if a unittest failed as soon as you compile the code, and
the -unittest option would be active by default when -release is not enabled.
What do you think?
-Dario
Jul 12 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Dario" <Dario_member pathlink.com> wrote in message
news:bepjq1$1q68$1 digitaldaemon.com...
 I would be nice that unittests were automatically launched by the compiler

 soon as it has finished the code-generation phase and that unittests

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the code,

 the -unittest option would be active by default when -release is not

 What do you think?

It's an intriguing idea, but the compiler has no way of executing code in a .obj file without doing a full link. Doing a full link gets you where you are today anyway with running unittests <g>.
Jul 13 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:
 
 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...
 I would be nice that unittests were automatically launched by the compiler

 soon as it has finished the code-generation phase and that unittests

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the code,

 the -unittest option would be active by default when -release is not

 What do you think?

It's an intriguing idea, but the compiler has no way of executing code in a .obj file without doing a full link. Doing a full link gets you where you are today anyway with running unittests <g>.

I think unittests should: - always tell which test(s) exactly have failed. - never stop on an error but always run through all tests. This is important if you break code while refactoring. You must have the complete information on the total effects. Typically this is accomplished by forking the process and reporting the state of the process back to the main unittest code. - should have a visual display to show how many tests have already been run and how many have failed (this is important for refactoring) because unittests may run (on more complex) systems up to hours. - run only once from an completed exe (an app.exe could create an marker file app.uok whose date-time-stamp is tested against app.exe) -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 13 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F11B6D6.EFEB8CFD chello.at...
 Walter wrote:
 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...
 I would be nice that unittests were automatically launched by the



 as
 soon as it has finished the code-generation phase and that unittests

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the



 and
 the -unittest option would be active by default when -release is not

 What do you think?



 .obj file without doing a full link. Doing a full link gets you where


 are today anyway with running unittests <g>.

I think unittests should: - always tell which test(s) exactly have failed.

It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.
    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must have
      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.

My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.
    - should have a visual display to show how many tests have already been
      run and how many have failed (this is important for refactoring)

      unittests may run (on more complex) systems up to hours.

You can do this by writing such code into the unittest blocks.
    - run only once from an completed exe (an app.exe could create an

      file app.uok whose date-time-stamp is tested against app.exe)

That's an interesting approach. I've used self-patching exe's in the past to do initializations; I had to stop the practice because anti-virus programs had fits about it.
Jul 13 2003
next sibling parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:
 I think unittests should:

    - always tell which test(s) exactly have failed.

It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.

We had an fault during ICFP-contest where it didn't.
    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must have
      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.

My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.

This may or may not be the case. But that's not important. Important is that the people that invented unittests and write books about unittests and advocate test driven development expect it - I think with good reasons - to work that way.
    - should have a visual display to show how many tests have already been
      run and how many have failed (this is important for refactoring)

      unittests may run (on more complex) systems up to hours.

You can do this by writing such code into the unittest blocks.

It should soon become a phobos module. -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 13 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F1247B1.2788A5B chello.at...
 Walter wrote:
 I think unittests should:
    - always tell which test(s) exactly have failed.



 can put any code you want in the unittest blocks, not just assert()'s.


What happened? It sounds like it could have been a GP fault, not an assert fail.
Jul 14 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:
 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1247B1.2788A5B chello.at...
 Walter wrote:
 I think unittests should:
    - always tell which test(s) exactly have failed.



 can put any code you want in the unittest blocks, not just assert()'s.


What happened? It sounds like it could have been a GP fault, not an assert fail.

Yes, something like that, or a DIV/0 fault. I think it was with a sin/cos function that faulted in my local sandbox. I got rid of it by updating DMD to 0.67. I think that Burton Radons knows better, what the real reason was, he seemed to recognize the problem. A unittest has to report an error position (and continue) no matter what caused it. -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 14 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F13934D.C12084C8 chello.at...
 Walter wrote:
 What happened? It sounds like it could have been a GP fault, not an


 fail.

function that faulted in my local sandbox. I got rid of it by updating DMD

 I think that Burton Radons knows better, what the real reason was, he

 to recognize the problem.

 A unittest has to report an error position (and continue) no matter what

The position of a GPF will be revealed by running it under a debugger. You can write contracts that continue by just using something other than the built-in assert: Print message and stop: assert(succeeded); Print message and continue: if (!succeeded) printf("test xyz failed\n");
Jul 15 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:
 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F13934D.C12084C8 chello.at...
 Walter wrote:
 What happened? It sounds like it could have been a GP fault, not an


 fail.

function that faulted in my local sandbox. I got rid of it by updating DMD

 I think that Burton Radons knows better, what the real reason was, he

 to recognize the problem.

 A unittest has to report an error position (and continue) no matter what

The position of a GPF will be revealed by running it under a debugger. You can write contracts that continue by just using something other than the built-in assert: Print message and stop: assert(succeeded); Print message and continue: if (!succeeded) printf("test xyz failed\n");

If you can't locate a GPF by a unittest, then it's not a real unittest. Real unittests fork the program and and GPF won't kill the main process. -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 15 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F1472BF.99DE612 chello.at...
 If you can't locate a GPF by a unittest, then it's not a real unittest.
 Real unittests fork the program and and GPF won't kill the main process.

I see we have different ideas on what unit tests should be!
Jul 16 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Walter wrote:
 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1472BF.99DE612 chello.at...
 If you can't locate a GPF by a unittest, then it's not a real unittest.
 Real unittests fork the program and and GPF won't kill the main process.

I see we have different ideas on what unit tests should be!

You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks. Even if you use a free open source C testing framework under windows, you - catch all errors (including GPFs) - run through all tests no matter how many errors and what errors occur I think it would not even be a problem to supply sample code how to accomplish that. But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words... -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 16 2003
next sibling parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
Surely if a test fails, then other subsequent tests cannot be guaranteed
their correct pre-conditions (not to mention a stable host) so their results
may be meaningless? A little like relying on later compiler errors when you
should be fixing the first one.



"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F163019.BDDD344E chello.at...
 Walter wrote:
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F1472BF.99DE612 chello.at...
 If you can't locate a GPF by a unittest, then it's not a real



 Real unittests fork the program and and GPF won't kill the main



 I see we have different ideas on what unit tests should be!

You have just implemented - and I can understand that - what's easy to implement. At some point you will have to live up to the expectations of test-oriented people used to unittest frameworks. Even if you use a free open source C testing framework under windows, you - catch all errors (including GPFs) - run through all tests no matter how many errors and what errors occur I think it would not even be a problem to supply sample code how to accomplish that. But obviously when you say "have different ideas on what unit tests should be" then the master has spoken his final words... -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com

Jul 16 2003
parent reply Helmut Leitner <leitner hls.via.at> writes:
Matthew Wilson wrote:
 
 Surely if a test fails, then other subsequent tests cannot be guaranteed
 their correct pre-conditions (not to mention a stable host) so their results
 may be meaningless? A little like relying on later compiler errors when you
 should be fixing the first one.

Just look at the way unittests are written. They typically don't build on each other. They are not like the logical flow of a program. -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 17 2003
parent reply "Luna Kid" <lunakid neuropolis.org> writes:
To the "stop or not to stop" debate: would it be
OK adding a compiler (err... linker??) switch like
"do not abort on failed asserts"? (I can imagine
that it may not be as trivial as it sounds, but
I don't know the D implementation of assert().)

Sz.
Jul 18 2003
parent "Walter" <walter digitalmars.com> writes:
"Luna Kid" <lunakid neuropolis.org> wrote in message
news:bf8ae1$18bu$1 digitaldaemon.com...
 To the "stop or not to stop" debate: would it be
 OK adding a compiler (err... linker??) switch like
 "do not abort on failed asserts"? (I can imagine
 that it may not be as trivial as it sounds, but
 I don't know the D implementation of assert().)

You can add an exception handler to catch them and continue. In fact, that's what the startup code does in order to catch & print any out, phobos\dmain2.d: try { _moduleCtor(); _moduleUnitTests(); for (i = 0; i < argc; i++) { int len = strlen(argv[i]); am[i] = argv[i][0 .. len]; } args = am[0 .. argc]; result = main(args); } catch (Object o) { printf("Error: "); o.print(); result = EXIT_FAILURE; }
Jul 20 2003
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F163019.BDDD344E chello.at...
 You have just implemented - and I can understand that - what's
 easy to implement. At some point you will have to live up to the
 expectations of test-oriented people used to unittest frameworks.

I agree that what D provides is basic. However, it is way, *way* ahead of what any other language provides, which is nothing. In my own use of D unit tests, it has proven its value time and again.
 Even if you use a free open source C testing framework under
 windows, you
    - catch all errors (including GPFs)
    - run through all tests no matter how many errors
      and what errors occur

 I think it would not even be a problem to supply sample code
 how to accomplish that.

 But obviously when you say "have different ideas on what unit tests
 should be" then the master has spoken his final words...

D doesn't prevent unittests from operating the way you want them too - but you'd need to write some additional code to implement that.
Jul 17 2003
parent reply Helmut Leitner <leitner hls.via.at> writes:
Walter wrote:
 
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F163019.BDDD344E chello.at...
 You have just implemented - and I can understand that - what's
 easy to implement. At some point you will have to live up to the
 expectations of test-oriented people used to unittest frameworks.

I agree that what D provides is basic. However, it is way, *way* ahead of what any other language provides, which is nothing. In my own use of D unit tests, it has proven its value time and again.
 Even if you use a free open source C testing framework under
 windows, you
    - catch all errors (including GPFs)
    - run through all tests no matter how many errors
      and what errors occur

 I think it would not even be a problem to supply sample code
 how to accomplish that.

 But obviously when you say "have different ideas on what unit tests
 should be" then the master has spoken his final words...

D doesn't prevent unittests from operating the way you want them too - but you'd need to write some additional code to implement that.

Only if you provide an interface to get at the unittest function units. Lets say any unittest is like a void unittest_xx(); then there should be an array of pointers in any module compiled for unittests array_unitest_modulexyz (pointer, codeline) and some datastructure (like a linked list or similar) list_unittest_arrays (arraypointer, modulename) There should be a way to compile the unittests without actually being called at the program start. It should be guaranteed that different unittest blocks within one module are not combined into one functional unit (I don't know if you do). -- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 17 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <leitner hls.via.at> wrote in message
news:3F165C52.F2624CD1 hls.via.at...

 D doesn't prevent unittests from operating the way you want them too -


 you'd need to write some additional code to implement that.

Lets say any unittest is like a void unittest_xx(); then there should be an array of pointers in any module compiled for

   array_unitest_modulexyz (pointer, codeline)
 and some datastructure (like a linked list or similar)
   list_unittest_arrays (arraypointer, modulename)

You can see how they are called in phobos/moduleinit.d. It's equivalent to as you suggest.
 There should be a way to compile the unittests without actually
 being called at the program start.

You can do this by modifying phobos/moduleinit.d.
 It should be guaranteed that different unittest blocks within one module
 are not combined into one functional unit (I don't know if you do).

Check out this code: unittest { printf("1\n"); } unittest { printf("2\n"); } void main() { } which compiles to: _DATA segment db 031h,00ah,000h,032h,00ah,000h,074h,065h db 073h,074h,000h,000h __ModuleInfo_test: db 000h,000h,000h,000h db 000h,000h,000h,000h,004h,000h,000h,000h dd offset FLAT:_DATA[6] db 000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h dd offset FLAT:__modtest_test _DATA ends CONST segment CONST ends _BSS segment _BSS ends FMB segment FMB ends FM segment dd offset FLAT:__ModuleInfo_test FM ends FME segment FME ends _Dtest_unittest0_FZv comdat assume CS:_Dtest_unittest0_FZv L0: push offset FLAT:_DATA call near ptr _printf add ESP,4 ret _Dtest_unittest0_FZv ends _Dtest_unittest1_FZv comdat assume CS:_Dtest_unittest1_FZv L0: push offset FLAT:_DATA[3] call near ptr _printf add ESP,4 ret _Dtest_unittest1_FZv ends __Dmain comdat assume CS:__Dmain ret __Dmain ends __modtest_test comdat assume CS:__modtest_test L0: call near ptr _Dtest_unittest0_FZv call near ptr _Dtest_unittest1_FZv ret __modtest_test ends end
Jul 20 2003
parent reply Helmut Leitner <helmut.leitner chello.at> writes:
Thank you for these hints. They may or may not be sufficient.

The problem is that unittests are implemented on top of almost
any programming language. Most of the time they are separate
testing frameworks.

The D claim to support unittests would make a lot of sense 
if - and only if - the support is out of the box. Otherwise 
it means nothing.

I'll try to build on the hooks you gave me, but it will only
make sense when reintegrated into D.

Walter wrote:
 
 "Helmut Leitner" <leitner hls.via.at> wrote in message
 news:3F165C52.F2624CD1 hls.via.at...
 
 D doesn't prevent unittests from operating the way you want them too -


 you'd need to write some additional code to implement that.

Lets say any unittest is like a void unittest_xx(); then there should be an array of pointers in any module compiled for

   array_unitest_modulexyz (pointer, codeline)
 and some datastructure (like a linked list or similar)
   list_unittest_arrays (arraypointer, modulename)

You can see how they are called in phobos/moduleinit.d. It's equivalent to as you suggest.
 There should be a way to compile the unittests without actually
 being called at the program start.

You can do this by modifying phobos/moduleinit.d.
 It should be guaranteed that different unittest blocks within one module
 are not combined into one functional unit (I don't know if you do).

Check out this code: unittest { printf("1\n"); } unittest { printf("2\n"); } void main() { } which compiles to: _DATA segment db 031h,00ah,000h,032h,00ah,000h,074h,065h db 073h,074h,000h,000h __ModuleInfo_test: db 000h,000h,000h,000h db 000h,000h,000h,000h,004h,000h,000h,000h dd offset FLAT:_DATA[6] db 000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h db 000h,000h,000h,000h,000h,000h,000h,000h dd offset FLAT:__modtest_test _DATA ends CONST segment CONST ends _BSS segment _BSS ends FMB segment FMB ends FM segment dd offset FLAT:__ModuleInfo_test FM ends FME segment FME ends _Dtest_unittest0_FZv comdat assume CS:_Dtest_unittest0_FZv L0: push offset FLAT:_DATA call near ptr _printf add ESP,4 ret _Dtest_unittest0_FZv ends _Dtest_unittest1_FZv comdat assume CS:_Dtest_unittest1_FZv L0: push offset FLAT:_DATA[3] call near ptr _printf add ESP,4 ret _Dtest_unittest1_FZv ends __Dmain comdat assume CS:__Dmain ret __Dmain ends __modtest_test comdat assume CS:__modtest_test L0: call near ptr _Dtest_unittest0_FZv call near ptr _Dtest_unittest1_FZv ret __modtest_test ends end

-- Helmut Leitner leitner hls.via.at Graz, Austria www.hls-software.com
Jul 26 2003
parent "Walter" <walter digitalmars.com> writes:
"Helmut Leitner" <helmut.leitner chello.at> wrote in message
news:3F226F64.EC1B9CCC chello.at...
 Thank you for these hints. They may or may not be sufficient.

 The problem is that unittests are implemented on top of almost
 any programming language. Most of the time they are separate
 testing frameworks.

 The D claim to support unittests would make a lot of sense
 if - and only if - the support is out of the box. Otherwise
 it means nothing.

 I'll try to build on the hooks you gave me, but it will only
 make sense when reintegrated into D.

Let me know how it goes. -Walter
Aug 12 2003
prev sibling parent reply "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
Hi,

    Comments embedded.

"Walter" <walter digitalmars.com> escreveu na mensagem
news:besod7$1qb8$1 digitaldaemon.com...
 "Helmut Leitner" <helmut.leitner chello.at> wrote in message
 news:3F11B6D6.EFEB8CFD chello.at...
 Walter wrote:
 "Dario" <Dario_member pathlink.com> wrote in message
 news:bepjq1$1q68$1 digitaldaemon.com...
 I would be nice that unittests were automatically launched by the



 as
 soon as it has finished the code-generation phase and that unittests

 really included in the object file at last.
 This way you'll know if a unittest failed as soon as you compile the



 and
 the -unittest option would be active by default when -release is not

 What do you think?



 .obj file without doing a full link. Doing a full link gets you where


 are today anyway with running unittests <g>.



I think a make-like utility is essential in D projects, I usually have a dummy "main" for unittests that just return 0, so I just type "make tests" and everything is fine. You shouldn't compile and link with your real "main" using -unittest.
 I think unittests should:

    - always tell which test(s) exactly have failed.

It does already, with the file/line of the failed contract. Of course, you can put any code you want in the unittest blocks, not just assert()'s.

It isn't enough for a couple of reasons: assert just give me the file/line of the assert failed, not the call stack. If I have a contract failing inside a method after twenty calls from my unittest the error message don't give me any context to work with. A very good solution would be making the contract failure info Eiffel-like, dumping the call stack and the variables in stack at the time (so we can figure out the context). The current state is really crude, I usually resort to insert several lines of printf to pinpoint the error. Of course I could debug instead of this, but I use TDD and in any other language the tests are enough.
    - never stop on an error but always run through all tests.
      This is important if you break code while refactoring. You must


      the complete information on the total effects.
      Typically this is accomplished by forking the process and reporting
      the state of the process back to the main unittest code.

My experience usually is that if one test fails, subsequent tests fail as well because of cascading errors from the first fault.

Today we have units of unittests so I think each unit would be independent. If I have three "unittest" blocks they are independent (they are test fixtures, and each must be independent). When I change something it's important to know that the "insert" operation failed but everything else is still fine. Today it's all or nothing.
    - should have a visual display to show how many tests have already


      run and how many have failed (this is important for refactoring)

      unittests may run (on more complex) systems up to hours.

You can do this by writing such code into the unittest blocks.

If unittests had names (like "unittest testFindString { ... }") using reflective libraries we could automate this process and create different test reporters for different environments (e.g. text, GUI, XML, etc.) like any of the xUnit tools make. Walter, how much do you know about the xUnit tools (e.g. jUnit, sUnit)? There's a lot of info in www.junit.org about how they could develop a powerful, extensible, tool using Java's reflexive mechanisms. I'll make a summary of jUnit basic features: - It has classes representing test cases, test suites, test results and test runners. - Test cases are just classes extending TestCase containing methods named testXXXXXXXX and two special methods: setUp and tearDown. When the test case is executed the test runner uses reflection to find all methods named testXXXXXX, and for each one executes the TestCase setUp method, the current test method and finally calls the TestCase tearDown method. If an assertion fails inside a TestCase test method, it raises an exception that is used to report the failure. - TestSuites are a bunch of TestCases bundled together. - TestResults encapsulate the details of the executed tests, including which failed, which succeded and which raised uncatched exceptions. - TestRunners run the tests and report the failures in a given medium (e.g. text, GUI, XML, e-mail to the developers, etc.). This is just a quick look of how it works, but almost every programmer coming to D will look after these features. I use jUnit all the time in Java, and D unittest support is pale in comparison. If we need some kind of house-keeping operations (i.e. setUp and tearDown) we have to remember calling them explicitly, if we need some kind of alternative medium (GUI is much better than text) we must explicit code each every time. I'm tempted to rewrite jUnit in D, actually I'm doing that right now, just to have these features.
    - run only once from an completed exe (an app.exe could create an

      file app.uok whose date-time-stamp is tested against app.exe)

That's an interesting approach. I've used self-patching exe's in the past

 do initializations; I had to stop the practice because anti-virus programs
 had fits about it.

Having unittests as first class objects is nice, but if the unittesting support is incomplete regarding other languages, it won't be necessary and probably discarded in larger projects. jUnit is an example of how it should be, for other frameworks you can check http://www.artima.com/suiterunner/index.html , http://www.fitnesse.org/ , http://fit.c2.com and http://c2.com/cgi/wiki?FrameworkForIntegratedTest Best regards, Daniel Yokomiso. A: Top-posters. Q: What is the most annoying thing on Usenet? --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.502 / Virus Database: 300 - Release Date: 18/7/2003
Jul 23 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> wrote in message
news:bfn8lg$2hm2$1 digitaldaemon.com...
 It does already, with the file/line of the failed contract. Of course,


 can put any code you want in the unittest blocks, not just assert()'s.

file/line of the assert failed, not the call stack. If I have a contract failing inside a method after twenty calls from my unittest the error message don't give me any context to work with. A very good solution would be making the contract failure info Eiffel-like, dumping the call stack

 the variables in stack at the time (so we can figure out the context). The
 current state is really crude, I usually resort to insert several lines of
 printf to pinpoint the error. Of course I could debug instead of this, but

 use TDD and in any other language the tests are enough.

The stack trace is a good idea. I use it regularly in a debugger, which prints it out just fine. I've thought of some ways to implement it in general for D exceptions, which would solve your issue.
     Today we have units of unittests so I think each unit would be
 independent. If I have three "unittest" blocks they are independent (they
 are test fixtures, and each must be independent). When I change something
 it's important to know that the "insert" operation failed but everything
 else is still fine. Today it's all or nothing.

You can continue by wrapping the tests you want to continue with a try-catch.
     If unittests had names (like "unittest testFindString { ... }") using
 reflective libraries we could automate this process and create different
 test reporters for different environments (e.g. text, GUI, XML, etc.) like
 any of the xUnit tools make. Walter, how much do you know about the xUnit
 tools (e.g. jUnit, sUnit)?

Nothing.
 There's a lot of info in www.junit.org about how
 they could develop a powerful, extensible, tool using Java's reflexive
 mechanisms. I'll make a summary of jUnit basic features:

 - It has classes representing test cases, test suites, test results and

 runners.
 - Test cases are just classes extending TestCase containing methods named
 testXXXXXXXX and two special methods: setUp and tearDown. When the test

 is executed the test runner uses reflection to find all methods named
 testXXXXXX, and for each one executes the TestCase setUp method, the

 test method and finally calls the TestCase tearDown method. If an

 fails inside a TestCase test method, it raises an exception that is used

 report the failure.
 - TestSuites are a bunch of TestCases bundled together.
 - TestResults encapsulate the details of the executed tests, including

 failed, which succeded and which raised uncatched exceptions.
 - TestRunners run the tests and report the failures in a given medium

 text, GUI, XML, e-mail to the developers, etc.).

    This is just a quick look of how it works, but almost every programmer
 coming to D will look after these features. I use jUnit all the time in
 Java, and D unittest support is pale in comparison. If we need some kind

 house-keeping operations (i.e. setUp and tearDown) we have to remember
 calling them explicitly, if we need some kind of alternative medium (GUI

 much better than text) we must explicit code each every time. I'm tempted

 rewrite jUnit in D, actually I'm doing that right now, just to have these
 features.

Sounds like a good idea. Such a "dUnit" be a great tool to add. Keep us posted on the progress on it!
Aug 12 2003
parent "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
Hi,

    Comments embedded.

"Walter" <walter digitalmars.com> escreveu na mensagem
news:bhbk0a$1p9k$1 digitaldaemon.com...
[snip]
 The stack trace is a good idea. I use it regularly in a debugger, which
 prints it out just fine. I've thought of some ways to implement it in
 general for D exceptions, which would solve your issue.

Great! It's one of the most annying things today.
     Today we have units of unittests so I think each unit would be
 independent. If I have three "unittest" blocks they are independent


 are test fixtures, and each must be independent). When I change


 it's important to know that the "insert" operation failed but everything
 else is still fine. Today it's all or nothing.

You can continue by wrapping the tests you want to continue with a try-catch.

I guess this is a matter of personal taste. I usually write orthogonal tests, so a failure in one doesn't imply in failures of others. The xUnit tools use this metaphor too, and many programmers use it, but I don't need it this way.
     If unittests had names (like "unittest testFindString { ... }")


 reflective libraries we could automate this process and create different
 test reporters for different environments (e.g. text, GUI, XML, etc.)


 any of the xUnit tools make. Walter, how much do you know about the


 tools (e.g. jUnit, sUnit)?

Nothing.

I think it'll be very good for you to read papers and code about these tools. They'll give you some perspective of the "standards" in unit testing.
 There's a lot of info in www.junit.org about how
 they could develop a powerful, extensible, tool using Java's reflexive
 mechanisms. I'll make a summary of jUnit basic features:

 - It has classes representing test cases, test suites, test results and

 runners.
 - Test cases are just classes extending TestCase containing methods


 testXXXXXXXX and two special methods: setUp and tearDown. When the test

 is executed the test runner uses reflection to find all methods named
 testXXXXXX, and for each one executes the TestCase setUp method, the

 test method and finally calls the TestCase tearDown method. If an

 fails inside a TestCase test method, it raises an exception that is used

 report the failure.
 - TestSuites are a bunch of TestCases bundled together.
 - TestResults encapsulate the details of the executed tests, including

 failed, which succeded and which raised uncatched exceptions.
 - TestRunners run the tests and report the failures in a given medium

 text, GUI, XML, e-mail to the developers, etc.).

    This is just a quick look of how it works, but almost every


 coming to D will look after these features. I use jUnit all the time in
 Java, and D unittest support is pale in comparison. If we need some kind

 house-keeping operations (i.e. setUp and tearDown) we have to remember
 calling them explicitly, if we need some kind of alternative medium (GUI

 much better than text) we must explicit code each every time. I'm


 to
 rewrite jUnit in D, actually I'm doing that right now, just to have


 features.

Sounds like a good idea. Such a "dUnit" be a great tool to add. Keep us posted on the progress on it!

Looks like this week I'll resume coding in D, but most of the dUnit tool depends on the stack trace information. Best regards, Daniel Yokomiso. "I drink to make other people interesting." - Groucho Marx --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.510 / Virus Database: 307 - Release Date: 14/8/2003
Aug 22 2003