www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - unit test that show more than one failure

reply Joel <joelcnz gmail.com> writes:
I've tired different unit test libraries, but they jump out on 
errors instead of just adding to failed numbers.

I'm thinking like this:

```
 ("dummy");
unittset {
   0.shouldEqual(0);
   1.shouldEqual(2);
   2.shouldEqual(3);
}
```

Test: dummy
test passed line 10: 0 is equal to 0
test failed line 11: 1 is not equal to 2
test failed line 12: 2 is not equal to 3

1 passed
2 failed

The unit tests I tried would jump out on the first failure.
Jun 15 2020
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 16 June 2020 at 06:19:51 UTC, Joel wrote:

 I've tired different unit test libraries, but they jump out on 
 errors instead of just adding to failed numbers.

 I'm thinking like this:

 ```
  ("dummy");
 unittset {
   0.shouldEqual(0);
   1.shouldEqual(2);
   2.shouldEqual(3);
 }
 ```

 Test: dummy
 test passed line 10: 0 is equal to 0
 test failed line 11: 1 is not equal to 2
 test failed line 12: 2 is not equal to 3

 1 passed
 2 failed

 The unit tests I tried would jump out on the first failure.


I assume most unit test frameworks doesn't work that way because
this violates the single responsibility principal. A test should
only test 1 thing. In your example you want to test multiple 
things
and this is an indicator that you should have multiple tests 
instead.

That said, it is of course possible that a unittest test framework
could work the way you propose, but it is not desired.

Kind regards
André
Jun 15 2020
prev sibling next sibling parent reply Luis <luis.panadero gmail.com> writes:
On Tuesday, 16 June 2020 at 06:19:51 UTC, Joel wrote:

 I've tired different unit test libraries, but they jump out on 
 errors instead of just adding to failed numbers.

 I'm thinking like this:

 ```
  ("dummy");
 unittset {
   0.shouldEqual(0);
   1.shouldEqual(2);
   2.shouldEqual(3);
 }
 ```

 Test: dummy
 test passed line 10: 0 is equal to 0
 test failed line 11: 1 is not equal to 2
 test failed line 12: 2 is not equal to 3

 1 passed
 2 failed

 The unit tests I tried would jump out on the first failure.


I understand that where the trivial test code is placed, must be 
something more complex being tested.

 ("dummy test 1");
unittest {
    /// Some test code that runs fine
}

 ("dummy test 2");
unittest {
    /// Some test code that fails
}

 ("dummy test 3");
unittest {
    /// Some test code that runs fine
}

Does (with Silly) :

✓ app dummy test 1
✗ app dummy test 2
✓ app dummy test 3

Summary: 2 passed, 1 failed in XX ms


If you literary expects that some test runner do what you 
write... sorry but you will never find it on ANY language (that I 
know).
Jun 16 2020
parent Joel <joelcnz gmail.com> writes:
On Tuesday, 16 June 2020 at 07:39:20 UTC, Luis wrote:

 On Tuesday, 16 June 2020 at 06:19:51 UTC, Joel wrote:

 [...]


 I understand that where the trivial test code is placed, must 
 be something more complex being tested.

  ("dummy test 1");
 unittest {
    /// Some test code that runs fine
 }

  ("dummy test 2");
 unittest {
    /// Some test code that fails
 }

  ("dummy test 3");
 unittest {
    /// Some test code that runs fine
 }

 Does (with Silly) :

 ✓ app dummy test 1
 ✗ app dummy test 2
 ✓ app dummy test 3

 Summary: 2 passed, 1 failed in XX ms


 If you literary expects that some test runner do what you 
 write... sorry but you will never find it on ANY language (that 
 I know).


Oh, that's what it is (one UDA a test). Thanks guys.
Jun 16 2020
prev sibling parent Dukc <ajieskola gmail.com> writes:
On Tuesday, 16 June 2020 at 06:19:51 UTC, Joel wrote:

 I've tired different unit test libraries, but they jump out on 
 errors instead of just adding to failed numbers.

 I'm thinking like this:

 ```
  ("dummy");
 unittset {
   0.shouldEqual(0);
   1.shouldEqual(2);
   2.shouldEqual(3);
 }
 ```

 Test: dummy
 test passed line 10: 0 is equal to 0
 test failed line 11: 1 is not equal to 2
 test failed line 12: 2 is not equal to 3

 1 passed
 2 failed

 The unit tests I tried would jump out on the first failure.


I have solved this. In my project, I have two modules like this:

```
module assertions;

auto assert_(A)(A canditate)
{	assert(canditate);
}

auto assertOp(alias pred, A)(A canditate)
if (is(typeof(cast(bool) pred(canditate))))
{	assert(pred(canditate));
}

auto assertOp(alias pred, A, B)(A canditate, B check)
if (is(typeof(cast(bool) pred(canditate, check))))
{	assert(pred(canditate, check));
}

auto assertEmpty(A)(A aEl)
{	assertOp!(a => a.empty)(aEl);
}

auto assertEqual(A, B)(A aEl, B bEl)
{	assertOp!((a,b) => a == b)(aEl, bEl);
}

auto assertClose(A, B)(A aEl, B bEl)
{	import std.math;
	assertOp!((a,b) => a.isClose(b))(aEl, bEl);
}
```

```
module assertions.characterization;

auto assert_(A)(A canditate)
{	import std.stdio : writeln;
	writeln(canditate, cast(bool) canditate? " (passes)": " 
(fails)");
}

auto assertOp(alias pred, A)(A canditate)
if (is(typeof(cast(bool) pred(canditate))))
{	import std.stdio : writeln;
	writeln(canditate, cast(bool) pred(canditate)? " (passes)": " 
(fails)");
}

auto assertOp(alias pred, A, B)(A canditate, B check)
if (is(typeof(cast(bool) pred(canditate, check))))
{	import std.stdio : writeln;
	writeln(canditate, cast(bool) pred(canditate, check)? " 
(passes)": " (fails)");
}

auto assertEmpty(A)(A aEl)
{	assertOp!(a => a.empty)(aEl);
}

auto assertEqual(A, B)(A aEl, B bEl)
{	assertOp!((a,b) => a == b)(aEl, bEl);
}

auto assertClose(A, B)(A aEl, B bEl)
{	import std.math;
	assertOp!((a,b) => a.isClose(b))(aEl, bEl);
}
```

So I usually have the test like this:
```
 safe unittest
{  import assertions;
    0.assertEqual(0);
    1.assertEqual(2);
    2.assertEqual(3);
}
```

...but when it fails, I just change the import to 
`assertions.characterization`, and all test results, along with 
info whether they're correct, are printed.
Jun 16 2020