www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: Unit tests in D

reply bearophile <bearophileHUGS lycos.com> writes:
Michel Fortin:

 Can't you do:
 	static assert(throws!OtherException(sqrt(-5)));
 ?

Some of those examples of mine were wrong because the main purpose of a 'static throws' is to catch static asserts, not to catch exceptions at compile time, sorry: static throws(foo(10), StaticException1); static throws(foo(10), StaticException1, StaticException2, ...); static throws!(StaticException1)(foo(10)); static throws!(StaticException1, StaticException2)(foo(10)); (And currently try-catch statements are not supported in CTFE, see http://d.puremagic.com/issues/show_bug.cgi?id=4067 ). Bye, bearophile
May 05 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-05-05 07:46:46 -0400, bearophile <bearophileHUGS lycos.com> said:

 Michel Fortin:
 
 Can't you do:
 	static assert(throws!OtherException(sqrt(-5)));
 ?

Some of those examples of mine were wrong because the main purpose of a 'static throws' is to catch static asserts, not to catch exceptions at compile time, sorry:

Am I right that what you want is this? static assert(!__traits(compiles, foo(10))); I agree that the __traits syntax leaves a lot of room for improvement.
 (And currently try-catch statements are not supported in CTFE, see 
 http://d.puremagic.com/issues/show_bug.cgi?id=4067 ).

Indeed. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Michel Fortin:

 Am I right that what you want is this?
 	static assert(!__traits(compiles, foo(10)));

I want a nice syntax that statically asserts if foo(10) doesn't statically asserts :-) What you have written is close (in D1 I have used the is() syntax for a similar purpose), but beside being syntactically ugly, it doesn't specifically detect static asserts inside foo. Bye, bearophile
May 05 2010
parent reply Don <nospam nospam.com> writes:
bearophile wrote:
 Michel Fortin:
 
 Am I right that what you want is this?
 	static assert(!__traits(compiles, foo(10)));

I want a nice syntax that statically asserts if foo(10) doesn't statically asserts :-) What you have written is close (in D1 I have used the is() syntax for a similar purpose), but beside being syntactically ugly, it doesn't specifically detect static asserts inside foo.

It seems pretty useless to me. Wanting to make a distinction between "this will not compile" and "this will not compile _because it hits a static assert_" is an *extremely* niche feature. Because the only times I can imagine that you'd care would be because you wanted to ensure it gave a "nice" error message. And to do that, you'd have to actually check the test of the static assert. And the idea of creating a whole heirarchy of compile-time exceptions for this ...
May 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:

It seems pretty useless to me. Wanting to make a distinction between "this will
not compile" and "this will not compile _because it hits a static assert_" is
an *extremely* niche feature. Because the only times I can imagine that you'd
care would be because you wanted to ensure it gave a "nice" error message. And
to do that, you'd have to actually check the test of the static assert.<

Let's assume you are right, that making such distinction is generally useless. In my unittests I write one or two tests every time something contains a static assert (plus other tests for other compile time errors). So even if you are right, I have to use a syntax like this often enough inside unittests: static assert(!__traits(compiles, foo(10))); So for this I'd like a simpler syntax.
And the idea of creating a whole heirarchy of compile-time exceptions for this
...<

No hierarchy required, sorry, the examples I have written for 'static throws' were all wrong. Thank you for your answers, bye, bearophile
May 05 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:
 I have to use a syntax like this often enough inside
 unittests:
 
 static assert(!__traits(compiles, foo(10)));

But why? Just use: foo(10);
May 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:
 But why? Just use: 
     foo(10);

I think you are missing something important here. Static asserts (or other implicit errors) inside a template, etc, test that some input types are correct, some template input values are in the correct range, etc. In this thread we are talking about unittests. The purpose of a unit Inside a unit test is to test that something that can be called Foo works as specified. Working as specified means such Foo must return the correct outputs when the inputs are in its intended range of possible inputs (otherwise the unittest has to fail), and it must produce a compile time assert, run time assert, or throw an exception if the input values are outside the allowed ones (otherwise the unittest has to fail again). So the purpose of the feature I am talking here is for the group of those unittests, to make sure something asserts at compile time (or otherwise doesn't compile) when the compile-time inputs are wrong. So I need something that inside the unittest asserts at compile time if Foo does not asserts at compile-time (or otherwise refuses to work). Bye, bearophile
May 05 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:
 Walter Bright:
 But why? Just use: foo(10);

I think you are missing something important here. Static asserts (or other implicit errors) inside a template, etc, test that some input types are correct, some template input values are in the correct range, etc. In this thread we are talking about unittests. The purpose of a unit Inside a unit test is to test that something that can be called Foo works as specified. Working as specified means such Foo must return the correct outputs when the inputs are in its intended range of possible inputs (otherwise the unittest has to fail), and it must produce a compile time assert, run time assert, or throw an exception if the input values are outside the allowed ones (otherwise the unittest has to fail again). So the purpose of the feature I am talking here is for the group of those unittests, to make sure something asserts at compile time (or otherwise doesn't compile) when the compile-time inputs are wrong. So I need something that inside the unittest asserts at compile time if Foo does not asserts at compile-time (or otherwise refuses to work).

I'm sorry, I simply don't understand this. If you want to test that something compiles in a unit test, just write the code. The compiler will let you know if it doesn't compile.
May 05 2010
next sibling parent reply Lutger <lutger.blijdestijn gmail.com> writes:
Walter Bright wrote:

 bearophile wrote:
 Walter Bright:
 But why? Just use: foo(10);

I think you are missing something important here. Static asserts (or other implicit errors) inside a template, etc, test that some input types are correct, some template input values are in the correct range, etc. In this thread we are talking about unittests. The purpose of a unit Inside a unit test is to test that something that can be called Foo works as specified. Working as specified means such Foo must return the correct outputs when the inputs are in its intended range of possible inputs (otherwise the unittest has to fail), and it must produce a compile time assert, run time assert, or throw an exception if the input values are outside the allowed ones (otherwise the unittest has to fail again). So the purpose of the feature I am talking here is for the group of those unittests, to make sure something asserts at compile time (or otherwise doesn't compile) when the compile-time inputs are wrong. So I need something that inside the unittest asserts at compile time if Foo does not asserts at compile-time (or otherwise refuses to work).

I'm sorry, I simply don't understand this. If you want to test that something compiles in a unit test, just write the code. The compiler will let you know if it doesn't compile.

He wants the opposite: to test that something detects an error correctly, making sure his template doesn't silently compile wrong code.
May 05 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
Lutger wrote:
 He wants the opposite: to test that something detects an error correctly, 
 making sure his template doesn't silently compile wrong code. 

I don't see that in the example given.
May 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright>I don't see that in the example given.<

I can try with one real example from my dlibs1. This template is the third I
use with foreach to create the fake static foreach, this accepts a step
(stride) value too.

But such step can't be zero, so I add a static assert inside it to make sure
you don't try to instantiate it with a step = 0.


template Range(int start, int stop, int step) {
    static assert(step != 0, "Range: step must be != 0");

    static if (step > 0) {
        static if (stop <= start)
            alias Tuple!() Range;
        else
            alias Tuple!(Range!(start, stop-step, step), stop-step) Range;
    } else {
        static if (stop >= start)
            alias Tuple!() Range;
        else
            alias Tuple!(Range!(start, stop-step, step), stop-step) Range;
    }
}


In D2 I can use a template constraint instead of a static assert here (this is
sometimes bad, because you lose the error text message given by the static
assert, that in complex situations can be useful):

template Range(int start, int stop, int step) if (step != 0) {


Now that Range needs unittests, things like this that test that Range gives the
right output:

unittest { // Tests of Range!()
    int[] a;
    foreach (n; Range!(0, 10, 2))
        a ~= n;
    assert(a == [0, 2, 4, 6, 8]);
}


But the "specs" of the Range template say that Range must not work if step is
zero (this means that the unittest has to fail if Range compiles when the given
step is zero). So I have to test this too. I can do that with one more unittest
(D1 code):

    static assert(!is(typeof( Range!(15, 3, 0) )));

In D2 I can use the __traits:

    static assert(!__traits(compiles, Range!(15, 3, 0) ));

Hope this helps.

Bye,
bearophile
May 05 2010
parent reply Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:

 
 But the "specs" of the Range template say that Range must not work if step is
zero (this means that the unittest has to fail if Range compiles when the given
step is zero). So I have to test this too. I can do that with one more unittest
(D1 code):
 
     static assert(!is(typeof( Range!(15, 3, 0) )));
 
 In D2 I can use the __traits:
 
     static assert(!__traits(compiles, Range!(15, 3, 0) ));

Oh, I see. You want to ensure it does not compile, rather than it compiles. I got the ! flipped around.
May 05 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-05-05 20:25:45 -0400, Walter Bright <newshound1 digitalmars.com> said:

 bearophile wrote:
 
 
 But the "specs" of the Range template say that Range must not work if 
 step is zero (this means that the unittest has to fail if Range 
 compiles when the given step is zero). So I have to test this too. I 
 can do that with one more unittest (D1 code):
 
     static assert(!is(typeof( Range!(15, 3, 0) )));
 
 In D2 I can use the __traits:
 
     static assert(!__traits(compiles, Range!(15, 3, 0) ));

Oh, I see. You want to ensure it does not compile, rather than it compiles. I got the ! flipped around.

If even Walter has difficulty figuring out the ! around __traits, I'll take that as the ultimate proof that the current syntax has too much cruft and is in need of a cleanup. Could we at least replace this: __traits(compiles, ...) with this: __traits.compiles(...) It looks more readable to me at least, and it can be applied to other traits. The next step would be to find a way to remove that ugly __traits keyword, ideally without stealing a useful identifier. Perhaps it should be made possible to do this: module std.traits; alias __traits.compiles compiles; Now you just import std.traits and never write __traits again! :-) static assert(!compiles( Range!(15, 3, 0) )); -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 05 2010
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Michel Fortin wrote:
 If even Walter has difficulty figuring out the ! around __traits,

Missing a ! is always a problem. People even do not see the word "not".
May 05 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-05-05 21:12:47 -0400, Walter Bright <newshound1 digitalmars.com> said:

 Michel Fortin wrote:
 If even Walter has difficulty figuring out the ! around __traits,

Missing a ! is always a problem. People even do not see the word "not".

It was an exaggeration. That "ultimate proof" wasn't meant to be too serious. I hope it didn't stop you from reading the rest of the post, because it seemed like a nice idea to improve the __trait syntax. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 05 2010
next sibling parent Don <nospam nospam.com> writes:
Michel Fortin wrote:
 On 2010-05-05 21:12:47 -0400, Walter Bright <newshound1 digitalmars.com> 
 said:
 
 Michel Fortin wrote:
 If even Walter has difficulty figuring out the ! around __traits,

Missing a ! is always a problem. People even do not see the word "not".

It was an exaggeration. That "ultimate proof" wasn't meant to be too serious. I hope it didn't stop you from reading the rest of the post, because it seemed like a nice idea to improve the __trait syntax.

May 05 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Leandro Lucarella:
 Please, remove the leading __ from features that are standard (as oposed
 from implementation extensions). 

I appreciate Walter's decision to use those __names: it allows us to use and try a feature now, it allows to create its implementation progressively, and gives time to design its user interface well after it's being used for some time. Bye, bearophile
May 06 2010
prev sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
Walter Bright wrote:
 Michel Fortin wrote:
 If even Walter has difficulty figuring out the ! around __traits,

Missing a ! is always a problem. People even do not see the word "not".

I think the problem is confusing this: static assert(!__traits(compiles, Range!(15, 3, 0) )); with this: static assert!(__traits(compiles, Range!(15, 3, 0) )); Probably a little obvious in this case, but for this: foo!(bar, baz) foo(!bar, baz) it gets trickier... hmmm... very bug prone :-P
May 07 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Michel Fortin:
 [...] I'll take that as the ultimate proof that [...]

It's not an ultimate proof. Bye, bearophile
May 05 2010
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
bearophile, el  6 de mayo a las 12:43 me escribiste:
 Leandro Lucarella:
 Please, remove the leading __ from features that are standard (as oposed
 from implementation extensions). 

I appreciate Walter's decision to use those __names: it allows us to use and try a feature now, it allows to create its implementation progressively, and gives time to design its user interface well after it's being used for some time.

Yes, but that are now all established features. It's OK to throw them in the wild for the first time, but when they prove themselves successful and re here to stay, and the language is close to be finished, I think it makes no sense to leave the leading __, when it has a long history of being used for non-standard extensions (not to mention the uglyness that they bring to the code). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- El discman vuelve locos a los controles, te lleva a cualquier lugar. Ajústense pronto los cinturones, nos vamos a estrellar. Evidentemente, no escuchaste el speech, que dio la azafata, antes de despegar.
May 06 2010
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-05-05 14:04:13 -0400, Walter Bright <newshound1 digitalmars.com> said:

 bearophile wrote:
 I have to use a syntax like this often enough inside
 unittests:
 
 static assert(!__traits(compiles, foo(10)));

But why? Just use: foo(10);

To put it more simply than bearophile: foo(10); // gives you an error when foo(10) does *not* compile static assert(!__traits(compiles, foo(10))); // gives you an error when foo(10) compiles with no error // (notice the negation operator "!") It is sometime a good idea to assert that a template cannot be instantiated with bogus arguments. -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 05 2010
parent dennis luehring <dl.soluz gmx.net> writes:
 It is sometime a good idea to assert that a template cannot be
 instantiated with bogus arguments.

or to be more generic - do not only test your untis with "working" scenarios, the "non-working" are also part of the test game :-)
May 05 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Adam D. Ruppe:
 	static assert(!__traits(compiles, octal!"not a number"));
 [...]
 So I see the use, but I don't think any special syntax is required. This
 works well enough.

Improving that syntax is not necessary, but in my dlibs1 I have to use something like that (using is()) for about 70 KB of templates, it gets tiring and it's easy to make mistakes :-) So even if it's not necessary, it can be handy for me. My post about unit test features contains other things worth discussing about. This was just the first point :-) Bye, bearophile
May 05 2010
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-05-05 14:24:37 -0400, bearophile <bearophileHUGS lycos.com> said:

 My post about unit test features contains other things worth discussing 
 about. This was just the first point :-)

Indeed. I agree it'd be handy to have named unit tests. I see two reasons to use named unit tests: 1. to print a list of all the tests as they run 2. to print a list of the tests that fails Currently the output of a unittest is to print the first assertion that fails. I think that's a good default, but if you add a name to your unit test it can give you more context when it fails. Here's a way to do it with the current syntax: unittest { scope (failure) writeln("Calculate pi using method X: FAIL"); auto result = methodX(); assert(result == 3.1416); } If the test fails, you'll get this output: Calculate pi using method X: FAIL file.d(5): assertion failure pi == 3.1416 which is a noticeable improvement because you don't have to go and look at the file to know what this test is about. If the test pass, it'll will output nothing. Whether we want to output a line for every test, I'm not sure. D encourages small unit tests scattered all around the place, and I'd be worried that being too verbose when the tests are run would discourage people from running the tests in the first place. On the other hand, it's useful, especially when a test hangs, takes too long, or crashes the program, to be able to see the list of all the tests as they run. So what I would suggest is two things. A way to attach a name to a unit test, like this: unittest "Calculate pi using method x" { ... } This is better than scope (failure) because the test runner is now in charge of deciding what to do with each test. I'd suggest that the runtime print the name of a test when it fails: Calculate pi using method X: FAIL file.d(5): assertion failure pi == 3.1416 If the environment variable D_VERBOSE_UNITTEST is set when the program is run, the runtime could print the name of each test as it executes the test, followed by "PASS" upon successful completion or "FAIL" on failure: Calculate e using method X: PASS Calculate pi using method X: FAIL file.d(5): assertion failure pi == 3.1416 The environment variable makes sure that no one is bothered by a long list of tests unless they explicitly ask for it. When you want to see which tests are run and get a general feel of the time they take. Any other use for named unit tests? -- Michel Fortin michel.fortin michelf.com http://michelf.com/
May 05 2010
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
Michel Fortin, el  5 de mayo a las 22:12 me escribiste:
 On 2010-05-05 21:12:47 -0400, Walter Bright <newshound1 digitalmars.com> said:
 
Michel Fortin wrote:
If even Walter has difficulty figuring out the ! around __traits,

Missing a ! is always a problem. People even do not see the word "not".

It was an exaggeration. That "ultimate proof" wasn't meant to be too serious. I hope it didn't stop you from reading the rest of the post, because it seemed like a nice idea to improve the __trait syntax.

Please, remove the leading __ from features that are standard (as oposed from implementation extensions). Is really ugly and make you feel it won't work in another compiler! This goes too for __gshared (and I don't remember if there is anything else). -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ ---------------------------------------------------------------------- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) ---------------------------------------------------------------------- Un barco con animales, amigos míos. Creyendo salvarse de la inundación. Mientras yo, sufro la ruptura de mi corazón.
May 06 2010
prev sibling next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wed, May 05, 2010 at 11:04:13AM -0700, Walter Bright wrote:
 But why? Just use:
 
    foo(10);

I used the static assert not compiles thing for the octal template - unittest{ static assert(!__traits(compiles, octal!"not a number")); // and checking that it doesn't convert implicitly long a; static assert(!__traits(compiles, a = octal!"7777777777777"); } That kind of thing - makes sure it gave the right types for the right input. So I see the use, but I don't think any special syntax is required. This works well enough. -- Adam D. Ruppe http://arsdnet.net
May 05 2010
prev sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Thu, 06 May 2010 04:14:27 +0200, Don wrote:
 Michel Fortin wrote:
 I hope it didn't stop you from reading the rest of the post, because it
 seemed like a nice idea to improve the __trait syntax.
 


This is now the fourth highest voted bug in Bugzilla. Just sayin'... ;) -Lars
May 05 2010