www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - dmd testsuite naming scheme

reply Trass3r <un known.com> writes:
Is there any pattern in the testsuite organization? There are loads of  
test[0-9]+. files etc. And folders are only used to group  
compilable/runnable...
I honestly wouldn't know where to add or search for a test case.
Jan 02 2012
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/2/2012 9:58 AM, Trass3r wrote:
 Is there any pattern in the testsuite organization?

No.
 There are loads of
 test[0-9]+. files etc. And folders are only used to group
compilable/runnable...
 I honestly wouldn't know where to add or search for a test case.

It doesn't really matter where they go. A collection of test cases with a theme to them might go in a named file, a random one might be appended to any of the test* files.
Jan 02 2012
next sibling parent reply "Trass3r" <un known.com> writes:
 I honestly wouldn't know where to add or search for a test 
 case.

cases with a theme to them might go in a named file, a random one might be appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run this isn't a desirable trend imho.
Jan 03 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.

with a theme to them might go in a named file, a random one might be appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run this isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it often happens that a tiny difference between two seemingly equal tests can be all that matters. On the other hand, grouping tests into files based on language features might be a good idea. If anything, to be able to navigate the test suite. - Alex
Jan 03 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 03-01-2012 16:44, Martin Nowak wrote:
 On Tue, 03 Jan 2012 15:39:34 +0100, Alex Rønne Petersen
 <xtzgzorex gmail.com> wrote:

 On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.

with a theme to them might go in a named file, a random one might be appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run this isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it often happens that a tiny difference between two seemingly equal tests can be all that matters. On the other hand, grouping tests into files based on language features might be a good idea. If anything, to be able to navigate the test suite. - Alex

There is some opportunity in creating systematic feature tests backed with coverage analysis. There are still too many uncovered areas. This not only helps to find remaining bugs but gives a specification like overview of a feature state.

I still say D needs a formal specification more than a test suite as some kind of excuse for a specification. (And no, I don't consider d-p-l.org a spec; a guide at best.) - Alex
Jan 03 2012
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/3/12 12:57 PM, Alex Rønne Petersen wrote:
 On 03-01-2012 16:44, Martin Nowak wrote:
 On Tue, 03 Jan 2012 15:39:34 +0100, Alex Rønne Petersen
 <xtzgzorex gmail.com> wrote:

 On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.

with a theme to them might go in a named file, a random one might be appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run this isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it often happens that a tiny difference between two seemingly equal tests can be all that matters. On the other hand, grouping tests into files based on language features might be a good idea. If anything, to be able to navigate the test suite. - Alex

There is some opportunity in creating systematic feature tests backed with coverage analysis. There are still too many uncovered areas. This not only helps to find remaining bugs but gives a specification like overview of a feature state.

I still say D needs a formal specification more than a test suite as some kind of excuse for a specification. (And no, I don't consider d-p-l.org a spec; a guide at best.) - Alex

Agreed. Andrei
Jan 03 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Tue, 03 Jan 2012 15:39:34 +0100, Alex R=C3=B8nne Petersen  =

<xtzgzorex gmail.com> wrote:

 On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.

with a theme to them might go in a named file, a random one might be=



 appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run t=


 isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it =

 often happens that a tiny difference between two seemingly equal tests=

 can be all that matters.

 On the other hand, grouping tests into files based on language feature=

 might be a good idea. If anything, to be able to navigate the test sui=

 - Alex

There is some opportunity in creating systematic feature tests backed with coverage analysis. There are still too many uncovered areas. This not only helps to find remaining bugs but gives a specification like overview of a feature state.
Jan 03 2012
prev sibling next sibling parent Gor Gyolchanyan <gor.f.gyolchanyan gmail.com> writes:
Definitely!

+1

On Tue, Jan 3, 2012 at 11:18 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:
 On 1/3/12 12:57 PM, Alex R=C3=B8nne Petersen wrote:
 On 03-01-2012 16:44, Martin Nowak wrote:
 On Tue, 03 Jan 2012 15:39:34 +0100, Alex R=C3=B8nne Petersen
 <xtzgzorex gmail.com> wrote:

 On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.

It doesn't really matter where they go. A collection of test cases with a theme to them might go in a named file, a random one might be appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run this isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it often happens that a tiny difference between two seemingly equal tests can be all that matters. On the other hand, grouping tests into files based on language features might be a good idea. If anything, to be able to navigate the test suite. - Alex

There is some opportunity in creating systematic feature tests backed with coverage analysis. There are still too many uncovered areas. This not only helps to find remaining bugs but gives a specification like overview of a feature state.

I still say D needs a formal specification more than a test suite as some kind of excuse for a specification. (And no, I don't consider d-p-l.org a spec; a guide at best.) - Alex

Agreed. Andrei

--=20 Bye, Gor Gyolchanyan.
Jan 04 2012
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Tue, 03 Jan 2012 19:57:14 +0100, Alex R=C3=B8nne Petersen  =

<xtzgzorex gmail.com> wrote:

 On 03-01-2012 16:44, Martin Nowak wrote:
 On Tue, 03 Jan 2012 15:39:34 +0100, Alex R=C3=B8nne Petersen
 <xtzgzorex gmail.com> wrote:

 On 03-01-2012 13:36, Trass3r wrote:
 I honestly wouldn't know where to add or search for a test case.






 with a theme to them might go in a named file, a random one might =





 appended to any of the test* files.

Won't this potentially lead to test duplication? Considering that the testsuite already takes quite some time to run=




 this
 isn't a desirable trend imho.

Test duplication isn't necessarily a bad thing. In my experience, it=



 often happens that a tiny difference between two seemingly equal tes=



 can be all that matters.

 On the other hand, grouping tests into files based on language
 features might be a good idea. If anything, to be able to navigate t=



 test suite.

 - Alex

There is some opportunity in creating systematic feature tests backed=


 with coverage analysis. There are still too many uncovered areas.
 This not only helps to find remaining bugs but gives a specification
 like overview of a feature state.

I still say D needs a formal specification more than a test suite as =

 some kind of excuse for a specification. (And no, I don't consider  =

 d-p-l.org a spec; a guide at best.)

 - Alex

I'd still like to see that the website, language specification and specification tests become the same.
Jan 05 2012