www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: DMD needs branches

reply Dan <murpsoft hotmail.com> writes:
Walter Bright Wrote:

 The test suite is run before every new release. If things break with a 
 new release, it is because the case isn't reflected in the test suite. 
 Every fixed bug goes in to the test suite. For example, Kris posted two 
 things that broke with the 1.011 update. Both are fixed now, and both 
 are now in the test suite. They'll stay fixed.
 
 Over time, the suite has a ratchet effect, with things getting better 
 and better. I've been using that system for decades with the C++ 
 compiler, and it's pretty rare for an update to break anything.
 
 But if bugs aren't reported, then they don't get fixed, and the test 
 case never winds up in the test suite.
 
 The only way to get a stable system is to report bugs, fix them, and put 
 the cases in the test suite. I tend to put priority on fixing things 
 that break existing code; I know how maddening that can be.



Hmm... I actually like the method to his madness.  : )

If we provide a means by which D can be thoroughly tested through and through,
then each version that he writes must conform to that test?

What if we were to then write a test suite that enforced proveable conformance
to the D specification for 1.0?  Evolving a truly complete test suite seems
more sensible than endlessly adding bug-able examples...
Apr 13 2007
parent reply Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dan schrieb am 2007-04-13:
[snip]

 What if we were to then write a test suite that enforced provable conformance
 to the D specification for 1.0?  Evolving a truly complete test suite seems 
 more sensible than endlessly adding bug-able examples...



You are welcome to do so. However the only sensible way to do so would
be to use formal specification to generate the tests. While this is a
relatively trivial task for the syntax tests (especially the no-compile
target) runtime tests will require a serious amount of work.

DStress[1] at least (don't know about Walter's test suite) contains quite a
number of pro-active test cases. I simply haven't gotten around to add
tests for the post DMD-1.004 features.

Another problem is the processing power required to run those tests.
e.g. the DMD-1.009 run contained 6801 test cases. Most of them were run
in 32 different configuration ("-O", "-inline", "-O -inline", etc.)
resulting in 213324 tests. In total that resulted in ca. 214000 compiler
invocations, 138000 linker invocations and ca. 137500 executions of
newly generated executables.

Thomas

[1] http://dstress.kuehne.cn

-----BEGIN PGP SIGNATURE-----

iD8DBQFGIK11LK5blCcjpWoRAtdiAJ4zftd2EdejPrH/9d4H4qwNwcVJTwCfY10D
I7E+6myEksiY/zwpV0ekN2U=
=BOPN
-----END PGP SIGNATURE-----
Apr 14 2007
next sibling parent reply "David B. Held" <dheld codelogicconsulting.com> writes:
Thomas Kuehne wrote:

 [...]
 Another problem is the processing power required to run those tests.
 e.g. the DMD-1.009 run contained 6801 test cases. Most of them were run
 in 32 different configuration ("-O", "-inline", "-O -inline", etc.)
 resulting in 213324 tests. In total that resulted in ca. 214000 compiler
 invocations, 138000 linker invocations and ca. 137500 executions of
 newly generated executables.
 [...]



C'mon, Thomas!  Quit making excuses and overclock that Commodore 64 
already!!

Dave
Apr 14 2007
parent reply Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David B. Held schrieb am 2007-04-14:

 Thomas Kuehne wrote:

 [...]
 Another problem is the processing power required to run those tests.
 e.g. the DMD-1.009 run contained 6801 test cases. Most of them were run
 in 32 different configuration ("-O", "-inline", "-O -inline", etc.)
 resulting in 213324 tests. In total that resulted in ca. 214000 compiler
 invocations, 138000 linker invocations and ca. 137500 executions of
 newly generated executables.
 [...]



 C'mon, Thomas!  Quit making excuses and overclock that Commodore 64 
 already!!



My test system is Turion 64 clocked at 1.6GHz with 2GB RAM. However the
tests only run with "nice -n 18" in the background thus donating hardware
might help. And yes, the testsuite has no trouble to keep all cores
in a N-cpu system (with 1 <= N <= 6000) busy.

;-P

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFGIS/yLK5blCcjpWoRAkfCAJ9IlkaaS1Pfhqmt9RpYND8txpbdTQCgoKYc
TXW3uGunf+hhBuO4JPOWD0M=
=jRXT
-----END PGP SIGNATURE-----
Apr 14 2007
parent reply BCS <ao pathlink.com> writes:
Reply to Thomas,


 My test system is Turion 64 clocked at 1.6GHz with 2GB RAM. However
 the
 tests only run with "nice -n 18" in the background thus donating
 hardware
 might help. And yes, the testsuite has no trouble to keep all cores
 in a N-cpu system (with 1 <= N <= 6000) busy.
 ;-P
 
 Thomas




BTW how long does it take dstress to run? I ask because I have a lexer that 
is able to lex almost all of the compile branch (~4700 files) in just over 
a minute. I'm wondering how this compares to DMD.
Apr 14 2007
parent Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

BCS schrieb am 2007-04-14:

 BTW how long does it take dstress to run? I ask because I have a lexer that 
 is able to lex almost all of the compile branch (~4700 files) in just over 
 a minute. I'm wondering how this compares to DMD.



I've send you a stubbed DMD-1.011 frontend (too large for posting here).

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFGIdK1LK5blCcjpWoRAtLUAJ0fP7XTJSSwo2Cnk9IpJzXA8TqrtgCgipSs
sc9JIKFjyB531b60LQlsERk=
=Hlvc
-----END PGP SIGNATURE-----
Apr 15 2007
prev sibling parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
Thomas Kuehne wrote:

 Another problem is the processing power required to run those tests.
 e.g. the DMD-1.009 run contained 6801 test cases. Most of them were run
 in 32 different configuration ("-O", "-inline", "-O -inline", etc.)
 resulting in 213324 tests. In total that resulted in ca. 214000 compiler
 invocations, 138000 linker invocations and ca. 137500 executions of
 newly generated executables.



How long does it take for all tests to complete? Big difference between 
GDC/DMD?

L.
Apr 15 2007
parent reply Thomas Kuehne <thomas-dloop kuehne.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lionello Lunesu schrieb am 2007-04-16:

 Thomas Kuehne wrote:

 Another problem is the processing power required to run those tests.
 e.g. the DMD-1.009 run contained 6801 test cases. Most of them were run
 in 32 different configuration ("-O", "-inline", "-O -inline", etc.)
 resulting in 213324 tests. In total that resulted in ca. 214000 compiler
 invocations, 138000 linker invocations and ca. 137500 executions of
 newly generated executables.



 How long does it take for all tests to complete? Big difference between 
 GDC/DMD?



If the testsuite is the only running application and neglection dead
locks: ~ 25 hours for DMD, ~ 36 hours for GDC

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFGIz9OLK5blCcjpWoRAkDcAJ9aY6qITQvtED3Ox6j+3+QiXH4wWACgmPad
lWbczCH+9Jfzg2QZHbBtrzc=
=J6/U
-----END PGP SIGNATURE-----
Apr 16 2007
parent reply Dan <murpsoft hotmail.com> writes:
Thomas Kuehne Wrote:

 How long does it take for all tests to complete? Big difference between 
 GDC/DMD?



 If the testsuite is the only running application and neglection dead
 locks: ~ 25 hours for DMD, ~ 36 hours for GDC



Holy moley batman!  I didn't know D had enough source to take that long to do
*anything* to it.  I could almost manually retype my entire scripting engine in
that much time!
Apr 16 2007
parent reply Sean Kelly <sean f4.ca> writes:
Dan wrote:

 Thomas Kuehne Wrote:

 How long does it take for all tests to complete? Big difference between 
 GDC/DMD?



 locks: ~ 25 hours for DMD, ~ 36 hours for GDC



 Holy moley batman!  I didn't know D had enough source to take that long to do
*anything* to it.  I could almost manually retype my entire scripting engine in
that much time!



Sounds like D needs a DDStress client, kinda like SETI Home ;-)


Sean
Apr 16 2007
parent Pragma <ericanderton yahoo.removeme.com> writes:
Sean Kelly wrote:

 Dan wrote:

 Thomas Kuehne Wrote:

 How long does it take for all tests to complete? Big difference 
 between GDC/DMD?



 locks: ~ 25 hours for DMD, ~ 36 hours for GDC



 Holy moley batman!  I didn't know D had enough source to take that 
 long to do *anything* to it.  I could almost manually retype my entire 
 scripting engine in that much time!



 Sounds like D needs a DDStress client, kinda like SETI Home ;-)
 
 
 Sean



Would that work?

0) download client
1) client runs and syncs with stress-test repository
1.1) client syncs compiler distribution(s) for needed tests
1.2) client gets list of incomplete test cases
1.3) client syncs stress-test code for needed test cases
2) client runs needed tests
3) client reports results
4) sleep while not idle
5) (system idle) loop to 1

That might work a treat for performance testing of discrete code samples, if
nothing else.

-- 
- EricAnderton at yahoo
Apr 16 2007