www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DStress thoughts

reply Stewart Gordon <smjg_1998 yahoo.com> writes:
1. .html files are served as text/plain, presumably to cause the HTML 
code to be displayed in the browser rather than rendered as an HTML 
page.  Shouldn't .d files have the same treatment?  With them as 
text/x-d-source, Mozilla just assumes it's a format it can't do anything 
with, and so I have to use the view-source: trick to see it.

2. The expected/actual results could be refined a bit.  A possible set 
of expected results:

- Run without error
- Exception (could include AssertError)
- Compile only (for tests that it doesn't make any sense to try running, 
e.g. my submission http://svn.kuehne.cn/dstress/compile/bug_e2ir_1158_A.d)
- Linker error
- Compiler error

Actual results would include as well as the above:

- Pass
- Runtime crash
- Compiler internal error/crash/segfault

There would also be 'expected output' and 'actual output' stuff where 
applicable, be it actual output or compiler errors.  A result of 'pass' 
would mean that the actual output matches the expected output exactly, 
whereas (expected result == actual result) could be true if they don't 
match.

For that matter, are we supposed to be able to find the PASS, XFAIL, 
etc. results that we're told about somewhere?

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Dec 06 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:
 1. .html files are served as text/plain

All html files should be served as text/html - the only exception are some html_encoding* / html_line_ending* tests. a test: wget http://svn.kuehne.cn/dstress/run/html_comment_01.html # 1 HTTP/1.1 200 OK # 2 Date: Mon, 06 Dec 2004 13:30:17 GMT # 3 Server: Apache/2.0.52 (Debian GNU/Linux) DAV/2 SVN/1.0.8 # 4 ETag: "140//dstress/run/html_comment_01.html" # 5 Accept-Ranges: bytes # 6 Content-Length: 215 # 7 Content-Type: text/html Please provide me with the URLs of any html file that are served incorrectly.
 With them as text/x-d-source, Mozilla just assumes it's a format it
 can't do anything with, and so I have to use the view-source: trick to
 see it.

On Linux, Mac, Window you can define a system wide mime and attach a default application to it - e.g IDE, editor etc. I'm not a regular Mozilla user but will inquire.
 2. The expected/actual results could be refined a bit.  A possible set 
 of expected results:

 - Run without error
 - Exception (could include AssertError)
 - Compile only (for tests that it doesn't make any sense to try running, 
 e.g. my submission http://svn.kuehne.cn/dstress/compile/bug_e2ir_1158_A.d)
 - Linker error
 - Compiler error

 Actual results would include as well as the above:

 - Pass
 - Runtime crash
 - Compiler internal error/crash/segfault

 There would also be 'expected output' and 'actual output' stuff where 
 applicable, be it actual output or compiler errors.  A result of 'pass' 
 would mean that the actual output matches the expected output exactly, 
 whereas (expected result == actual result) could be true if they don't 
 match.

see summary / readme: http://svn.kuehne.cn/dstress/readme.txt http://svn.kuehne.cn/dstress/www/dstress.html To keep it simple: * should compile - http://svn.kuehne.cn/dstress/compile/ * shouldn't compile - http://svn.kuehne.cn/dstress/nocompile/ * should run - http://svn.kuehne.cn/dstress/run/ * shouldn't run - http://svn.kuehne.cn/dstress/norun/
 For that matter, are we supposed to be able to find the PASS, XFAIL, 
 etc. results that we're told about somewhere?

combined chart - http://svn.kuehne.cn/dstress/www/dstress.html individual listing - http://svn.kuehne.cn/dstress/raw_results/ A hint: Use Subversion to stay up to date, see log messages etc. Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.9.13 (GNU/Linux) iD8DBQFBtFNV3w+/yD4P9tIRAo7/AJ4iqKCBGKjg8TDXxF8USTBByYTv4gCgzAOm jEbRG54CQKXz6OHF4JfUXZs= =8uBd -----END PGP SIGNATURE-----
Dec 06 2004
next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Thomas Kuehne wrote:
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:
 
1. .html files are served as text/plain

All html files should be served as text/html - the only exception are some html_encoding* / html_line_ending* tests.

 Please provide me with the URLs of any html file that are served incorrectly.

Not sure if these count: http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_01.html http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_02.html http://svn.kuehne.cn/dstress/nocompile/html_empty_entity_03.html But you're right, the rest seem to be either served as text/html or encoding issues.
With them as text/x-d-source, Mozilla just assumes it's a format it
can't do anything with, and so I have to use the view-source: trick to
see it.

On Linux, Mac, Window you can define a system wide mime and attach a default application to it - e.g IDE, editor etc.

ATM I can't see this as enabling a link I click in Mozilla to be displayed there and then. But maybe it's a Mozilla problem rather a DStress problem.... <snip>
 To keep it simple:
 * should compile - http://svn.kuehne.cn/dstress/compile/
 * shouldn't compile - http://svn.kuehne.cn/dstress/nocompile/
 * should run -  http://svn.kuehne.cn/dstress/run/
 * shouldn't run - http://svn.kuehne.cn/dstress/norun/

I see.
For that matter, are we supposed to be able to find the PASS, XFAIL, 
etc. results that we're told about somewhere?

combined chart - http://svn.kuehne.cn/dstress/www/dstress.html individual listing - http://svn.kuehne.cn/dstress/raw_results/

Excellent. But it would be helpful to work out where there are platform-dependent failures - I've noticed that you have one or two Mac results but no Windows results. Maybe I'll take your tests home and try them.... BTW I noticed that you somehow miscopied my name on a few tests (at least the abstract ones). Stewart. -- My e-mail is valid but not my primary mailbox. Please keep replies on the 'group where everyone may benefit.
Dec 06 2004
next sibling parent reply Brad Anderson <brad dsource.dot.org> writes:
Stewart Gordon wrote:
 Thomas Kuehne wrote:
 
 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1

 Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:

 1. .html files are served as text/plain

All html files should be served as text/html - the only exception are some html_encoding* / html_line_ending* tests.


This may help, unless the html files are meant to be text/plain for testing purposes. It will recursively (current dir) set the mime-type to text/html in Subversion for all html files. # svn propset -R svn:mime-type text/html *.html BA
Dec 06 2004
parent Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brad Anderson schrieb am Mon, 06 Dec 2004 09:12:01 -0600:
 1. .html files are served as text/plain

All html files should be served as text/html - the only exception are some html_encoding* / html_line_ending* tests.


This may help, unless the html files are meant to be text/plain for testing purposes. It will recursively (current dir) set the mime-type to text/html in Subversion for all html files. # svn propset -R svn:mime-type text/html *.html

Thanks, I just somehow missed them when marking all the file properties. Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.9.13 (GNU/Linux) iD8DBQFBtI8+3w+/yD4P9tIRAuMdAKC1gFNKkPx+zcH6OTzLPDHzO8HaJgCeLTtQ LIPTaEzUf8pf8kVwOrRR+mc= =HEcJ -----END PGP SIGNATURE-----
Dec 06 2004
prev sibling parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stewart Gordon schrieb am Mon, 06 Dec 2004 14:10:36 +0000:
With them as text/x-d-source, Mozilla just assumes it's a format it
can't do anything with, and so I have to use the view-source: trick to
see it.

On Linux, Mac, Window you can define a system wide mime and attach a default application to it - e.g IDE, editor etc.

ATM I can't see this as enabling a link I click in Mozilla to be displayed there and then. But maybe it's a Mozilla problem rather a DStress problem....

https://bugzilla.mozilla.org/show_bug.cgi?id=57342 https://bugzilla.mozilla.org/show_bug.cgi?id=196078
 But it would be helpful to work out where there are 
 platform-dependent failures - I've noticed that you have one or two Mac 
 results but no Windows results.  Maybe I'll take your tests home and try 
 them....

You are welcome! There are several problems: 1) There is no Windows box within my reach *g* 2) The Makefile uses lot's of tiny GNU tools(grep, cat, ...). JCC has started to write a bat generator (http://svn.kuehne.cn/dstress/beta/dstresstest.d) to solve parts of this problem. 3) By producing an error pop-up the segfaulting test cases pose a hurdle for automated bat files.
 BTW I noticed that you somehow miscopied my name on a few tests (at 
 least the abstract ones).

Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.9.13 (GNU/Linux) iD8DBQFBtI7f3w+/yD4P9tIRAvLsAKCO3Mv2cyOS0sweZs83SWpmiHCLlgCfZdDi IC0eVoQV787jQRZrMKHZ++Q= =EiKm -----END PGP SIGNATURE-----
Dec 06 2004
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 6 Dec 2004 17:54:56 +0100, Thomas Kuehne  
<thomas-dloop kuehne.thisisspam.cn> wrote:
 There are several problems:

 1) There is no Windows box within my reach *g*

Have you heard of: http://www.vmware.com/products/desktop/ws_features.html If you can get your hands on a copy of that I recommend it. Basically it allows you to install any OS into a file on your HDD and then you can boot and run that OS any time you want. You can run several at once, and telnet to/from/between them all. Regan
Dec 06 2004
prev sibling next sibling parent reply Georg Wrede <Georg_member pathlink.com> writes:
In article <mj0e82-kt4.ln1 kuehne.cn>, Thomas Kuehne says...
Stewart Gordon schrieb am Mon, 06 Dec 2004 11:35:02 +0000:
 1. .html files are served as text/plain

some html_encoding* / html_line_ending* tests.

I wish the D tests there were in html, formatted with nice colors. It is unreasonable to have each person reading that stuff to adjust the browser on each machine he uses (whether regularly, or just once to show a specific point to somebody at her computer). What makes this request the more reasonable is that D boasts being "html-compatible". So anybody could just download the pretty html and the D compiler will accept that as input. Oh, and I'd like the motivation for having xpass and xfail to be explained somewhere prominently. Like, what the "unexpected" is? Not per test, but in general. There are now test that are xpass version after version, and the general reader might wonder how come this remains unexpected. .-) (I'm talking about http://svn.kuehne.cn/dstress/www/dstress.html)
Dec 06 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):
 I wish the D tests there were in html, formatted with nice colors.
 It is unreasonable to have each person reading that stuff to adjust
 the browser on each machine he uses (whether regularly, or just once
 to show a specific point to somebody at her computer).

 What makes this request the more reasonable is that D boasts being
 "html-compatible". So anybody could just download the pretty html
 and the D compiler will accept that as input.

Have a look at the HTML tests: http://svn.kuehne.cn/dstress/www/dstress.html#html_comment_01 and scroll down. FYI I have send Walter several patches that solve most of the HTML FAIL and XPASS tests.
 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.

Are you talking about an abstract like in the readme? If not, suggest some general explanations.
 There are now test that are xpass version after version, and the
 general reader might wonder how come this remains unexpected.  .-)

How come there are long standing issues with almost all software projects? Some ideas: * The documentation isn't inline with the implementation. * Fixes with scores of dependencies / implications. * Bugs that aren't on the priority list. ... The D frontend is open source and the gcc backend is open source too, thus: If you feel you need a bug fix and it hasn't be tackled by the maintainer(big bright W) you might: a) fix it yourself and send a patch b) pay someone to do the work for you c) send a petition *g* Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.9.13 (GNU/Linux) iD8DBQFBtLDE3w+/yD4P9tIRAlNaAJ4keqMh9LNUz7gyDsQoRGADJwYDogCeILLD 1PSNZkRKtGJojbVc005rcv8= =aNax -----END PGP SIGNATURE-----
Dec 06 2004
next sibling parent Georg Wrede <Georg_member pathlink.com> writes:
In article <5vne82-dk2.ln1 kuehne.cn>, Thomas Kuehne says...
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):
 I wish the D tests there were in html, formatted with nice colors.
 It is unreasonable to have each person reading that stuff to adjust
 the browser on each machine he uses (whether regularly, or just once
 to show a specific point to somebody at her computer).

 What makes this request the more reasonable is that D boasts being
 "html-compatible". So anybody could just download the pretty html
 and the D compiler will accept that as input.

Have a look at the HTML tests: http://svn.kuehne.cn/dstress/www/dstress.html#html_comment_01 and scroll down.

Yes! If only we could get those above it into html too. :-)
FYI I have send Walter several patches that solve most of the HTML FAIL and
XPASS tests.

Few can have missed the value your work brings to D!
 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.

Are you talking about an abstract like in the readme? If not, suggest some general explanations.

I formulated myself rather badly, sorry. On the same page, under subheading "^Legend", I'd like to see a definition of what constitutes pass, xpass, fail, and xfail. The others (error, and not tested) are self-explanatory. There is a reference to the DejaGnu test framework, but a smallish forage there gave nothing concrete in hand. Also the POSIX standard 1003.3 seems to be for paid-only readers. Pass and fail were easy if they were the only ones. But definition of "unexpected" (for both) is what I'd like to see right there. To sum it up, all that's needed is an answer to "why do we have the category xpass, and why the category xfail?"
Dec 07 2004
prev sibling parent reply Georg Wrede <Georg_member pathlink.com> writes:
In article <5vne82-dk2.ln1 kuehne.cn>, Thomas Kuehne says...
-----BEGIN PGP SIGNED MESSAGE-----

Georg Wrede schrieb am Mon, 6 Dec 2004 18:11:28 +0000 (UTC):
 Oh, and I'd like the motivation for having xpass and xfail to be 
 explained somewhere prominently. Like, what the "unexpected" is?
 Not per test, but in general.


Maybe I'm getting slow, but I haven't figured out why xfail is light GREEN? One would assume that the colors were in order. So, out of the 4 alternatives (pass, fail, xpass, xfail) the worst ( = most bad) result would be VERY RED. The next "baddest" RED, and the least bad would be LIGHT RED, or orange. Pass of course has to be green. Oh, and the point-of-view here should be that of the compiler writer (and the community at large), not that of the test writer. (No disrespect intended!) The reason being that they are the ones who (hopefully) choose what to tackle next. I don't feel qualified to even suggest a particular ordering here, be it by difficulty of fixing, "unexpectedness"(?), priority for D 1.0, or whatever else. ----- ps, using the same prefix for expected and unexpected may go unnoticed for quite some time -- for readers like me, at least. While writing this, I took another hard look at the coloring, and I started to suspect that the light green for expected- failure is because the test writer is happy: the test gave the result he intended. (So presumably at least the test itself is ok.) I sure hope I'm wrong.
Dec 07 2004
parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Wrede schrieb am Tue, 7 Dec 2004 08:57:02 +0000 (UTC):
 One would assume that the colors were in order. So, out of the 4
 alternatives (pass, fail, xpass, xfail) the worst ( = most bad)
 result would be VERY RED. The next "baddest" RED, and the least
 bad would be LIGHT RED, or orange. Pass of course has to be green.

That's the intention.
 Oh, and the point-of-view here should be that of the compiler
 writer (and the community at large), not that of the test writer.
 (No disrespect intended!) The reason being that they are the ones
 who (hopefully) choose what to tackle next.

I'm trying to use community standards. * http://www.cs.utah.edu/dept/old/texinfo/dejagnu/dejagnu_2.html * http://gcc.gnu.org/install/test.html#TOC3 * http://www.kerneltraffic.org/wine/wn20020105_112_print.html#1 (search for XFAIL)
 While writing this, I took another hard look at the coloring,
 and I started to suspect that the light green for expected-
 failure is because the test writer is happy: the test gave
 the result he intended. (So presumably at least the test itself
 is ok.) I sure hope I'm wrong.

When all hope was gone only the Suspicion remained *g* Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.9.13 (GNU/Linux) iD8DBQFBtYBc3w+/yD4P9tIRAosYAKC7iMMY/W0+aVdJqkGlhtV0IRxC/QCeLQFy L2nWXcEyodp1zi/rDil/Vl4= =F4DS -----END PGP SIGNATURE-----
Dec 07 2004
parent Georg Wrede <Georg_member pathlink.com> writes:
In article <srbg82-a93.ln1 kuehne.cn>, Thomas Kuehne says...
When all hope was gone only the Suspicion remained *g*

Thanks, now the Legend seems much clearer!! georg
Dec 10 2004
prev sibling parent Mike Parker <aldacron71 yahoo.com> writes:
Thomas Kuehne wrote:
 On Linux, Mac, Window you can define a system wide mime and attach a
 default application to it - e.g IDE, editor etc.

But don't you find it much more convenient when you can see a source file in your browser? It's rather annoying to switch between apps. ViewCVS serves source files as plain text, many personal pages I've seen serve source files as plain text. I always get peeved at Sun because some Java sources on their site are served as plain text while others aren't. I suppose it's a matter of just configuring the mime type to be used with the browser rather than an editor. But it seems to me much simpler to just serve it as plain text, and if someone wants to open it somewhere other than the browser they can right-click->saveas.
Dec 06 2004