www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Benchmark suite

reply bearophile <bearophileHUGS lycos.com> writes:
As you may know recently the Computer Shootout site has dropped many languages,
among them there's D, Psyco (Python), etc. People that like Ruby language (and
Jruby, etc) have created a benchmark suite, useful to tune implementations,
compare their performance, spot some bugs, spot some performance bugs, etc:

http://groups.google.com/group/ruby-benchmark-suite

A similar performance suite may be created for D too, but to not put blinders
on the eyes it's essential to add to such suite some benchmarks in other
languages too, like Java (for the GC), C or C++ with GCC (for general
optimization), and maybe more.

As starting point the programs of the Shoout are fine, but I think few other
things can be added (to benchmark exceptions, compiling time, and some other
interesting things that the Shootout doesn't benchmark now).

Bye,
bearophile
Sep 19 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Sep 19, 2008 at 8:43 AM, bearophile <bearophileHUGS lycos.com> wrote:

 As you may know recently the Computer Shootout site has dropped many
languages, among them there's D, Psyco (Python), etc. People that like Ruby
language (and Jruby, etc) have created a benchmark suite, useful to tune
implementations, compare their performance, spot some bugs, spot some
performance bugs, etc:

 http://groups.google.com/group/ruby-benchmark-suite

 A similar performance suite may be created for D too, but to not put blinders
on the eyes it's essential to add to such suite some benchmarks in other
languages too, like Java (for the GC), C or C++ with GCC (for general
optimization), and maybe more.

 As starting point the programs of the Shoout are fine, but I think few other
things can be added (to benchmark exceptions, compiling time, and some other
interesting things that the Shootout doesn't benchmark now).

 Bye,
 bearophile


I'd imagine the Computer Shootout has only dropped many languages on
the newer platforms since there aren't implementations for them.
Getting GDC to work on x86-64 is a daunting task, or so I've heard.
D's still in all the older benchmarks.
Sep 19 2008
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Jarrett Billingsley:


 I'd imagine the Computer Shootout has only dropped many languages on
 the newer platforms since there aren't implementations for them.


Yep, and probably to reduce the work load. Managing the shootout site isn't a
little work.



 D's still in all the older benchmarks.


But they are dead now: the author has told me that the benchmarks on the old
machine will not be updated.

Bye,
bearophile
Sep 19 2008
prev sibling parent reply Gregor Richards <Richards codu.org> writes:
Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 8:43 AM, bearophile <bearophileHUGS lycos.com> wrote:

 As you may know recently the Computer Shootout site has dropped many
languages, among them there's D, Psyco (Python), etc. People that like Ruby
language (and Jruby, etc) have created a benchmark suite, useful to tune
implementations, compare their performance, spot some bugs, spot some
performance bugs, etc:

 http://groups.google.com/group/ruby-benchmark-suite

 A similar performance suite may be created for D too, but to not put blinders
on the eyes it's essential to add to such suite some benchmarks in other
languages too, like Java (for the GC), C or C++ with GCC (for general
optimization), and maybe more.

 As starting point the programs of the Shoout are fine, but I think few other
things can be added (to benchmark exceptions, compiling time, and some other
interesting things that the Shootout doesn't benchmark now).

 Bye,
 bearophile


 
 I'd imagine the Computer Shootout has only dropped many languages on
 the newer platforms since there aren't implementations for them.
 Getting GDC to work on x86-64 is a daunting task, or so I've heard.
 D's still in all the older benchmarks.


Yeah, getting GDC to work on x86_64 is crazy. First you have to build 
it, then you have to install it! And at that point, you have to run it 
for it to do anything, but how am I supposed to know how you run 
binaries?! Egad, I can't be expected to figure this completely insane 
system out!

  - Gregor Richards
Sep 19 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Sep 19, 2008 at 9:19 AM, Gregor Richards <Richards codu.org> wrote:

 Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 8:43 AM, bearophile <bearophileHUGS lycos.com>
 wrote:

 As you may know recently the Computer Shootout site has dropped many
 languages, among them there's D, Psyco (Python), etc. People that like Ruby
 language (and Jruby, etc) have created a benchmark suite, useful to tune
 implementations, compare their performance, spot some bugs, spot some
 performance bugs, etc:

 http://groups.google.com/group/ruby-benchmark-suite

 A similar performance suite may be created for D too, but to not put
 blinders on the eyes it's essential to add to such suite some benchmarks in
 other languages too, like Java (for the GC), C or C++ with GCC (for general
 optimization), and maybe more.

 As starting point the programs of the Shoout are fine, but I think few
 other things can be added (to benchmark exceptions, compiling time, and some
 other interesting things that the Shootout doesn't benchmark now).

 Bye,
 bearophile


 I'd imagine the Computer Shootout has only dropped many languages on
 the newer platforms since there aren't implementations for them.
 Getting GDC to work on x86-64 is a daunting task, or so I've heard.
 D's still in all the older benchmarks.


 Yeah, getting GDC to work on x86_64 is crazy. First you have to build it,
 then you have to install it! And at that point, you have to run it for it to
 do anything, but how am I supposed to know how you run binaries?! Egad, I
 can't be expected to figure this completely insane system out!

  - Gregor Richards


Snyde demeanor aside, I'm talking more complex problems, like weird
linking issues and binaries that crash on running for no apparent
reason.
Sep 19 2008
parent reply downs <default_357-line yahoo.de> writes:
Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 9:19 AM, Gregor Richards <Richards codu.org> wrote:

 Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 8:43 AM, bearophile <bearophileHUGS lycos.com>
 wrote:

 As you may know recently the Computer Shootout site has dropped many
 languages, among them there's D, Psyco (Python), etc. People that like Ruby
 language (and Jruby, etc) have created a benchmark suite, useful to tune
 implementations, compare their performance, spot some bugs, spot some
 performance bugs, etc:

 http://groups.google.com/group/ruby-benchmark-suite

 A similar performance suite may be created for D too, but to not put
 blinders on the eyes it's essential to add to such suite some benchmarks in
 other languages too, like Java (for the GC), C or C++ with GCC (for general
 optimization), and maybe more.

 As starting point the programs of the Shoout are fine, but I think few
 other things can be added (to benchmark exceptions, compiling time, and some
 other interesting things that the Shootout doesn't benchmark now).

 Bye,
 bearophile


 I'd imagine the Computer Shootout has only dropped many languages on
 the newer platforms since there aren't implementations for them.
 Getting GDC to work on x86-64 is a daunting task, or so I've heard.
 D's still in all the older benchmarks.


 Yeah, getting GDC to work on x86_64 is crazy. First you have to build it,
 then you have to install it! And at that point, you have to run it for it to
 do anything, but how am I supposed to know how you run binaries?! Egad, I
 can't be expected to figure this completely insane system out!

  - Gregor Richards


 
 Snyde demeanor aside, I'm talking more complex problems, like weird
 linking issues and binaries that crash on running for no apparent
 reason.


Works here .. are you perhaps using a certain distribution's binary versions?
Those 4.2 files that are known to crash on trivial testcases?

(If yes: don't do that then)
Sep 19 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Sep 19, 2008 at 10:25 AM, downs <default_357-line yahoo.de> wrote:

 Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 9:19 AM, Gregor Richards <Richards codu.org> wrote:

 Jarrett Billingsley wrote:

 On Fri, Sep 19, 2008 at 8:43 AM, bearophile <bearophileHUGS lycos.com>
 wrote:

 As you may know recently the Computer Shootout site has dropped many
 languages, among them there's D, Psyco (Python), etc. People that like Ruby
 language (and Jruby, etc) have created a benchmark suite, useful to tune
 implementations, compare their performance, spot some bugs, spot some
 performance bugs, etc:

 http://groups.google.com/group/ruby-benchmark-suite

 A similar performance suite may be created for D too, but to not put
 blinders on the eyes it's essential to add to such suite some benchmarks in
 other languages too, like Java (for the GC), C or C++ with GCC (for general
 optimization), and maybe more.

 As starting point the programs of the Shoout are fine, but I think few
 other things can be added (to benchmark exceptions, compiling time, and some
 other interesting things that the Shootout doesn't benchmark now).

 Bye,
 bearophile


 I'd imagine the Computer Shootout has only dropped many languages on
 the newer platforms since there aren't implementations for them.
 Getting GDC to work on x86-64 is a daunting task, or so I've heard.
 D's still in all the older benchmarks.


 Yeah, getting GDC to work on x86_64 is crazy. First you have to build it,
 then you have to install it! And at that point, you have to run it for it to
 do anything, but how am I supposed to know how you run binaries?! Egad, I
 can't be expected to figure this completely insane system out!

  - Gregor Richards


 Snyde demeanor aside, I'm talking more complex problems, like weird
 linking issues and binaries that crash on running for no apparent
 reason.


 Works here .. are you perhaps using a certain distribution's binary versions?
Those 4.2 files that are known to crash on trivial testcases?

 (If yes: don't do that then)


I've only tried building it once and ended up with a compiler that
built crashy EXEs, but I don't run x86-64, I was only trying to get it
working on a friend's machine.

I've just heard of a lot of people having problems with it, is all,
and am suggesting that that's a possible reason why Alioth hasn't get
added D to the new archs.
Sep 19 2008
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jarrett Billingsley:

 and am suggesting that that's a possible reason why Alioth hasn't get
 added D to the new archs.


No, as I have said in another answer that site author mostly just wants to
reduce his workload.

Bye,
bearophile
Sep 19 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Sep 19, 2008 at 11:09 AM, bearophile <bearophileHUGS lycos.com> wrote:

 Jarrett Billingsley:

 and am suggesting that that's a possible reason why Alioth hasn't get
 added D to the new archs.


 No, as I have said in another answer that site author mostly just wants to
reduce his workload.


I suppose there's no reason to say one of us or the other is wrong
without hearing from the author himself, is there?
Sep 19 2008
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jarrett Billingsley:

 I suppose there's no reason to say one of us or the other is wrong
 without hearing from the author himself, is there?


I think I am right:
https://alioth.debian.org/forum/forum.php?thread_id=14497&forum_id=999

bearophile
Sep 19 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Fri, Sep 19, 2008 at 5:05 PM, bearophile <bearophileHUGS lycos.com> wrote:

 Jarrett Billingsley:

 I suppose there's no reason to say one of us or the other is wrong
 without hearing from the author himself, is there?


 I think I am right:
 https://alioth.debian.org/forum/forum.php?thread_id=14497&forum_id=999

 bearophile


Then I am sad too.
Sep 19 2008
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jarrett Billingsley:

 Then I am sad too.


But look, they have added ATS instead!

http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all

It can't appeal people that like C/Algol syntax only, but it comes out at the
top as performance...

Bye,
bearophile
Sep 20 2008
parent reply The Anh Tran <trtheanh gmail.com> writes:
bearophile wrote:

 Jarrett Billingsley:

 Then I am sad too.


 
 But look, they have added ATS instead!
 
 http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all
 
 It can't appeal people that like C/Algol syntax only, but it comes out at the
top as performance...
 
 Bye,
 bearophile


It firstly compiles to C, then invokes gcc to do the rest. With the same 
algo, same idea, same trick, and same background compiler, you should 
have similar speed.

The one who contributes ats entries is also ATS author. He knows every 
odd & end of his language.
Sep 20 2008
parent bearophile <bearophileHUGS lycos.com> writes:
The Anh Tran:

 It firstly compiles to C, then invokes gcc to do the rest. With the same 
 algo, same idea, same trick, and same background compiler, you should 
 have similar speed.


Think about replacing C with asm in your period:


It firstly compiles to asm, then invokes the assembler to do the rest. With the
same algo, same idea, same trick, and same background assembler, you should
have similar speed.<


And ATS has a refined (compared to C) type system, that can make practical
difference in the way you write ATS code, even if at the end it's all converted
to C.

Bye,
bearophile
Sep 20 2008