www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.benchmark

reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
Although I should have known about it long ago, I have only just become
aware of Andrei's std.benchmark, which appears in his fork of Phobos,
but not in real Phobos. Is there an intention it should be in Phobos, or
is it intended to be a separate thing?

For me a benchmark framework should be outputting means, standard
deviations, and degrees of freedom of the timing of the function under
test. However the documentation comments of the std.benchmark module in
Andrei's repository only talk about a time and a relationship between a
group of related tests. So for me what is there is a great start but not
enough to be really useful.

Is anyone working on this? Is there an intention to make this part of
Phobos?  

Hopefully I am not blundering into a quagmire raising this.
-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
May 28 2014
next sibling parent reply "Joakim" <dlang joakim.airpost.net> writes:
On Wednesday, 28 May 2014 at 15:44:22 UTC, Russel Winder via 
Digitalmars-d wrote:
 Although I should have known about it long ago, I have only 
 just become
 aware of Andrei's std.benchmark, which appears in his fork of 
 Phobos,
 but not in real Phobos. Is there an intention it should be in 
 Phobos, or
 is it intended to be a separate thing?

 For me a benchmark framework should be outputting means, 
 standard
 deviations, and degrees of freedom of the timing of the 
 function under
 test. However the documentation comments of the std.benchmark 
 module in
 Andrei's repository only talk about a time and a relationship 
 between a
 group of related tests. So for me what is there is a great 
 start but not
 enough to be really useful.

 Is anyone working on this? Is there an intention to make this 
 part of
 Phobos?

 Hopefully I am not blundering into a quagmire raising this.

A google search turns up a long review thread a couple years ago: http://forum.dlang.org/thread/mailman.73.1347916419.5162.digitalmars-d puremagic.com Looks like it was remanded for further fixes, it shows as a "work in progress" in the review queue: http://wiki.dlang.org/Review_Queue
May 28 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/28/14, 9:33 AM, Russel Winder via Digitalmars-d wrote:
 On Wed, 2014-05-28 at 15:54 +0000, Joakim via Digitalmars-d wrote:
 […]
 A google search turns up a long review thread a couple years ago:

 http://forum.dlang.org/thread/mailman.73.1347916419.5162.digitalmars-d puremagic.com

A somewhat depressing thread.
 Looks like it was remanded for further fixes, it shows as a "work
 in progress" in the review queue:

 http://wiki.dlang.org/Review_Queue

Given there has been no activity in over 2 years, I guess it is deemed a dead duck in it's current form/place. Instead of wallowing in the backwaters of non-Phobosity, it should become a module accessible independently via a DVCS (presumably Git), and Dub? (though I will be using SCons) such that people can amend, extend and send in pull requests.

It was a solid review cycle that prompted good things to fix and improve. std.benchmark will be part of std iff we make it so. -- Andrei
May 29 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/28/14, 10:28 AM, Dicebot wrote:
 I'd love to see Andrei comment about it to either update its
 status or remove from review queue.

I will work on it but std.allocator takes precedence. If anyone is in a hurry, please take it over. I think std.benchmark can and will be awesome. -- Andrei
May 29 2014
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2014-05-28 at 15:54 +0000, Joakim via Digitalmars-d wrote:
[…]
 A google search turns up a long review thread a couple years ago:
 
 http://forum.dlang.org/thread/mailman.73.1347916419.5162.digitalmars-d puremagic.com

A somewhat depressing thread.
 Looks like it was remanded for further fixes, it shows as a "work 
 in progress" in the review queue:
 
 http://wiki.dlang.org/Review_Queue

Given there has been no activity in over 2 years, I guess it is deemed a dead duck in it's current form/place. Instead of wallowing in the backwaters of non-Phobosity, it should become a module accessible independently via a DVCS (presumably Git), and Dub? (though I will be using SCons) such that people can amend, extend and send in pull requests. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
May 28 2014
prev sibling next sibling parent "TheFlyingFiddle" <kurtyan student.chalmers.se> writes:
On Wednesday, 28 May 2014 at 16:33:13 UTC, Russel Winder via
Digitalmars-d wrote:
 On Wed, 2014-05-28 at 15:54 +0000, Joakim via Digitalmars-d 
 wrote:
 […]
 A google search turns up a long review thread a couple years 
 ago:
 
 http://forum.dlang.org/thread/mailman.73.1347916419.5162.digitalmars-d puremagic.com

A somewhat depressing thread.
 Looks like it was remanded for further fixes, it shows as a 
 "work in progress" in the review queue:
 
 http://wiki.dlang.org/Review_Queue

Given there has been no activity in over 2 years, I guess it is deemed a dead duck in it's current form/place. Instead of wallowing in the backwaters of non-Phobosity, it should become a module accessible independently via a DVCS (presumably Git), and Dub? (though I will be using SCons) such that people can amend, extend and send in pull requests.

If i am not mistaken, parts of std.benchmark moved into std.datetime. After this work on std.benchmark stopped.
May 28 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 28 May 2014 at 16:33:13 UTC, Russel Winder via
Digitalmars-d wrote:
 On Wed, 2014-05-28 at 15:54 +0000, Joakim via Digitalmars-d 
 wrote:
 […]
 A google search turns up a long review thread a couple years 
 ago:
 
 http://forum.dlang.org/thread/mailman.73.1347916419.5162.digitalmars-d puremagic.com

A somewhat depressing thread.
 Looks like it was remanded for further fixes, it shows as a 
 "work in progress" in the review queue:
 
 http://wiki.dlang.org/Review_Queue

Given there has been no activity in over 2 years, I guess it is deemed a dead duck in it's current form/place. Instead of wallowing in the backwaters of non-Phobosity, it should become a module accessible independently via a DVCS (presumably Git), and Dub? (though I will be using SCons) such that people can amend, extend and send in pull requests.

I'd love to see Andrei comment about it to either update its status or remove from review queue.
May 28 2014
prev sibling next sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2014-05-28 at 17:33 +0100, Russel Winder wrote:
[…]
 Instead of wallowing in the backwaters of non-Phobosity, it should
 become a module accessible independently via a DVCS (presumably Git),
 and Dub? (though I will be using SCons) such that people can amend,
 extend and send in pull requests.

On the grounds of "do, don't just talk", I have created https://github.com/russel/D_Benchmark initialized with a copy of https://github.com/andralex/phobos/blob/benchmark/std/benchmark.d. Anyone interested in evolving this, please feel free to clone and send in pull requests. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
May 28 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Friday, 30 May 2014 at 05:10:31 UTC, Andrei Alexandrescu wrote:
 On 5/28/14, 10:28 AM, Dicebot wrote:
 I'd love to see Andrei comment about it to either update its
 status or remove from review queue.

I will work on it but std.allocator takes precedence. If anyone is in a hurry, please take it over. I think std.benchmark can and will be awesome. -- Andrei

Seconded. std.allocator takes precedence, but a std.benchmark with proper statistics built-in will be awesome. David
May 30 2014
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 2014-05-29 at 22:07 -0700, Andrei Alexandrescu via Digitalmars-d
wrote:
[…]
 It was a solid review cycle that prompted good things to fix and 
 improve. std.benchmark will be part of std iff we make it so. -- Andrei

Clearly the data about the things is in the email thread, but might it be worth creating a summary document so there is a definitive "todo list"? There is then also the issue of: 1. Completing the todo list for a new review for entry into Phobos; versus 2. Extending the framework to handle multiple runs, and statistics on the dataset. Should these be seen as in competition (so that all work on 2 should happen in a separate branch to that on 1), or whether 2 should be seen as contributing to 1. No problems with either direction, but it is probably a good idea to take a view on this early. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
May 30 2014