• Robert burner Schadek (10/10) Sep 02 2015 Everybody is talking about benchmarks and making code faster, yet
• Edwin van Leeuwen (7/15) Sep 08 2015 This does sound like a really good idea. Is the plan to turn
• Robert burner Schadek (5/10) Sep 08 2015 you would add specific benchmark blocks, because the benchmarks
• Edwin van Leeuwen (8/15) Sep 08 2015 Just to be clear. The PR is to add the needed ground work for
• Robert burner Schadek (7/14) Sep 08 2015 The PR adds benchmarks for some functions of std.string.
• tchaloupka (7/12) Sep 08 2015 I sort of like what Rust is using:
• Robert burner Schadek (5/19) Sep 08 2015 Have you taken a look at the source of the PR. It does exactly

"Robert burner Schadek" <rburners gmail.com> writes:

Everybody is talking about benchmarks and making code faster, yet 
phobos is still lacking any long term benchmark gathering and 
monitoring solution.

PR https://github.com/D-Programming-Language/phobos/pull/2995 
provides all that, and is done some time now. It will give us 
pretty pictures we can put on the webpage.
These will show how awesome phobos already is, and how much we 
improve performance wise with time progressing.

Please have a look, and let us make phobos the most benchmarked 
library out there.

"Edwin van Leeuwen" <edder tkwsping.nl> writes:

On Wednesday, 2 September 2015 at 08:30:40 UTC, Robert burner 
Schadek wrote:
 Everybody is talking about benchmarks and making code faster, 
 yet phobos is still lacking any long term benchmark gathering 
 and monitoring solution.

 PR https://github.com/D-Programming-Language/phobos/pull/2995 
 provides all that, and is done some time now. It will give us 
 pretty pictures we can put on the webpage.
 These will show how awesome phobos already is, and how much we 
 improve performance wise with time progressing.

This does sound like a really good idea. Is the plan to turn 
every unittest block into a benchmark (automatically), or did you 
add specific benchmarks blocks?

Might be nice to have a mock up of the webpage with all the 
results/benchmarks.

"Robert burner Schadek" <rburners gmail.com> writes:

On Tuesday, 8 September 2015 at 08:42:59 UTC, Edwin van Leeuwen 
wrote:
 This does sound like a really good idea. Is the plan to turn 
 every unittest block into a benchmark (automatically), or did 
 you add specific benchmarks blocks?

you would add specific benchmark blocks, because the benchmarks 
might run to long for a rapid development cycle.

 Might be nice to have a mock up of the webpage with all the 
 results/benchmarks.

that is the idea

"Edwin van Leeuwen" <edder tkwsping.nl> writes:

On Tuesday, 8 September 2015 at 09:01:22 UTC, Robert burner 
Schadek wrote:
 On Tuesday, 8 September 2015 at 08:42:59 UTC, Edwin van Leeuwen 
 wrote:
 This does sound like a really good idea. Is the plan to turn 
 every unittest block into a benchmark (automatically), or did 
 you add specific benchmarks blocks?

 you would add specific benchmark blocks, because the benchmarks 
 might run to long for a rapid development cycle.

Just to be clear. The PR is to add the needed ground work for 
adding benchmark, or does it also add some benchmarks itself?

It does seem a bit of a shame not to use the unittests at all for 
baseline benchmarks. The unittests already exist, with reasonable 
good coverage. Getting good benchmark coverage by writing 
specific/new benchmarks would mean a huge amount of extra effort.

"Robert burner Schadek" <rburners gmail.com> writes:

On Tuesday, 8 September 2015 at 09:07:10 UTC, Edwin van Leeuwen 
wrote:
 Just to be clear. The PR is to add the needed ground work for 
 adding benchmark, or does it also add some benchmarks itself?

The PR adds benchmarks for some functions of std.string.

 It does seem a bit of a shame not to use the unittests at all 
 for baseline benchmarks. The unittests already exist, with 
 reasonable good coverage. Getting good benchmark coverage by 
 writing specific/new benchmarks would mean a huge amount of 
 extra effort.

I disagree, the unittests in phobos are often very short, only 
test corner cases, test regression fix, serve add doc unittests 
or do some basic sanity checking.

Benchmarks have other requirements.

"tchaloupka" <chalucha gmail.com> writes:

On Tuesday, 8 September 2015 at 09:20:08 UTC, Robert burner 
Schadek wrote:
 On Tuesday, 8 September 2015 at 09:07:10 UTC, Edwin van Leeuwen 
 wrote:
 Just to be clear. The PR is to add the needed ground work for 
 adding benchmark, or does it also add some benchmarks itself?

 The PR adds benchmarks for some functions of std.string.

I sort of like what Rust is using: 
https://github.com/rust-lang/rust/tree/master/src/test/bench

If we can have similar benchmarks suite which can be tested with 
every release, it can help to spot speed regressions or promote 
new optimisations.

"Robert burner Schadek" <rburners gmail.com> writes:

On Tuesday, 8 September 2015 at 12:24:19 UTC, tchaloupka wrote:
 On Tuesday, 8 September 2015 at 09:20:08 UTC, Robert burner 
 Schadek wrote:
 On Tuesday, 8 September 2015 at 09:07:10 UTC, Edwin van 
 Leeuwen wrote:
 Just to be clear. The PR is to add the needed ground work for 
 adding benchmark, or does it also add some benchmarks itself?

 The PR adds benchmarks for some functions of std.string.

 I sort of like what Rust is using: 
 https://github.com/rust-lang/rust/tree/master/src/test/bench

 If we can have similar benchmarks suite which can be tested 
 with every release, it can help to spot speed regressions or 
 promote new optimisations.

Have you taken a look at the source of the PR. It does exactly 
that.
Of course not ever function of phobos will get its benchmark in 
this PR, but this makes it easy to create benchmarks.