digitalmars.D - Compile time performance for metaprogramming is somewhat inconsistent

maik klein (41/41) Mar 02 2016 Consider the following code

cym13 (2/16) Mar 02 2016 As often, compiler-version-flags please?

maik klein (2/20) Mar 02 2016 default dub.sdl and `dub build` with dmd v2.070

Steven Schveighoffer (3/24) Mar 03 2016 Try dub build -b release

John Colvin (5/22) Mar 03 2016 What happens if you add a few extra branches to

maik klein (14/38) Mar 03 2016 I have tried the same thing in general and stuff like this is

Stefan Koch (3/17) Mar 03 2016 Templates are by default more work then mixins.

maik klein <maikklein googlemail.com> writes:

Consider the following code

void main()
{
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
}

't' compiles on my machine in ~3.5 seconds while 't1' needs ~1 
minute to compile. It seems that mixins are just way more 
performant than template instantiations. Any ideas why? What 
causes the slowdown and what can I improve?

Also I compared some meta stuff in C++ and D.

For example filtering

enum isEven(alias i) = i % 2 is 0;
void main()
{
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: AliasSeq, Filter;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
to!string(i)).join(",") ~ ")";

     alias t = AliasSeq!(mixin(types));
     alias evenTypes = Filter!(isEven,t);
}

Someone was also so kind to create this in C++ though it looks a 
bit more crazy because he wanted to do roughly the same thing.

https://gist.github.com/ricejasonf/8c2b54c182e6038fd0ce

The D version compiles in ~14.5 seconds while the C++ version 
compiles in ~4.2 seconds. This was very surprising to me.

The more Hana like code is here 
https://github.com/boostorg/hana/blob/master/benchmark/filter/compile.hana.tuple.erb.cpp

*I wasn't yet able to run the Hana benchmarks yet because the 
build script doesn't detect my ruby executable.

Mar 02 2016

cym13 <cpicard openmailbox.org> writes:

On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 [...]

As often, compiler-version-flags please?

Mar 02 2016

maik klein <maikklein googlemail.com> writes:

On Thursday, 3 March 2016 at 02:26:09 UTC, cym13 wrote:
 On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 [...]

 As often, compiler-version-flags please?

default dub.sdl and `dub build` with dmd v2.070

Mar 02 2016

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 3/2/16 9:33 PM, maik klein wrote:
 On Thursday, 3 March 2016 at 02:26:09 UTC, cym13 wrote:
 On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i =>
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 [...]

 As often, compiler-version-flags please?

 default dub.sdl and `dub build` with dmd v2.070

Try dub build -b release

-Steve

Mar 03 2016

John Colvin <john.loughran.colvin gmail.com> writes:

On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 't' compiles on my machine in ~3.5 seconds while 't1' needs ~1 
 minute to compile. It seems that mixins are just way more 
 performant than template instantiations. Any ideas why? What 
 causes the slowdown and what can I improve?

What happens if you add a few extra branches to 
std.meta.aliasSeqOf, e.g. 
https://github.com/D-Programming-Language/phobos/commit/5d2cdf103bd697b8ff1a939c204dd2ed0eec0b59

Only a linear improvement but maybe worth a try?

Mar 03 2016

maik klein <maikklein googlemail.com> writes:

On Thursday, 3 March 2016 at 11:40:29 UTC, John Colvin wrote:
 On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 't' compiles on my machine in ~3.5 seconds while 't1' needs ~1 
 minute to compile. It seems that mixins are just way more 
 performant than template instantiations. Any ideas why? What 
 causes the slowdown and what can I improve?

 What happens if you add a few extra branches to 
 std.meta.aliasSeqOf, e.g. 
 https://github.com/D-Programming-Language/phobos/commit/5d2cdf103bd697b8ff1a939c204dd2ed0eec0b59

 Only a linear improvement but maybe worth a try?

I have tried the same thing in general and stuff like this is 
always a huge improvement.

In this case it goes down from ~60 seconds to ~3.8 seconds. I 
have done the same thing with my compile time map function, which 
gave me a drastic improvement.

I think recursion is just really bad in general for compile time 
stuff, for example your version

alias t1 = aliasSeqOf!(iota(0, 10000));

compiles in 3.8 seconds and uses roughly 600mb of ram while

alias t1 = aliasSeqOf!(iota(0, 20000));

compiles in 10.2 seconds and uses 1.9gb ram.

The mixin version is always the fastest but it also consumes way 
more memory and explodes before 20k elements

Mar 03 2016

Stefan Koch <uplink.coder googlemail.com> writes:

On Thursday, 3 March 2016 at 02:03:01 UTC, maik klein wrote:
 Consider the following code

 void main()
 {
     import std.stdio;
     import std.range: iota, join;
     import std.algorithm.iteration: map;
     import std.conv: to;
     import std.meta: aliasSeqOf, staticMap, AliasSeq;
     enum types = "AliasSeq!(" ~ iota(0,10000).map!(i => 
 to!string(i)).join(",") ~ ")";
     alias t = AliasSeq! (mixin(types));
     //alias t1 = aliasSeqOf!(iota(0, 10000));
 }

 [...]

Templates are by default more work then mixins.
If you want me to I can elaborate on that.

Mar 03 2016

D Programming

C/C++ Programming

Other

digitalmars.D - Compile time performance for metaprogramming is somewhat inconsistent