www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why is indexed foreach restricted to build in array ?

reply "matovitch" <camille.brugel laposte.net> writes:
Hello,

The question is in the title. It should be possible for a finite 
random access ranges to perform an indexed foreach no ? I mean 
like :

foreach(size_t i = 0, auto ref x; R)
{
     /*...*/
}

Why are other foreach statements overloadable but this one ?

Thanks in advance.
Apr 11 2015
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 11 April 2015 at 10:50:17 UTC, matovitch wrote:

 Hello,

 The question is in the title. It should be possible for a 
 finite random access ranges to perform an indexed foreach no ? 
 I mean like :

 foreach(size_t i = 0, auto ref x; R)
 {
     /*...*/
 }

 Why are other foreach statements overloadable but this one ?

 Thanks in advance.


As of 2.067, you can use std.range.enumerate[1]. See the PR that 
added it[2] and the enhancement request that proposed it[3] for 
more information about why it's a library function.

[1] http://dlang.org/phobos/std_range#enumerate
[2] https://github.com/D-Programming-Language/phobos/pull/1866
[3] https://issues.dlang.org/show_bug.cgi?id=5550
Apr 11 2015
parent reply "matovitch" <camille.brugel laposte.net> writes:
On Saturday, 11 April 2015 at 10:53:46 UTC, Jakob Ovrum wrote:

 On Saturday, 11 April 2015 at 10:50:17 UTC, matovitch wrote:

 Hello,

 The question is in the title. It should be possible for a 
 finite random access ranges to perform an indexed foreach no ? 
 I mean like :

 foreach(size_t i = 0, auto ref x; R)
 {
    /*...*/
 }

 Why are other foreach statements overloadable but this one ?

 Thanks in advance.


 As of 2.067, you can use std.range.enumerate[1]. See the PR 
 that added it[2] and the enhancement request that proposed 
 it[3] for more information about why it's a library function.

 [1] http://dlang.org/phobos/std_range#enumerate
 [2] https://github.com/D-Programming-Language/phobos/pull/1866
 [3] https://issues.dlang.org/show_bug.cgi?id=5550


Thanks for the tip...I just tried it on the generic kmeans 
algorithm I coded, there are huge performance issue with dmd 
2.0.67 :

//With foreach ennumerate
cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
Point(0.742677, 0.749284, 0.746855)
Point(0.246975, 0.247246, 0.251123)
Point(0.751372, 0.754126, 0.247526)
Point(0.250743, 0.754682, 0.250682)
Point(0.755332, 0.249898, 0.749533)
Point(0.254945, 0.25063, 0.750403)
Point(0.746505, 0.258751, 0.249303)
Point(0.244185, 0.748149, 0.750536)
4.72user 0.00system 0:04.73elapsed 99%CPU (0avgtext+0avgdata 
4796maxresident)k
0inputs+0outputs (0major+697minor)pagefaults 0swaps

//with a classic for loop
cbrugel eleanor ~/w/D/kmeans++> dmd kmeans_example.d kmeans.d
cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
Point(0.744609, 0.251452, 0.252298)
Point(0.750793, 0.754791, 0.248945)
Point(0.752109, 0.245865, 0.754593)
Point(0.752743, 0.746093, 0.748006)
Point(0.250339, 0.749277, 0.746064)
Point(0.249227, 0.24674, 0.751623)
Point(0.250478, 0.745153, 0.245349)
Point(0.243387, 0.249, 0.24996)
1.30user 0.00system 0:01.30elapsed 99%CPU (0avgtext+0avgdata 
6048maxresident)k
0inputs+0outputs (0major+560minor)pagefaults 0swaps

How does ennumerate work does it provide an other methods the 
other range don't and that is used by the indexed foreach ?
Apr 11 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Saturday, 11 April 2015 at 11:03:28 UTC, matovitch wrote:

 On Saturday, 11 April 2015 at 10:53:46 UTC, Jakob Ovrum wrote:

 On Saturday, 11 April 2015 at 10:50:17 UTC, matovitch wrote:

 Hello,

 The question is in the title. It should be possible for a 
 finite random access ranges to perform an indexed foreach no 
 ? I mean like :

 foreach(size_t i = 0, auto ref x; R)
 {
   /*...*/
 }

 Why are other foreach statements overloadable but this one ?

 Thanks in advance.


 As of 2.067, you can use std.range.enumerate[1]. See the PR 
 that added it[2] and the enhancement request that proposed 
 it[3] for more information about why it's a library function.

 [1] http://dlang.org/phobos/std_range#enumerate
 [2] https://github.com/D-Programming-Language/phobos/pull/1866
 [3] https://issues.dlang.org/show_bug.cgi?id=5550


 Thanks for the tip...I just tried it on the generic kmeans 
 algorithm I coded, there are huge performance issue with dmd 
 2.0.67 :

 //With foreach ennumerate
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.742677, 0.749284, 0.746855)
 Point(0.246975, 0.247246, 0.251123)
 Point(0.751372, 0.754126, 0.247526)
 Point(0.250743, 0.754682, 0.250682)
 Point(0.755332, 0.249898, 0.749533)
 Point(0.254945, 0.25063, 0.750403)
 Point(0.746505, 0.258751, 0.249303)
 Point(0.244185, 0.748149, 0.750536)
 4.72user 0.00system 0:04.73elapsed 99%CPU (0avgtext+0avgdata 
 4796maxresident)k
 0inputs+0outputs (0major+697minor)pagefaults 0swaps

 //with a classic for loop
 cbrugel eleanor ~/w/D/kmeans++> dmd kmeans_example.d kmeans.d
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.744609, 0.251452, 0.252298)
 Point(0.750793, 0.754791, 0.248945)
 Point(0.752109, 0.245865, 0.754593)
 Point(0.752743, 0.746093, 0.748006)
 Point(0.250339, 0.749277, 0.746064)
 Point(0.249227, 0.24674, 0.751623)
 Point(0.250478, 0.745153, 0.245349)
 Point(0.243387, 0.249, 0.24996)
 1.30user 0.00system 0:01.30elapsed 99%CPU (0avgtext+0avgdata 
 6048maxresident)k
 0inputs+0outputs (0major+560minor)pagefaults 0swaps


enumerate will kill unoptimised performance. Try with -O -release 
-inline and see what times you get.

Even better, get ldc or gdc and try them.
Apr 11 2015
parent reply "matovitch" <camille.brugel laposte.net> writes:
On Saturday, 11 April 2015 at 11:24:32 UTC, John Colvin wrote:

 On Saturday, 11 April 2015 at 11:03:28 UTC, matovitch wrote:

 On Saturday, 11 April 2015 at 10:53:46 UTC, Jakob Ovrum wrote:

 On Saturday, 11 April 2015 at 10:50:17 UTC, matovitch wrote:

 Hello,

 The question is in the title. It should be possible for a 
 finite random access ranges to perform an indexed foreach no 
 ? I mean like :

 foreach(size_t i = 0, auto ref x; R)
 {
  /*...*/
 }

 Why are other foreach statements overloadable but this one ?

 Thanks in advance.


 As of 2.067, you can use std.range.enumerate[1]. See the PR 
 that added it[2] and the enhancement request that proposed 
 it[3] for more information about why it's a library function.

 [1] http://dlang.org/phobos/std_range#enumerate
 [2] https://github.com/D-Programming-Language/phobos/pull/1866
 [3] https://issues.dlang.org/show_bug.cgi?id=5550


 Thanks for the tip...I just tried it on the generic kmeans 
 algorithm I coded, there are huge performance issue with dmd 
 2.0.67 :

 //With foreach ennumerate
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.742677, 0.749284, 0.746855)
 Point(0.246975, 0.247246, 0.251123)
 Point(0.751372, 0.754126, 0.247526)
 Point(0.250743, 0.754682, 0.250682)
 Point(0.755332, 0.249898, 0.749533)
 Point(0.254945, 0.25063, 0.750403)
 Point(0.746505, 0.258751, 0.249303)
 Point(0.244185, 0.748149, 0.750536)
 4.72user 0.00system 0:04.73elapsed 99%CPU (0avgtext+0avgdata 
 4796maxresident)k
 0inputs+0outputs (0major+697minor)pagefaults 0swaps

 //with a classic for loop
 cbrugel eleanor ~/w/D/kmeans++> dmd kmeans_example.d kmeans.d
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.744609, 0.251452, 0.252298)
 Point(0.750793, 0.754791, 0.248945)
 Point(0.752109, 0.245865, 0.754593)
 Point(0.752743, 0.746093, 0.748006)
 Point(0.250339, 0.749277, 0.746064)
 Point(0.249227, 0.24674, 0.751623)
 Point(0.250478, 0.745153, 0.245349)
 Point(0.243387, 0.249, 0.24996)
 1.30user 0.00system 0:01.30elapsed 99%CPU (0avgtext+0avgdata 
 6048maxresident)k
 0inputs+0outputs (0major+560minor)pagefaults 0swaps


 enumerate will kill unoptimised performance. Try with -O 
 -release -inline and see what times you get.

 Even better, get ldc or gdc and try them.


well ldc doesn't compile :

kmeans.d(40): Error: no property 'enumerate' for type 'Range'

With -O -release -inline  I get around 2s with foreach and 0.5s 
with a simple for.
Apr 11 2015
next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 11 April 2015 at 12:04:06 UTC, matovitch wrote:

 well ldc doesn't compile :

 kmeans.d(40): Error: no property 'enumerate' for type 'Range'

 With -O -release -inline  I get around 2s with foreach and 0.5s 
 with a simple for.


LDC does not yet support the 2.067 front-end version in which 
`enumerate` was made available. You could get the `enumerate` 
implementation from the DMD release and it should work with older 
FE versions (within reason - it may or may not depend on 
relatively new language features).

Performance with range-based code requires a sophisticated 
optimizer, the kind that is used to optimize idiomatic C++ code. 
In particular, inlining is important as there are a lot of tiny 
generic functions involved. Unfortunately, DMD's optimizer is not 
up to this task.
Apr 11 2015
prev sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Saturday, 11 April 2015 at 12:04:06 UTC, matovitch wrote:

 On Saturday, 11 April 2015 at 11:24:32 UTC, John Colvin wrote:

 On Saturday, 11 April 2015 at 11:03:28 UTC, matovitch wrote:

 On Saturday, 11 April 2015 at 10:53:46 UTC, Jakob Ovrum wrote:

 On Saturday, 11 April 2015 at 10:50:17 UTC, matovitch wrote:

 Hello,

 The question is in the title. It should be possible for a 
 finite random access ranges to perform an indexed foreach 
 no ? I mean like :

 foreach(size_t i = 0, auto ref x; R)
 {
 /*...*/
 }

 Why are other foreach statements overloadable but this one ?

 Thanks in advance.


 As of 2.067, you can use std.range.enumerate[1]. See the PR 
 that added it[2] and the enhancement request that proposed 
 it[3] for more information about why it's a library function.

 [1] http://dlang.org/phobos/std_range#enumerate
 [2] 
 https://github.com/D-Programming-Language/phobos/pull/1866
 [3] https://issues.dlang.org/show_bug.cgi?id=5550


 Thanks for the tip...I just tried it on the generic kmeans 
 algorithm I coded, there are huge performance issue with dmd 
 2.0.67 :

 //With foreach ennumerate
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.742677, 0.749284, 0.746855)
 Point(0.246975, 0.247246, 0.251123)
 Point(0.751372, 0.754126, 0.247526)
 Point(0.250743, 0.754682, 0.250682)
 Point(0.755332, 0.249898, 0.749533)
 Point(0.254945, 0.25063, 0.750403)
 Point(0.746505, 0.258751, 0.249303)
 Point(0.244185, 0.748149, 0.750536)
 4.72user 0.00system 0:04.73elapsed 99%CPU (0avgtext+0avgdata 
 4796maxresident)k
 0inputs+0outputs (0major+697minor)pagefaults 0swaps

 //with a classic for loop
 cbrugel eleanor ~/w/D/kmeans++> dmd kmeans_example.d kmeans.d
 cbrugel eleanor ~/w/D/kmeans++> time ./kmeans_example
 Point(0.744609, 0.251452, 0.252298)
 Point(0.750793, 0.754791, 0.248945)
 Point(0.752109, 0.245865, 0.754593)
 Point(0.752743, 0.746093, 0.748006)
 Point(0.250339, 0.749277, 0.746064)
 Point(0.249227, 0.24674, 0.751623)
 Point(0.250478, 0.745153, 0.245349)
 Point(0.243387, 0.249, 0.24996)
 1.30user 0.00system 0:01.30elapsed 99%CPU (0avgtext+0avgdata 
 6048maxresident)k
 0inputs+0outputs (0major+560minor)pagefaults 0swaps


 enumerate will kill unoptimised performance. Try with -O 
 -release -inline and see what times you get.

 Even better, get ldc or gdc and try them.


 well ldc doesn't compile :

 kmeans.d(40): Error: no property 'enumerate' for type 'Range'

 With -O -release -inline  I get around 2s with foreach and 0.5s 
 with a simple for.


This is roughly as expected. DMD is not good at optimising 
range-based code.

What OS are you on?
Apr 11 2015
parent "matovitch" <camille.brugel laposte.net> writes:
On Saturday, 11 April 2015 at 14:01:07 UTC, John Colvin wrote:

 What OS are you on?


Ubuntu 14.10.
Apr 11 2015