www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - foreach on interval index by ref increment

reply bearophile <bearophileHUGS lycos.com> writes:
In the last days Walter and other people are closing and fixing many bugs. But
there is one bug that Walter has closed that I am not so sure about:
http://d.puremagic.com/issues/show_bug.cgi?id=5306


Regarding this code:


import core.stdc.stdio;
void main() {
    foreach (ref i; 0 .. 10) {
        printf("i = %d\n", i);
        i++;
    }
}


He says it "works as expected":

i = 0
i = 2
i = 4
i = 6
i = 8


But I don't expect and don't like that output (the same probably happens to
Python programmers that see that code).

0..10 is a subset of the natural numbers, so it is an immutable entity.

And even if you see the items of a subset of the natural numbers as mutable
entities, "i++" means increasing by 1 each one of them, as you see for an array:


import std.stdio;
void main() {
    auto array = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
    foreach (ref x; array)
        x++;
    writeln(array);
}


That prints:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]


Or as you see with iota():

import std.stdio, std.range;
void main() {
    foreach (ref x; iota(0, 10)) {
        x++;
        write(x, " ");
    }
}


That prints:
1 2 3 4 5 6 7 8 9 10 


Skipping to the next number to me looks like "i++" is doing something more like
a pointer increment. It's ugly, and looks bug prone. foreach is not a light
syntax sugar over a for loop, it's a bit different. I have discussed a similar
topic some time ago.

I think some consistency with mathematics and iota() is good to have. Also keep
in mind that code like "foreach(i;N..M)" is used _everywhere_ in my programs,
so being a so common D idiom it's not a secondary issue.

I suggest to turn the "i" of a foreach(i;N..M) to an immutable. So if you use
"ref" it's a "immutable ref" still, so "i++" is a forbidden operation.
Programmers that want to increment the loop index inside the loop itself are
free to use a normal for loop, and keep the semantics of foreach loops more
tidy.

Bye,
bearophile
Jan 21 2012
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
I've had a use for it recently in some string processing, and I quite
liked it that all I had to do was make my index ref just to skip an
element. If I couldn't do that then I'd have to create a state
variable or use the ancient for loops.
Jan 21 2012
parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrej Mitrovic:


 I've had a use for it recently in some string processing, and I quite
 liked it that all I had to do was make my index ref just to skip an
 element. If I couldn't do that then I'd have to create a state
 variable or use the ancient for loops.


It's bad code, that relies on a not intuitive special case.
I suggest you to use a more clean and readable ancient for loop for that
purpose.
Relying on magical syntax is not a good idea for production code.

Bye,
bearophile
Jan 21 2012
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 1/22/12, bearophile <bearophileHUGS lycos.com> wrote:

 It's bad code, that relies on a not intuitive special case.
 I suggest you to use a more clean and readable ancient for loop for that
 purpose.
 Relying on magical syntax is not a good idea for production code.


Mr. one-letter-long-variable-names is telling me to write cleaner and
more readable code.

The ref keyword isn't magic, and immutability isn't the answer to everything.
Jan 21 2012
prev sibling next sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
It can be very useful to skip array elements.
---------
import std.stdio;

void main()
{
     auto ary = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
     foreach(ref i, e; ary)
         writeln(i++, "|", e);
}
--------
I made a pull request to distinguish between ref and non-ref index  
parameters.
http://d.puremagic.com/issues/show_bug.cgi?id=6652
But there is no deprecation path from the current behavior.
Jan 21 2012
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, January 21, 2012 22:38:48 bearophile wrote:

 In the last days Walter and other people are closing and fixing many bugs.
 But there is one bug that Walter has closed that I am not so sure about:
 http://d.puremagic.com/issues/show_bug.cgi?id=5306


[snip]

 I don't see any issue with the current behavior with regards to

foreach(i; 0 .. 10)

If anything, I'd argue that you shouldn't be allowed to use ref with iota, 
because instead of iterating over a fixed set of values, you're just taking the 
next value that a function gives you, in which case, ref makes no sense.

- Jonathan M Davis
Jan 21 2012
parent bearophile <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 If anything, I'd argue that you shouldn't be allowed to use ref with iota,
 because instead of iterating over a fixed set of values,


Isn't an interval of natural numbers something fixed?




 (and a language which really has no relation to D no less)<


I remember the word "Python" on the home page of the D language, but you are
right, Python is not much significant here, if only for being a language that
tries to be semantically clean. Turning foreach into a semantic hodge-podge is
not good.

--------------------

Martin Nowak:


 It can be very useful to skip array elements.


Perl too is very useful, but turning the foreach semantics into something so
dirty is not a good idea, given the presence of the normal for loops that allow
you do any thing.

--------------------

Andrej Mitrovic:


 The ref keyword isn't magic, and immutability isn't the answer to everything.


ref is not magic, but here it's used in a dirty way. And I think this will
cause troubles. We will see.

--------------------

Marco Leise:

Actually it is light syntax sugar over a for loop.

But it doesn't look like it. It looks like a higher level language construct.
So much that in years of usage of D foreach I have not thought that
incrementing the item of an interval, you skip to the successive one instead of
just incrementing it.

Allowing foreach to act just as light syntax sugar on a for means allowing it
to be a significant leaking abstraction
(http://en.wikipedia.org/wiki/Leaky_abstraction ) and this is not good for a
language, even a system language.

Bye,
bearophile
Jan 22 2012
prev sibling next sibling parent reply "F i L" <witte2008 gmail.com> writes:
The way it is now is makes sense to me. Behavior I would've 
expected, but then, I never truly wrote in Python beyond a few 
modifications to Blender interface scripts.
Jan 21 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, January 22, 2012 07:51:28 F i L wrote:

 The way it is now is makes sense to me. Behavior I would've
 expected, but then, I never truly wrote in Python beyond a few
 modifications to Blender interface scripts.


In general, the behavior of  the various constructs in the language should 
make sense (and I think that it does in this case), and if a lot of languages 
do something differently, that might be an argument for doing what they do, but 
just because one language (and a language which really has no relation to D no 
less) does differently isn't really an argument for D doing the same IMHO - not 
in and of itself anyway.

- Jonathan M Davis
Jan 21 2012
prev sibling next sibling parent reply "Marco Leise" <Marco.Leise gmx.de> writes:
Am 22.01.2012, 04:38 Uhr, schrieb bearophile <bearophileHUGS lycos.com>:


 Regarding this code:


 import core.stdc.stdio;
 void main() {
     foreach (ref i; 0 .. 10) {
         printf("i = %d\n", i);
         i++;
     }
 }


 He says it "works as expected":

 i = 0
 i = 2
 i = 4
 i = 6
 i = 8


[...]


 Skipping to the next number to me looks like "i++" is doing something  
 more like a pointer increment. It's ugly, and looks bug prone. foreach  
 is not a light syntax sugar over a for loop, it's a bit different. I  
 have discussed a similar topic some time ago.


Actually it is light syntax sugar over a for loop. The compiler sometimes  
prints out the syntax tree. But I have to agree that this use of ref  
doesn't look kosher. If I had my little way, I would adapt the ideas from  
VB, where you would write the above loop as "for i=0 to 9 step 2". So in D:

	foreach (i; 0 ... 9, +2)

also nice would be:

	foreach (i; 9 ... 0)

The alternative:

	foreach_reverse(i; 0 .. 10)

is really hard on the human brain :D
Jan 22 2012
parent "F i L" <witte2008 gmail.com> writes:
Marco Leise wrote:

 	foreach (i; 0 ... 9, +2)


This syntax looks nice. In fact it could replace 
"foreach_reverse" simply by specifying a negative step value: 
foreach (i; 9 .. 0, -1)



 also nice would be:

 	foreach (i; 9 ... 0)

 The alternative:

 	foreach_reverse(i; 0 .. 10)

 is really hard on the human brain :D


Wouldn't this require additional runtime checks? If "foreach (i; 
9 .. 0)" was sugar for "foreach (i; 9 .. 0, -1)" in cases where 
each range value could be determined at compile time that would 
make sense. Otherwise I'd rather have my brain do a bit more work 
in these cases over slowing down loops everywhere.
Jan 22 2012
prev sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 22 January 2012 at 03:38:48 UTC, bearophile wrote:

 In the last days Walter and other people are closing and fixing 
 many bugs. But there is one bug that Walter has closed that I 
 am not so sure about:
 http://d.puremagic.com/issues/show_bug.cgi?id=5306


I completely agree with your analysis.

foreach (i; 0..10) means to do something for every integer in the 
0..10 range. It does *not* mean "start an integer at 0 and 
repeatedly do something then increment it until it reaches 10". 
That's the implementation detail. Adding ref should not leak the 
implementation.

It doesn't for foreach (ref i; iota(0, 10))
It doesn't for foreach (ref i; /* an array of 0..10 */)

Why should foreach (ref i; 0..10) be a special case?

Arguing that it is sometimes convenient is not a strong argument. 
There are plenty of things that are sometimes convenient (e.g. 
implicit casting between any type), but are error-prone and 
disallowed for good reasons.

If you want control over the way the index variable increments 
then use a standard for-loop. That's what it's there for.
Jan 23 2012
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 23 Jan 2012 04:12:23 -0500, Peter Alexander  
<peter.alexander.au gmail.com> wrote:


 On Sunday, 22 January 2012 at 03:38:48 UTC, bearophile wrote:

 In the last days Walter and other people are closing and fixing many  
 bugs. But there is one bug that Walter has closed that I am not so sure  
 about:
 http://d.puremagic.com/issues/show_bug.cgi?id=5306


 I completely agree with your analysis.

 foreach (i; 0..10) means to do something for every integer in the 0..10  
 range. It does *not* mean "start an integer at 0 and repeatedly do  
 something then increment it until it reaches 10". That's the  
 implementation detail. Adding ref should not leak the implementation.

 It doesn't for foreach (ref i; iota(0, 10))
 It doesn't for foreach (ref i; /* an array of 0..10 */)

 Why should foreach (ref i; 0..10) be a special case?

 Arguing that it is sometimes convenient is not a strong argument. There  
 are plenty of things that are sometimes convenient (e.g. implicit  
 casting between any type), but are error-prone and disallowed for good  
 reasons.

 If you want control over the way the index variable increments then use  
 a standard for-loop. That's what it's there for.


I think the ref version is not an issue.  I personally think it should be  
invalid syntax, like this is invalid syntax:

foreach(ref i, x; [1,2,3,4,5])

But if it has to be valid, then the current behavior makes sense.

However, my biggest issue is with:

foreach(i; 1..10)
    ++i; // alters iteration.

IOW, see Martin's bug.  That is a real issue.

-Steve
Jan 24 2012