www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Closure capture loop variables

reply "Freddy" <Hexagonalstar64 gmail.com> writes:
I understand that
----
import std.stdio;
void main(){
	int delegate() func;
	foreach(i;0..10){
		if(i==5){
			func= () => i;
		}
	}
	writeln(func());//9
}
----
captures the loop variable,but why does
----
import std.stdio;

void main(){
	int delegate() func;
	foreach(i;0..10){
		auto copy=i;
		if(i==5){
			func= () => copy;
		}
	}
	writeln(func());//should be 5
}

----
still print 9.
Apr 29 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 30 April 2015 at 01:16:20 UTC, Freddy wrote:
 I understand that
 ----
 import std.stdio;
 void main(){
 	int delegate() func;
 	foreach(i;0..10){
 		if(i==5){
 			func= () => i;
 		}
 	}
 	writeln(func());//9
 }
 ----
 captures the loop variable,but why does
 ----
 import std.stdio;

 void main(){
 	int delegate() func;
 	foreach(i;0..10){
 		auto copy=i;
 		if(i==5){
 			func= () => copy;
 		}
 	}
 	writeln(func());//should be 5
 }

 ----
 still print 9.
Because "copy" is still modified every time "i" is. You can either put "copy" inside the "if": ---- import std.stdio; void main(){ int delegate() func; foreach(i;0..10){ if(i==5){ auto copy=i; func= () => copy; } } writeln(func());//should be 5 } ---- Or you can create a closure for each iteration: ---- import std.stdio; void main(){ int delegate() func; foreach(i;0..10){ (){ auto copy=i; if(i==5){ func= () => copy; } }(); } writeln(func());//should be 5 } ----
Apr 29 2015
parent reply "Freddy" <Hexagonalstar64 gmail.com> writes:
On Thursday, 30 April 2015 at 01:19:45 UTC, Vladimir Panteleev 
wrote:
 Because "copy" is still modified every time "i" is.
But shouldn't copy be redeclared every loop iteration (or the compiler could pretend to redeclare it).
Apr 29 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 30 April 2015 at 03:58:44 UTC, Freddy wrote:
 On Thursday, 30 April 2015 at 01:19:45 UTC, Vladimir Panteleev 
 wrote:
 Because "copy" is still modified every time "i" is.
But shouldn't copy be redeclared every loop iteration (or the compiler could pretend to redeclare it).
No, it will have the same address every time.
Apr 29 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 30 April 2015 at 05:23:55 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 30 April 2015 at 03:58:44 UTC, Freddy wrote:
 On Thursday, 30 April 2015 at 01:19:45 UTC, Vladimir Panteleev 
 wrote:
 Because "copy" is still modified every time "i" is.
But shouldn't copy be redeclared every loop iteration (or the compiler could pretend to redeclare it).
No, it will have the same address every time.
The current behaviour is wrong: https://issues.dlang.org/show_bug.cgi?id=2043 https://issues.dlang.org/show_bug.cgi?id=8621
Apr 30 2015
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Thursday, 30 April 2015 at 12:01:32 UTC, Marc Schütz wrote:
 On Thursday, 30 April 2015 at 05:23:55 UTC, Vladimir Panteleev 
 wrote:
 On Thursday, 30 April 2015 at 03:58:44 UTC, Freddy wrote:
 On Thursday, 30 April 2015 at 01:19:45 UTC, Vladimir 
 Panteleev wrote:
 Because "copy" is still modified every time "i" is.
But shouldn't copy be redeclared every loop iteration (or the compiler could pretend to redeclare it).
No, it will have the same address every time.
The current behaviour is wrong: https://issues.dlang.org/show_bug.cgi?id=2043 https://issues.dlang.org/show_bug.cgi?id=8621
These are slightly different problems. I think Freddy's programs are working as designed. D closures should work in the same way as, e.g., JS closures. Try rewriting the program in JavaScript. If it behaves in the same way, it's not a D bug.
Apr 30 2015
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 04/30/2015 05:55 AM, Vladimir Panteleev wrote:

 D closures should work in the same way as, e.g., JS closures. Try
 rewriting the program in JavaScript. If it behaves in the same way, it's
 not a D bug.
Right. I remember Seth Ladd's Dart language presentation at the local ACCU in Silicon Valley. He was explaining how Dart was different from other languages in this respect. I think he had mentioned JavaScript with the behavior that he was not fond of, so he had designed the language the way OP wants. He explains it here: http://blog.sethladd.com/2012/01/for-loops-in-dart-or-fresh-bindings-for.html Ali
Apr 30 2015
prev sibling next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
On Thu, 30 Apr 2015 12:55:16 +0000, Vladimir Panteleev wrote:

 D closures should work in the same way as, e.g., JS closures.
js closures are fubared.=
Apr 30 2015
prev sibling next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 30 April 2015 at 12:55:18 UTC, Vladimir Panteleev 
wrote:
 On Thursday, 30 April 2015 at 12:01:32 UTC, Marc Schütz wrote:
 On Thursday, 30 April 2015 at 05:23:55 UTC, Vladimir Panteleev 
 wrote:
 On Thursday, 30 April 2015 at 03:58:44 UTC, Freddy wrote:
 On Thursday, 30 April 2015 at 01:19:45 UTC, Vladimir 
 Panteleev wrote:
 Because "copy" is still modified every time "i" is.
But shouldn't copy be redeclared every loop iteration (or the compiler could pretend to redeclare it).
No, it will have the same address every time.
The current behaviour is wrong: https://issues.dlang.org/show_bug.cgi?id=2043 https://issues.dlang.org/show_bug.cgi?id=8621
These are slightly different problems. I think Freddy's programs are working as designed. D closures should work in the same way as, e.g., JS closures. Try rewriting the program in JavaScript. If it behaves in the same way, it's not a D bug.
I don't think so. JS is not comparable, because it's scoping rules are different from D. Closures should capture the variable, not its memory location. And both `i` and `copy` are distinct variables in each loop iteration, they just happen to share the same location, because they have non-intersecting lifetimes.
May 01 2015
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/30/2015 5:55 AM, Vladimir Panteleev wrote:
 I think Freddy's programs are working as designed.
Yes, they are. D closures capture variables by reference. No, we're not changing that.
May 01 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 1 May 2015 at 17:51:05 UTC, Walter Bright wrote:
 Yes, they are.
I thought this until just a couple weeks ago when I was shown to be pretty conclusively wrong. See the discussion here: https://issues.dlang.org/show_bug.cgi?id=2043 When a new scope is introduced, a new variable is created. It might happen to share memory as an optimization in the implementation, but it is conceptually a whole new variable. foreach(i; 0..10) { int a; // new variable declared, it is set to 0 right now assert(a == 0); // always passes a = 5; // this isn't kept on the next iteration through } When you capture a variable from an inner scope, the optimization of sharing memory with the same variable on a previous iteration is no longer valid because the old variable now continues to exist. The correct behavior is analogous to: { auto a = new Object(); } { auto a = new Object(); } There, the GC might collect the first a and reuse the memory for the second a, but they are still different a's. When you do a closure, you're doing: Object capturedVariable, otherCapturedVariable; { auto a = new Object(); capturedVariable = a; } { auto a = new Object(); otherCapturedVariable = a; } Note that this is exactly what happens now if you call the function twice, but a scoped variable inside a loop is the same idea. If the GC collected the first a and reused its memory for the second a, that'd be a bug - there's another reference to it in capturedVariable, so the memory is not safe to reuse. Javascript does D's current behavior, so I thought it was correct too, but C# doesn't it that way. And thinking about it, Javascript doesn't really do it that way either because it's `var`s are hoisted up to function scope anyway - there's no such thing as a variable whose lifetime is only inside a loop there. (Note: the new `let` keyword in javascript is supposed to do scoping... but has the same closure behavior as `var` in firefox. However, looking at the docs, this seems to be a bug (perhaps in my test, or perhaps in my oldish version of firefox. Take a look: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Closures "Prior to the introduction of the let keyword in ECMAScript 6, a common problem with closures occurred when they were created inside a loop. " The let keyword, which adds lexical scoping rather than hoisted to function scoping, is said to change this situation. D's variables all work like `let` in JS. Therefore, we should do what it does too, which is what C# also does.)
 D closures capture variables by reference.
If this is the standard, D's implementation is still wrong. It isn't capturing the inner variable by reference, it is capturing the reused memory by reference. It is analogous to the GC collecting and reusing memory that is still referenced in an outer scope - a clear bug. The D standard says "The stack variables referenced by a nested function are still valid even after the function exits (this is different from D 1.0). ", so arguably you could say it is doing the right thing and capturing the stack, something I agreed with again until just ten days ago. See my change of mind here too in the edit: http://stackoverflow.com/questions/29759419/closures-in-loops-capturing-by-reference/29760081#29760081 There, I say it is expected because a longstanding bug is expected to work around.... but that doesn't make it *right*.
May 01 2015
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/1/2015 11:08 AM, Adam D. Ruppe wrote:
 There, I say it is expected because a longstanding bug is expected to work
 around.... but that doesn't make it *right*.
I did agree in the bug report on that that it was a bug.
May 01 2015
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Friday, 1 May 2015 at 18:08:09 UTC, Adam D. Ruppe wrote:
 Javascript does D's current behavior, so I thought it was 
 correct too, but C# doesn't it that way.
No it does not. In JS, var declare a variable at function level, so that is why you see the behavior you see. Since JS 1.7, you can declare scope level variable using let, and they have the behavior of scope level variable (ie like in C#). See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe ence/Statements/let for reference.
May 01 2015
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 1 May 2015 at 21:46:15 UTC, deadalnix wrote:
 No it does not. In JS, var declare a variable at function 
 level, so that is why you see the behavior you see.
Yes, I know, I said that a short while down in that post.
May 01 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 2 May 2015 at 00:55:19 UTC, Adam D. Ruppe wrote:
 On Friday, 1 May 2015 at 21:46:15 UTC, deadalnix wrote:
 No it does not. In JS, var declare a variable at function 
 level, so that is why you see the behavior you see.
Yes, I know, I said that a short while down in that post.
Saw that later :) The important point is that we are in violent agreement here.
May 01 2015
prev sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
On Fri, 01 May 2015 18:08:07 +0000, Adam D. Ruppe wrote:

 Javascript does D's current behavior, so I thought it was correct too,
if js doing something, big chances are that it's wrong. Brendan failed=20 his Scheme classes, especially those where he was taught about closures.=
May 01 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Saturday, 2 May 2015 at 03:35:17 UTC, ketmar wrote:
 if js doing something, big chances are that it's wrong. Brendan 
 failed
 his Scheme classes, especially those where he was taught about 
 closures.
Here's another fun thing about javascript: a = new Number(1); b = new Number(1); a<=b; // true a>=b; // true a+b == b+a; // true a==b; // false ! Good luck finding this bug in your codebase... Implicit type conversion, blargh.
May 02 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Friday, 1 May 2015 at 17:51:05 UTC, Walter Bright wrote:
 On 4/30/2015 5:55 AM, Vladimir Panteleev wrote:
 I think Freddy's programs are working as designed.
Yes, they are. D closures capture variables by reference. No, we're not changing that.
The variable is declared in the block, therefore it is a DIFFERENT variable at every iteration. This or delegate are unable to respect constness/immutability.
May 01 2015
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Friday, 1 May 2015 at 21:42:22 UTC, deadalnix wrote:
 On Friday, 1 May 2015 at 17:51:05 UTC, Walter Bright wrote:
 On 4/30/2015 5:55 AM, Vladimir Panteleev wrote:
 I think Freddy's programs are working as designed.
Yes, they are. D closures capture variables by reference. No, we're not changing that.
The variable is declared in the block, therefore it is a DIFFERENT variable at every iteration. This or delegate are unable to respect constness/immutability.
Conceptually - I agree. The variables declared in the block are different on each iteration, and I seriously doubt anyone escaping references to them(be it by closure or by direct pointer) means the same variable in all the iterations. Pragmatically - I'm not sure if this should be changed. This problem only appears when the closure escapes the scope - which from my experience seems to be the less common case. Unless the compiler can accurately tell when a closure escapes it's scope and when it doesn't, doing the correct thing will mean allocating call-stack frames on the heap *for every iteration* when a closure captures variable inside a loop. This can be a big hit on performance in the majority of the cases where it isn't really needed...
May 02 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Saturday, 2 May 2015 at 18:02:34 UTC, Idan Arye wrote:
 On Friday, 1 May 2015 at 21:42:22 UTC, deadalnix wrote:
 On Friday, 1 May 2015 at 17:51:05 UTC, Walter Bright wrote:
 On 4/30/2015 5:55 AM, Vladimir Panteleev wrote:
 I think Freddy's programs are working as designed.
Yes, they are. D closures capture variables by reference. No, we're not changing that.
The variable is declared in the block, therefore it is a DIFFERENT variable at every iteration. This or delegate are unable to respect constness/immutability.
Conceptually - I agree. The variables declared in the block are different on each iteration, and I seriously doubt anyone escaping references to them(be it by closure or by direct pointer) means the same variable in all the iterations. Pragmatically - I'm not sure if this should be changed. This problem only appears when the closure escapes the scope - which from my experience seems to be the less common case. Unless the compiler can accurately tell when a closure escapes it's scope and when it doesn't, doing the correct thing will mean allocating call-stack frames on the heap *for every iteration* when a closure captures variable inside a loop. This can be a big hit on performance in the majority of the cases where it isn't really needed...
It's trivial to check for escaping with a low false-positive rate. And even if not - correctness comes before performance.
May 03 2015