www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Can we fix this?

reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
https://issues.dlang.org/show_bug.cgi?id=2043

Impossible?
Sep 29 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 29 September 2021 at 10:44:53 UTC, Imperatorn wrote:

 https://issues.dlang.org/show_bug.cgi?id=2043

 Impossible?


... Documentation:

D has two kinds of blocks, the `{ ... }` kind and the `(){ ... 
}();` kind. Within the former, for efficiency, [something about 
this bug]. Within the latter, closure environments will be as 
expected from other languages at the cost of additional 
allocation. These "bubble blocks" are admittedly ugly but this is 
part of D's firm stand against the programming practices of the 
Scheme community.

```d
void delegate()[] dgList;

void main() {
     import std.stdio : writeln;

     foreach(int i; [1, 2, 3]) {
         auto b = i+2;
         dgList ~= { writeln(b); };
         writeln(&b); // b will be the *same* on each iteration
     }
     foreach (dg; dgList)
         dg(); // output: 5 5 5
}
```

vs.

```d
void delegate()[] dgList;

void main() {
     import std.stdio : writeln;

     foreach(int i; [1, 2, 3]) (){
         auto b = i+2;
         dgList ~= { writeln(b); };
         writeln(&b); // b will be *unique* on each iteration
     }();
     foreach (dg; dgList)
         dg(); // output: 3 4 5
}
```
Sep 29 2021
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 29 September 2021 at 14:16:50 UTC, jfondren wrote:

 On Wednesday, 29 September 2021 at 10:44:53 UTC, Imperatorn 
 wrote:

 https://issues.dlang.org/show_bug.cgi?id=2043

 Impossible?


 ... Documentation:

 D has two kinds of blocks, the `{ ... }` kind and the `(){ ... 
 }();` kind.


No, the later is just a lambda, with a perfectly normal scope in 
it.


 Within the former, for efficiency, [something about this bug].


Doing something fast and wrong is not exactly efficiency.
Sep 29 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Wednesday, 29 September 2021 at 14:34:01 UTC, deadalnix wrote:

 On Wednesday, 29 September 2021 at 14:16:50 UTC, jfondren wrote:

 On Wednesday, 29 September 2021 at 10:44:53 UTC, Imperatorn 
 wrote:

 https://issues.dlang.org/show_bug.cgi?id=2043

 Impossible?


 ... Documentation:

 D has two kinds of blocks, the `{ ... }` kind and the `(){ ... 
 }();` kind.


 No, the later is just a lambda, with a perfectly normal scope 
 in it.


 Within the former, for efficiency, [something about this bug].


 Doing something fast and wrong is not exactly efficiency.


I'm still a bit confused. What is the recommended approach here? 
Should we fix the language or have best practices/documentation 
on how to "work around" the problem.
Sep 29 2021
next sibling parent jfondren <julian.fondren gmail.com> writes:
On Wednesday, 29 September 2021 at 16:23:34 UTC, Imperatorn wrote:

 Should we


What 'we' have been doing for 13 years is to have this bugzilla 
link that documents the bug, and that it is a bug, with some 
commentary on it, including a workaround.

I'm not prepared to fix the bug, and per Lindy's Law the bug 
should get fixed in 2034, so *my* interest is

1. I'd like to not get surprised by this bug,

2. I'd like to have a reliable workaround for the bug.

And those would've been satisfied with documentation. Something 
that's been the case for 13 years should just be in the spec with 
its workaround. People read the spec and expect to know the 
language; they don't read all of bugzilla. It's not like it's 
hard to edit the spec when the bug is fixed.

Also, once you expect users to anticipate when they'll need the 
workaround, this objection becomes lighter: "People have made it 
clear they don't particularly like hidden allocations in 
innocuous looking code. Hence the genesis of the  nogc attribute. 
For this particular issue, it would be hard to look at a random 
loop and see if allocations are occurring - i.e. a nasty surprise 
if a small change suddenly made a big hit in performance. 
Profiling is not the answer, as very, very few people profile 
code."

As those random loops where allocations occur are precisely the 
loops that need the workaround.


 fix the language or have best practices/documentation on how to 
 "work around" the problem.
Sep 29 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 29 September 2021 at 16:23:34 UTC, Imperatorn wrote:

 I'm still a bit confused. What is the recommended approach 
 here? Should we fix the language or have best 
 practices/documentation on how to "work around" the problem.


I'm afraid this topic has come again and again, and Walter 
consistently says that it is not a bug, so we are stuck 
complaining. Maybe you can convince him.
Sep 29 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 30 September 2021 at 01:00:40 UTC, deadalnix wrote:

 On Wednesday, 29 September 2021 at 16:23:34 UTC, Imperatorn 
 wrote:

 I'm still a bit confused. What is the recommended approach 
 here? Should we fix the language or have best 
 practices/documentation on how to "work around" the problem.


 I'm afraid this topic has come again and again, and Walter 
 consistently says that it is not a bug, so we are stuck 
 complaining. Maybe you can convince him.


He doesn't exactly that it's not a bug, but that some design that 
leads to it is desirable, and that some solutions are 
undesirable. Specifically this part:


D closures are "by reference" rather than "by value". I make use 
of it being by reference all the time, as well as it being much 
more efficient. Changing it would likely break all sorts of code.


But the bugzilla issue isn't closed, he added the 'safe' keyword 
as it breaks immutability, and there's all kinds of language that 
accepts that this is a bug (edited to add emphasis):


Here's a better illustration of __the problem__: ...



The most practical __solution__ is ...


Very nice explanation with the while loops, btw.
Sep 29 2021
parent Basile B. <b2.temp gmx.com> writes:
On Thursday, 30 September 2021 at 01:19:54 UTC, jfondren wrote:

 He doesn't exactly that it's not a bug, but that some design 
 that leads to it is desirable, and that some solutions are 
 undesirable. Specifically this part:


D closures are "by reference" rather than "by value". I make 
use of it being by reference all the time, as well as it being 
much more efficient. Changing it would likely break all sorts 
of code.




I think that D could do the captures by ref, always:

If the lambda lives longer than the parent scope of one of its 
capture then the capture must be copied to a new'd value, (as 
updating the original makes no sense btw) and otherwise it can be 
a true reference to the original thing.
Sep 29 2021
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 30 September 2021 at 01:00:40 UTC, deadalnix wrote:

 On Wednesday, 29 September 2021 at 16:23:34 UTC, Imperatorn 
 wrote:

 I'm still a bit confused. What is the recommended approach 
 here? Should we fix the language or have best 
 practices/documentation on how to "work around" the problem.


 I'm afraid this topic has come again and again, and Walter 
 consistently says that it is not a bug, so we are stuck 
 complaining. Maybe you can convince him.


Maybe, idk.


does.

No var in loop = closure captures the variable
var in loop = closure captures the value of the variable

Sounds to me like a pretty "easy" thing to fix (if one could 
agree that such a solution would be acceptible)
Sep 30 2021
parent reply Tejas <notrealemail gmail.com> writes:
On Thursday, 30 September 2021 at 13:23:06 UTC, Imperatorn wrote:

 On Thursday, 30 September 2021 at 01:00:40 UTC, deadalnix wrote:

 On Wednesday, 29 September 2021 at 16:23:34 UTC, Imperatorn 
 wrote:

 I'm still a bit confused. What is the recommended approach 
 here? Should we fix the language or have best 
 practices/documentation on how to "work around" the problem.


 I'm afraid this topic has come again and again, and Walter 
 consistently says that it is not a bug, so we are stuck 
 complaining. Maybe you can convince him.


 Maybe, idk.

 But I don't understand why we can't just do what (for example) 


 No var in loop = closure captures the variable
 var in loop = closure captures the value of the variable

 Sounds to me like a pretty "easy" thing to fix (if one could 
 agree that such a solution would be acceptible)



other languages' will result in a huge performance hit, leading 
users to get very frustrated.

Here's the link : 
https://forum.dlang.org/post/s83nb0$12rh$1 digitalmars.com
Sep 30 2021
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 13:26:21 UTC, Tejas wrote:


 other languages' will result in a huge performance hit, leading 
 users to get very frustrated.

 Here's the link : 
 https://forum.dlang.org/post/s83nb0$12rh$1 digitalmars.com


It is generally not very difficult to do the wrong thing very 
fast.

In fact, let's just push this idea to the limit and not run any 
of the code at all. The performance win are humongous. Every 
single algorithm is now O(1) and terminates in milliseconds.
Sep 30 2021
prev sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 30 September 2021 at 13:26:21 UTC, Tejas wrote:

 On Thursday, 30 September 2021 at 13:23:06 UTC, Imperatorn 
 wrote:

 On Thursday, 30 September 2021 at 01:00:40 UTC, deadalnix 
 wrote:

 [...]


 Maybe, idk.

 But I don't understand why we can't just do what (for example) 


 No var in loop = closure captures the variable
 var in loop = closure captures the value of the variable

 Sounds to me like a pretty "easy" thing to fix (if one could 
 agree that such a solution would be acceptible)



 other languages' will result in a huge performance hit, leading 
 users to get very frustrated.

 Here's the link : 
 https://forum.dlang.org/post/s83nb0$12rh$1 digitalmars.com


I understand the reasoning, but imo it's worth it to be able to 
write simpler code.

But on the other hand performance is ofc important. But on the 
third hand, if something is a hurdle for many to write correct 
code, it should be looked at again and re-evaluated.

Just my opinion
Sep 30 2021
parent Max Samukha <maxsamukha gmail.com> writes:
On Thursday, 30 September 2021 at 17:29:09 UTC, Imperatorn wrote:

 I understand the reasoning, but imo it's worth it to be able to 
 write simpler code.


The reason you would want to use closures inside loops is to 
capture the loop variants, and therefore you agree to accept the 
performance hit. The current situation is that people whom Walter 
wanted to save from frustration don't use closures at all, and 
those who have a use for them are frustrated.


 But on the other hand performance is ofc important. But on the 
 third hand, if something is a hurdle for many to write correct 
 code, it should be looked at again and re-evaluated.

 Just my opinion
Sep 30 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/29/21 10:16 AM, jfondren wrote:

 On Wednesday, 29 September 2021 at 10:44:53 UTC, Imperatorn wrote:

 https://issues.dlang.org/show_bug.cgi?id=2043

 Impossible?


 
 ... Documentation:
 
 D has two kinds of blocks, the `{ ... }` kind and the `(){ ... }();` 
 kind. Within the former, for efficiency, [something about this bug]. 
 Within the latter, closure environments will be as expected from other 
 languages at the cost of additional allocation. These "bubble blocks" 
 are admittedly ugly but this is part of D's firm stand against the 
 programming practices of the Scheme community.
 
 ```d
 void delegate()[] dgList;
 
 void main() {
      import std.stdio : writeln;
 
      foreach(int i; [1, 2, 3]) {
          auto b = i+2;
          dgList ~= { writeln(b); };
          writeln(&b); // b will be the *same* on each iteration
      }
      foreach (dg; dgList)
          dg(); // output: 5 5 5
 }
 ```
 
 vs.
 
 ```d
 void delegate()[] dgList;
 
 void main() {
      import std.stdio : writeln;
 
      foreach(int i; [1, 2, 3]) (){
          auto b = i+2;
          dgList ~= { writeln(b); };
          writeln(&b); // b will be *unique* on each iteration
      }();
      foreach (dg; dgList)
          dg(); // output: 3 4 5
 }
 ```


This is not what I would have ever thought of, and it's kind of prone to 
error, since `i` is still used from within the lambda. It would not be 
hard to mess it up:

```d
foreach(int i; [1, 2, 3]) (){
     dgList ~= { writeln(i + 2); }; // still outputs 5 5 5
}();
```

What we need is a syntax to specify which values are captured and which 
variables are referenced.

What I normally do if I need something like this is:

```d
foreach(int i; [1, 2, 3]) {
    dgList ~= (b) { return {writeln(b);};} (i + 2);
    // or less error prone:
    dgList ~= (i) { return {writeln(i + 2);};} (i);
}

```

Which specifies the captured variable. But it's a lot of punctuation.

Some imagined syntax?

```d
dgList ~= ( capture i) {writeln(i + 2);};
// or:
dgList ~= ( capture i) => writeln(i + 2);
```

-Steve
Sep 29 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
Schveighoffer wrote:

 ```d
 foreach(int i; [1, 2, 3]) {
    dgList ~= (b) { return {writeln(b);};} (i + 2);
    // or less error prone:
    dgList ~= (i) { return {writeln(i + 2);};} (i);
 }

 ```

 Which specifies the captured variable. But it's a lot of 
 punctuation.


```
dgList ~= (i => { writeln(i + 2); })(i);
```

is cleaner, but ISRT that this usage will be banned because 
people rarely intend what it actually means.

I thought I tried this exact workaround and it didn't work, so I 
guessed that the optimizer had deleted the extra closure. But 
that does work, there.


 Some imagined syntax?

 ```d
 dgList ~= ( capture i) {writeln(i + 2);};
 // or:
 dgList ~= ( capture i) => writeln(i + 2);
 ```

 -Steve


Or a separate param list, to be similar to template params? 
`(i)() { writeln(i + 2); };`

Explicit captures are at least easy to explain since other 
languages have already gone this route.
Sep 29 2021
parent Elronnd <elronnd elronnd.net> writes:
On Wednesday, 29 September 2021 at 17:00:41 UTC, jfondren wrote:

 ```
 dgList ~= (i => { writeln(i + 2); })(i);
 ```

 is cleaner, but ISRT that this usage will be banned because 
 people rarely intend what it actually means.


i => () => writeln(...)
Sep 29 2021
prev sibling next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
Schveighoffer wrote:

 On 9/29/21 10:16 AM, jfondren wrote:

 [...]


 This is not what I would have ever thought of, and it's kind of 
 prone to error, since `i` is still used from within the lambda. 
 It would not be hard to mess it up:

 ```d
 foreach(int i; [1, 2, 3]) (){
     dgList ~= { writeln(i + 2); }; // still outputs 5 5 5
 }();
 ```

 What we need is a syntax to specify which values are captured 
 and which variables are referenced.

 What I normally do if I need something like this is:

 ```d
 foreach(int i; [1, 2, 3]) {
    dgList ~= (b) { return {writeln(b);};} (i + 2);
    // or less error prone:
    dgList ~= (i) { return {writeln(i + 2);};} (i);
 }

 ```

 Which specifies the captured variable. But it's a lot of 
 punctuation.

 Some imagined syntax?

 ```d
 dgList ~= ( capture i) {writeln(i + 2);};
 // or:
 dgList ~= ( capture i) => writeln(i + 2);
 ```

 -Steve


Tbh I'd be for any change that could improve something.

But why can't we just do something minimal, like use what we have 
(commenting on imagined syntax)

(i) { /*use i here*/ } or
(i) => { /*use i here*/ }

Could we solve it without introducing attributes or reusing some?

Personally I'm used to the lambda expression so I wouldn't mind 
using that. But can we just do something more sane than what we 
currently have?

If I ask kindly?
Sep 29 2021
prev sibling next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
Schveighoffer wrote:

 Some imagined syntax?

 ```d
 dgList ~= ( capture i) {writeln(i + 2);};
 // or:
 dgList ~= ( capture i) => writeln(i + 2);
 ```


or,

"Fix this bug or I'll write this code."

```d
string closure(string vars, string dg) {
     import std.string : split;

     string res = "{\n";
     foreach (var; vars.split(" ")) {
         res ~= "    auto " ~ var ~ " = " ~ var ~ ";\n";
     }
     res ~= "    return " ~ dg ~ ";\n}()";
     return res;
}

void main() {
     import std.stdio : writeln;

     void delegate()[] list;

     foreach (int i; [1, 2, 3]) {
         list ~= mixin(`i`.closure(`{ writeln(i); }`));
     }
     foreach (int i; [1, 2, 3]) {
         auto b = i + 2;
         list ~= mixin(/+ oops: +/`i`.closure(`{ writeln(b); }`));
     }

     foreach (dg; list)
         dg(); // output: 1 2 3 5 5 5
}
```

https://en.wikipedia.org/wiki/If_You_Don't_Buy_This_Book,_We'll_Kill_This_Dog!
Sep 29 2021
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Wednesday, 29 September 2021 at 22:04:33 UTC, jfondren wrote:

 On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
 Schveighoffer wrote:

 [...]


 or,

 [...]


Lol 👍
Sep 29 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
Schveighoffer wrote:

 What we need is a syntax to specify which values are captured 
 and which variables are referenced.

 What I normally do if I need something like this is:

 ```d
 foreach(int i; [1, 2, 3]) {
    dgList ~= (b) { return {writeln(b);};} (i + 2);
    // or less error prone:
    dgList ~= (i) { return {writeln(i + 2);};} (i);
 }


This is where things go off rails. We don't need any new syntax. 
We need to stop adding a new gizmo every time something is not 
doing the right thing. The ed result is that the original thing 
still don't do the right thing and the gizmo also doesn't do the 
right thing because it has been though to solve a specific edge 
case.

Now, foreach is a high level construct, and just get the same 
semantic as what it lowers into. Let's use while loops.

```d
int i = 1;
while (i < 4) {
    dgList ~= { writeln(i + 2); }; // still outputs 5 5 5
}
```

Now, this is expected isn't it? There is one and only one i 
variable. But let's change things a bit.

```d
int i = 1;
while (i < 4) {
    int n = i + 2;
    dgList ~= { writeln(n); }; // still outputs 5 5 5
}
```

Now this is wrong. A new n variable is created at each loop 
iteration, this isn't the same n. It's easy to convince oneself 
that this is the case: n can be made immutable and the code still 
compiles, which is evidence that the semantic is that each loop 
iteration has a new n variable.

So either we create a new closure for each loop iteration (if a 
variable locale to the loop is captured), or making n immutable 
must be rejected, because either n in one variable across all 
iterations, and it is mutated, or it is not, but it can't be both.

For completeness, it must be noted that the former tends to b
Sep 29 2021
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Thursday, 30 September 2021 at 00:59:26 UTC, deadalnix wrote:

 [...]
 Now, this is expected isn't it? There is one and only one i 
 variable. But let's change things a bit.

 ```d
 int i = 1;
 while (i < 4) {
    int n = i + 2;
    dgList ~= { writeln(n); }; // still outputs 5 5 5
 }
 ```

 Now this is wrong.


Why is this wrong ? Do you expect one alloca per iteration for n ?
It's pretty obvious that variables declared in loops also use the 
same alloca, always. Otherwise what would be required is stack 
save and restore after each iteration.
Sep 29 2021
parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 30 September 2021 at 02:03:39 UTC, Basile B. wrote:

 On Thursday, 30 September 2021 at 00:59:26 UTC, deadalnix wrote:

 [...]
 Now, this is expected isn't it? There is one and only one i 
 variable. But let's change things a bit.

 ```d
 int i = 1;
 while (i < 4) {
    int n = i + 2;
    dgList ~= { writeln(n); }; // still outputs 5 5 5
 }
 ```

 Now this is wrong.


 Why is this wrong ? Do you expect one alloca per iteration for 
 n ?


No, what I expect is for the closure to close over the n, and to 
do whatever it needs to do for that to work. In basically every 
language that exists the expected behavior from this code is 3 4 
5. Doing it otherwise is like PHP getting the ternary operator 
backwards -- it's not different because it's innovating, it's 
different because it's wrong.
Sep 29 2021
next sibling parent reply jfondren <julian.fondren gmail.com> writes:
On Thursday, 30 September 2021 at 02:25:53 UTC, jfondren wrote:

 In basically every language that exists the expected behavior 
 from this code is 3 4 5.


Exceptions: Python, Nim, Rust, probably C++, probably Zig.

In order:

```python
funcs = []

for i in range(0, 3):
     n = i + 2
     funcs.append(lambda: print(n))

for f in funcs:

```

```nim
var funcs: seq[proc(): void]

for i in 0..2:
   let n = i + 2
   funcs.add (proc = echo n)

for f in funcs:

```

```rust
fn main() {
     let mut funcs = Vec::new();
     for i in 0 .. 3 {
         let n = i + 2;
         funcs.push(|| println!("{}", n));
//                 --                ^ borrowed value does not 
live long enough
//                 |
//                 value captured here
     }
//  - `n` dropped here while still borrowed
     for f in funcs {
         f();
     }
}
```

So, more like, "basically every GC language that exists", except 
for Python which has famously garbage lambdas. At the least, you 
can't say that this behavior is a barrier to popularity with 
Python getting lambdas so wrong.

The workaround in Rust's case is to add a `move` before the `||` 
there.

Nim documents this exact complaint in 
https://nim-lang.org/docs/manual.html#closures-creating-closures-in-loops ,
which points to stdlib workarounds.
Sep 29 2021
parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 02:50:13 UTC, jfondren wrote:

 On Thursday, 30 September 2021 at 02:25:53 UTC, jfondren wrote:

 In basically every language that exists the expected behavior 
 from this code is 3 4 5.


 Exceptions: Python, Nim, Rust, probably C++, probably Zig.

 In order:

 ```python
 funcs = []

 for i in range(0, 3):
     n = i + 2
     funcs.append(lambda: print(n))

 for f in funcs:

 ```


No, in python, variable are not scoped. The semantic is 
consistent.

You'll note that it generally doesn't matter in python, because 
there is no notion of immutability or construction/destruction, 
so both behaviors are acceptable for python.

D either needs to ditch constructor, destruction, immutable, and 
anything that has to do with lifetime.
Sep 30 2021
parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 30 September 2021 at 11:00:15 UTC, deadalnix wrote:

 D either needs to ditch constructor, destruction, immutable, 
 and anything that has to do with lifetime.


I rather have closures that require you to state what you capture.
Sep 30 2021
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Thursday, 30 September 2021 at 11:32:28 UTC, Sebastiaan Koppe 
wrote:

 On Thursday, 30 September 2021 at 11:00:15 UTC, deadalnix wrote:

 D either needs to ditch constructor, destruction, immutable, 
 and anything that has to do with lifetime.


 I rather have closures that require you to state what you 
 capture.


I really like that in C++ because you don't clutter memory with 
unnecessary references. Your closure will only reference what you 
capture.
Sep 30 2021
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 30 September 2021 at 11:36:01 UTC, bauss wrote:

 I really like that in C++ because you don't clutter memory with 
 unnecessary references. Your closure will only reference what 
 you capture.


That's actually true in D as well. The compiler looks at the 
variables referenced in the closure automatically.
Sep 30 2021
parent bauss <jj_1337 live.dk> writes:
On Thursday, 30 September 2021 at 11:44:36 UTC, Adam D Ruppe 
wrote:

 On Thursday, 30 September 2021 at 11:36:01 UTC, bauss wrote:

 I really like that in C++ because you don't clutter memory 
 with unnecessary references. Your closure will only reference 
 what you capture.


 That's actually true in D as well. The compiler looks at the 
 variables referenced in the closure automatically.


The more you know
Sep 30 2021
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 11:36:01 UTC, bauss wrote:

 On Thursday, 30 September 2021 at 11:32:28 UTC, Sebastiaan 
 Koppe wrote:

 On Thursday, 30 September 2021 at 11:00:15 UTC, deadalnix 
 wrote:

 D either needs to ditch constructor, destruction, immutable, 
 and anything that has to do with lifetime.


 I rather have closures that require you to state what you 
 capture.


 I really like that in C++ because you don't clutter memory with 
 unnecessary references. Your closure will only reference what 
 you capture.


I have good news for you: the compiler knows what you capture and 
what you don't, so the only case where in which you'll have 
unnecessary reference, is if you capture explicitly and mess it 
up.
Sep 30 2021
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Thursday, 30 September 2021 at 13:55:53 UTC, deadalnix wrote:

 On Thursday, 30 September 2021 at 11:36:01 UTC, bauss wrote:

 On Thursday, 30 September 2021 at 11:32:28 UTC, Sebastiaan 
 Koppe wrote:

 I rather have closures that require you to state what you 
 capture.


 I really like that in C++ because you don't clutter memory 
 with unnecessary references. Your closure will only reference 
 what you capture.


 I have good news for you: the compiler knows what you capture 
 and what you don't, so the only case where in which you'll have 
 unnecessary reference, is if you capture explicitly and mess it 
 up.


The upside with explicit capture is that you can specify you want 
a move/copy instead of a reference, side-stepping the OT's decade 
long problem.
Sep 30 2021
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 14:07:18 UTC, Sebastiaan Koppe 
wrote:

 I have good news for you: the compiler knows what you capture 
 and what you don't, so the only case where in which you'll 
 have unnecessary reference, is if you capture explicitly and 
 mess it up.


 The upside with explicit capture is that you can specify you 
 want a move/copy instead of a reference, side-stepping the OT's 
 decade long problem.


You can create a copy in a local variable though, if you need a 
capture by value.

Now, one thing that we might want are delegates with a strongly 
typed closure, as in C++. There are obvious downsides. To begin 
with, each delegate is of a different type and ABI.

If that is ever added to D, it should probably need to be a new 
feature, different from current delegates.
Sep 30 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/30/21 9:55 AM, deadalnix wrote:

 On Thursday, 30 September 2021 at 11:36:01 UTC, bauss wrote:

 On Thursday, 30 September 2021 at 11:32:28 UTC, Sebastiaan Koppe wrote:

 On Thursday, 30 September 2021 at 11:00:15 UTC, deadalnix wrote:

 D either needs to ditch constructor, destruction, immutable, and 
 anything that has to do with lifetime.


 I rather have closures that require you to state what you capture.


 I really like that in C++ because you don't clutter memory with 
 unnecessary references. Your closure will only reference what you 
 capture.


 
 I have good news for you: the compiler knows what you capture and what 
 you don't, so the only case where in which you'll have unnecessary 
 reference, is if you capture explicitly and mess it up.


You may want to capture variables that would normally be referenced. The 
compiler can't possibly know what you want exactly.

Consider:

```d
void delegate() dg;
for(int i = 0; i < 100; ++i)
{
    dg = { writeln(i); };
}

dg();
```

What `i` is printed? Note that there is only one `i` variable over the 
entire loop.

-Steve
Sep 30 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/30/21 10:42 AM, Steven Schveighoffer wrote:


 ```d
 void delegate() dg;
 for(int i = 0; i < 100; ++i)
 {
     dg = { writeln(i); };
 }
 
 dg();
 ```


Haha of course, I shouldn't overwrite the same delegate.

It should have been an array of delegates, but the point still stands.

-Steve
Sep 30 2021
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 14:44:52 UTC, Steven 
Schveighoffer wrote:

 On 9/30/21 10:42 AM, Steven Schveighoffer wrote:


 ```d
 void delegate() dg;
 for(int i = 0; i < 100; ++i)
 {
     dg = { writeln(i); };
 }
 
 dg();
 ```


 Haha of course, I shouldn't overwrite the same delegate.

 It should have been an array of delegates, but the point still 
 stands.

 -Steve


In that case, you should expect all the delegates to print the 
same value. There is one i variable accross all loop iterations.
Sep 30 2021
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 14:42:34 UTC, Steven 
Schveighoffer wrote:

 Consider:

 ```d
 void delegate() dg;
 for(int i = 0; i < 100; ++i)
 {
    dg = { writeln(i); };
 }

 dg();
 ```

 What `i` is printed? Note that there is only one `i` variable 
 over the entire loop.

 -Steve


It's a good example of things that the optimizer shoudl handle. 
In that specific case, you shouldn't expect any memory allocation 
because the delegate never escape.
Sep 30 2021
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 11:32:28 UTC, Sebastiaan Koppe 
wrote:

 On Thursday, 30 September 2021 at 11:00:15 UTC, deadalnix wrote:

 D either needs to ditch constructor, destruction, immutable, 
 and anything that has to do with lifetime.


 I rather have closures that require you to state what you 
 capture.


I'm not sure what that buys. If you undercapture, you get an 
error, if you overcapture, you get suboptimal code.

On the other hand,t he compiler always exactly knows what you 
capture and what you do not capture, so it can make the optimal 
choice without you having to state anything.
Sep 30 2021
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 02:25:53 UTC, jfondren wrote:

 On Thursday, 30 September 2021 at 02:03:39 UTC, Basile B. wrote:

 On Thursday, 30 September 2021 at 00:59:26 UTC, deadalnix 
 wrote:

 [...]
 Now, this is expected isn't it? There is one and only one i 
 variable. But let's change things a bit.

 ```d
 int i = 1;
 while (i < 4) {
    int n = i + 2;
    dgList ~= { writeln(n); }; // still outputs 5 5 5
 }
 ```

 Now this is wrong.


 Why is this wrong ? Do you expect one alloca per iteration for 
 n ?


 No, what I expect is for the closure to close over the n, and 
 to do whatever it needs to do for that to work. In basically 
 every language that exists the expected behavior from this code 
 is 3 4 5. Doing it otherwise is like PHP getting the ternary 
 operator backwards -- it's not different because it's 
 innovating, it's different because it's wrong.


It is worse than this.

PHP getting the ternary operator backward is unfortunate, error 
prone and all, but all in all, it is consistent.

This is inconsistent, which is much more serious problem, as it 
means the language is unable to provide the invariants its 
constructs are supposed to provide.
Sep 30 2021
prev sibling next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 30 September 2021 at 00:59:26 UTC, deadalnix wrote:

 On Wednesday, 29 September 2021 at 16:47:23 UTC, Steven 
 Schveighoffer wrote:

[...]


 This is where things go off rails. We don't need any new 
 syntax. We need to stop adding a new gizmo every time something 
 is not doing the right thing. The ed result is that the 
 original thing still don't do the right thing and the gizmo 
 also doesn't do the right thing because it has been though to 
 solve a specific edge case.

 [...]


Yes. Declaring a new variable inside the loop should make the 
closure capture that unique variable.

If D does not do that, it's wrong.

Fixing that should not break code.
Sep 29 2021
prev sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Thursday, 30 September 2021 at 00:59:26 UTC, deadalnix wrote:

 Now this is wrong. A new n variable is created at each loop 
 iteration, this isn't the same n. It's easy to convince oneself 
 that this is the case: n can be made immutable and the code 
 still compiles, which is evidence that the semantic is that 
 each loop iteration has a new n variable.


Yes, that is the semantics I would expect. Note the complication 
when vars from multiple scopes are captured:

```d
int i = 1;
dgList ~= { writeln(i); };

while (i < 4) {
    immutable int n = i++;
    dgList ~= { writeln(i, " ", n); };
}
```
Expected:
4
4 1
4 2
4 3


That would require an implementation like this:
```d
static struct __C0
{
     int i;
     void call() { writeln(i); }
}

auto c0 = new __C0(1);
dgList ~= &c0.call;

while (c0.i < 4) {
    static struct __C1 {
       __C0* c0;
       immutable int n;
       void call() { writeln(c0.i, " ", n); }
    }

    dgList ~= &(new __C1(c0, c0.i++)).call;
}
```
Sep 30 2021
parent deadalnix <deadalnix gmail.com> writes:
On Thursday, 30 September 2021 at 12:55:48 UTC, Max Samukha wrote:

 Yes, that is the semantics I would expect. Note the 
 complication when vars from multiple scopes are captured:

 ```d
 int i = 1;
 dgList ~= { writeln(i); };

 while (i < 4) {
    immutable int n = i++;
    dgList ~= { writeln(i, " ", n); };
 }
 ```
 Expected:
 4
 4 1
 4 2
 4 3


 That would require an implementation like this:
 ```d
 static struct __C0
 {
     int i;
     void call() { writeln(i); }
 }

 auto c0 = new __C0(1);
 dgList ~= &c0.call;

 while (c0.i < 4) {
    static struct __C1 {
       __C0* c0;
       immutable int n;
       void call() { writeln(c0.i, " ", n); }
    }

    dgList ~= &(new __C1(c0, c0.i++)).call;
 }
 ```


That is indeed the most expected behavior that is consistent with 
D's semantic in other areas of the language - notably 
construction/destruction and immutability.
Sep 30 2021