www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Value closures (no GC allocation)

reply Vittorio Romeo <vittorio.romeo outlook.com> writes:
Hello everyone, I recently started learning D (I come from a 
Modern C++ background) and I was curious about closures that 
require GC allocation. I wrote this simple example:

     auto bar(T)(T x)  nogc
     {
         return x(10);
     }

     auto foo(int x)  nogc
     {
         return bar((int y) => x + y + 10);
     }

     int main()  nogc
     {
         return foo(10);
     }



It doesn't compile with the following error:

     Error: function example.foo is  nogc yet allocates closures 
with the GC
            example.foo.__lambda2 closes over variable x at [...]



Live example on godbolt: https://godbolt.org/g/tECDh4



I was wondering whether or not D could provide some syntax that 
allowed the user to create a "value closure", similar to how C++ 
lambdas work. How would you feel about something like:


     auto bar(T)(T x)  nogc
     {
         return x(10);
     }

     auto foo(int x)  nogc
     {
         return bar([x](int y) => x + y + 10);
     }

     int main()  nogc
     {
         return foo(10);
     }



The syntax:

     [x](int y) => x + y + 10

would mean "create a 'value closure' that captures `x` by value 
inside it". It would be equivalent to the following program:

     struct AnonymousClosure
     {
         int captured_x;

         this(int x)  nogc
         {
             captured_x = x;
         }

         auto opCall(int y)  nogc
         {
             return captured_x + y + 10;
         }
     }

     auto foo(int x)  nogc
     {
         return bar(AnonymousClosure(x));
     }



Which is very similar to how C++ lambdas work. This would allow 
closures to be used in  nogc contexts with minimal syntactical 
overhead over classical closures.

Live example on godbolt: https://godbolt.org/g/ML2dlP

What are your thoughts? Has something similar been proposed 
before?
May 20 2017
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 21 May 2017 at 00:33:30 UTC, Vittorio Romeo wrote:

     auto bar(T)(T x)  nogc


Make that `scope T x` and it will compile, using stack 
allocation. Only worry is that you must not escape a reference to 
the delegate; you are only allowed to use it in that function's 
scope.


 What are your thoughts? Has something similar been proposed 
 before?


It has, and I actually don't hate it, but I also don't think it 
is necessary because of the `scope` storage class being one 
option and just manually writing out the struct functor being 
another viable one. The c++ syntax sugar is nice, but I can live 
without it.
May 20 2017
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Sunday, 21 May 2017 at 00:43:58 UTC, Adam D. Ruppe wrote:


 What are your thoughts? Has something similar been proposed 
 before?


 It has, and I actually don't hate it, but I also don't think it 
 is necessary because of the `scope` storage class being one 
 option and just manually writing out the struct functor being 
 another viable one. The c++ syntax sugar is nice, but I can 
 live without it.


It's not just syntax sugar. We don't always need by-reference 
captures. And it would be nice to be able to store the delegate 
longer than current scope...
May 20 2017
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 21 May 2017 at 00:49:23 UTC, Stanislav Blinov wrote:

 And it would be nice to be able to store the delegate longer 
 than current scope...


Important to note that this thing won't be a delegate, since 
delegates don't have room to store by-value captures. It can 
still be passed to isCallable templates and such, but it will be 
a struct of some sort, and you *can* write out the struct now, it 
just has more tedious syntax.

I don't object to the C++ syntax being added though, I just want 
to remind everyone that the result is possible with a struct.
May 20 2017
parent reply Vittorio Romeo <vittorio.romeo outlook.com> writes:
Thanks for the suggestions regarding the `scope` parameter 
attribute. I'll look into it!

On Sunday, 21 May 2017 at 01:30:51 UTC, Adam D. Ruppe wrote:

 On Sunday, 21 May 2017 at 00:49:23 UTC, Stanislav Blinov wrote:
 [...] you *can* write out the struct now, it just has more 
 tedious syntax.


This exact statement applied to C++ before C++11, but the 
introduction of lambda expression significantly changed the way 
people write and think about C++. Sometimes syntactic sugar can 
have huge impact on a language.

I think that creating anonymous structs on the spot (value type 
closures) is not a replacement for the current GC'd closures - it 
has a completely different meaning that can be exactly what you 
need in particular situations, not only for performance-related 
issues. As an example, consider this code:

     void delegate()[] arr;

     foreach(i; 0..5)
     {
         arr ~= () => writeln(i);
     }

     foreach(f; arr)
     {
         f();
     }

This is going to print "4 4 4 4", which might be the desired 
behavior. In other occasions you want to create a closure that 
captures the outer context by value. Here's some example 
pseudocode:

     void value_delegate()[] arr;

     foreach(i; 0..5)
     {
         arr ~= [i]() => writeln(i);
     }

     foreach(f; arr)
     {
         f();
     }

The pseudocode above would explicitly capture `i` by value and 
produce a `struct`-like closure. It would print "0 1 2 3".
May 20 2017
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Hi Vittorio, wonderful to see you here after C++Now! :)

On 05/20/2017 09:08 PM, Vittorio Romeo wrote:


consider this code:

     void delegate()[] arr;

     foreach(i; 0..5)
     {
         arr ~= () => writeln(i);
     }

     foreach(f; arr)
     {
         f();
     }

 This is going to print "4 4 4 4", which might be the desired behavior.


It's a bug. :/

   https://issues.dlang.org/show_bug.cgi?id=8621

Ali
May 21 2017
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 21 May 2017 at 04:08:04 UTC, Vittorio Romeo wrote:

 This exact statement applied to C++ before C++11, but the 
 introduction of lambda expression significantly changed the way 
 people write and think about C++. Sometimes syntactic sugar can 
 have huge impact on a language.


Oh absolutely, make no mistake, I would be FOR this addition. I 
like it and do think it would be worth it in a lot of places.

But, using the struct stuff, we can add some artificial sweetener 
now:

return bar(lambda!(x, q{ (int y) => x + y }));

You pass the captures first, then a q{} string literal of the 
lambda. Here's the implementation of that lambda template:



     template lambda(Args...) {
         import std.conv;
         import std.range;
         import std.string;
         string evil() {
                 // build the functor
                 import std.meta;
                 string code = "static struct anon {";
                 foreach(i; aliasSeqOf!(iota(0, Args.length-1)))
                         code ~= "typeof(Args[" ~ to!string(i) ~ 
"]) " ~ Args[i].stringof ~ ";";

                 string func = Args[$-1];
                 auto idx = func.indexOf("=>");
                 if(idx == -1)
                         throw new Exception("No => in lambda"); 
// or we could use one of the other styles

                 auto args = func[0 .. idx];
                 auto bod  = func[idx + 2 .. $];

                 code ~= "auto opCall(T...)" ~ args ~ "{ return " 
~ bod ~ "; }";

                 code ~= "this(T...)(T t) {
                         this.tupleof = t;
                 };";

                 code ~= "}";
                 return code;
         }
         mixin(evil());
         anon lambda() {
                 anon a;
                 // copy the values in
                 a.tupleof = Args[0 .. $-1];
                 return a;
         }
     }





Yes, the C++ syntax is still a bit better and can give MUCH nicer 
error messages, but for short things, this isn't bad.



     foreach(i; 0..5)
     {
         arr ~= () => writeln(i);
     }


so that's actually a long standing bug, but it hasn't been fixed 
for a long time....

But to work with that, you can do a capture with a wrapper 
function:

      arr ~= ((i) => delegate() { writeln(i); })(i);

So you define a new function that returns the delegate and pass 
the argument right there. This technique is common in Javascript.

Or, of course, using the artificial sweetener above, you can do:

arr ~= lambda!(i, q{ writeln(i); });

...assuming the imports are correct to call that library 
function... so i'll grant the artificial sweetener can leave a 
bitter aftertaste. (This is a good case to use in a feature 
request as to why the string mixin trick isn't actually a great 
replacement!)
May 21 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Sunday, 21 May 2017 at 18:17:57 UTC, Adam D. Ruppe wrote:


 But, using the struct stuff, we can add some artificial 
 sweetener now:

 return bar(lambda!(x, q{ (int y) => x + y }));

 You pass the captures first, then a q{} string literal of the 
 lambda. Here's the implementation of that lambda template:

     template lambda(Args...) {
// ...

         anon lambda() {
                 anon a;
                 // copy the values in
                 a.tupleof = Args[0 .. $-1];
                 return a;
         }
     }


Looks cool, but it'd still want a GC closure, won't it?
May 21 2017
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 21 May 2017 at 19:11:46 UTC, Stanislav Blinov wrote:

 Looks cool, but it'd still want a GC closure, won't it?


No, it just generates the same struct you'd write manually. That 
should work fine with  nogc now.
May 21 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Sunday, 21 May 2017 at 19:43:32 UTC, Adam D. Ruppe wrote:

 On Sunday, 21 May 2017 at 19:11:46 UTC, Stanislav Blinov wrote:

 Looks cool, but it'd still want a GC closure, won't it?


 No, it just generates the same struct you'd write manually. 
 That should work fine with  nogc now.


auto create()  nogc {
     auto y = 11;
     return lambda!(y, q{ (int x) => x+y } )();
}

Error: function create is  nogc yet it allocates closures with 
the GC


This:

     anon lambda() {
         anon a;
         // copy the values in
         a.tupleof = Args[0 .. $-1];
         return a;
     }

is a function that is generated by the template. It accesses the 
frame of create(). Am I missing something?

How recent of "now" do you mean? I've tested that with 
v2.075.0-master-5222639.
May 21 2017
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 21 May 2017 at 20:09:14 UTC, Stanislav Blinov wrote:

 is a function that is generated by the template. It accesses 
 the frame of create(). Am I missing something?


It does access the frame, but only long enough to copy the values 
into the struct.... there's no reason for that to allocate....

And, indeed it doesn't if you pass the object directly to another 
function; only when you return it does the compiler complain 
(which I didn't try before posting, just passing it down the 
function chain like in the OP). That's annoying. The mixin will 
have to be moved up a level.

Blah. Well, let's go ahead and formally propose the C++ syntax, 
our library solutions are all fat.
May 21 2017
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Sunday, 21 May 2017 at 20:25:14 UTC, Adam D. Ruppe wrote:


 Blah. Well, let's go ahead and formally propose the C++ syntax, 
 our library solutions are all fat.


:)
May 21 2017
prev sibling parent reply Vittorio Romeo <vittorio.romeo outlook.com> writes:
On Sunday, 21 May 2017 at 20:25:14 UTC, Adam D. Ruppe wrote:

 Blah. Well, let's go ahead and formally propose the C++ syntax, 
 our library solutions are all fat.


Are you going to create a DIP for this? I would be happy to 
contribute, but I don't feel confident enough to create a DIP on 
my own (I just started learning the language) :)
May 22 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Monday, 22 May 2017 at 14:06:54 UTC, Vittorio Romeo wrote:

 On Sunday, 21 May 2017 at 20:25:14 UTC, Adam D. Ruppe wrote:

 Blah. Well, let's go ahead and formally propose the C++ 
 syntax, our library solutions are all fat.


 Are you going to create a DIP for this? I would be happy to 
 contribute, but I don't feel confident enough to create a DIP 
 on my own (I just started learning the language) :)


The three of us could do it together through the magics of github.
May 22 2017
parent reply Vittorio Romeo <vittorio.romeo outlook.com> writes:
On Monday, 22 May 2017 at 15:17:24 UTC, Stanislav Blinov wrote:

 On Monday, 22 May 2017 at 14:06:54 UTC, Vittorio Romeo wrote:

 On Sunday, 21 May 2017 at 20:25:14 UTC, Adam D. Ruppe wrote:

 Blah. Well, let's go ahead and formally propose the C++ 
 syntax, our library solutions are all fat.


 Are you going to create a DIP for this? I would be happy to 
 contribute, but I don't feel confident enough to create a DIP 
 on my own (I just started learning the language) :)


 The three of us could do it together through the magics of 
 github.


I've created a very WIP draft here:
https://github.com/SuperV1234/DIPs/blob/master/DIPs/DIP1009.md

If you're interested in contributing, please let me know and I'll 
add you as a collaborator.
May 24 2017
next sibling parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Wednesday, 24 May 2017 at 20:15:37 UTC, Vittorio Romeo wrote:


 I've created a very WIP draft here:
 https://github.com/SuperV1234/DIPs/blob/master/DIPs/DIP1009.md

 If you're interested in contributing, please let me know and 
 I'll add you as a collaborator.


Yep, I've made a small PR to make myself visible :)
May 24 2017
prev sibling next sibling parent reply MysticZach <reachzach ggmail.com> writes:
On Wednesday, 24 May 2017 at 20:15:37 UTC, Vittorio Romeo wrote:

 On Monday, 22 May 2017 at 15:17:24 UTC, Stanislav Blinov wrote:

 On Monday, 22 May 2017 at 14:06:54 UTC, Vittorio Romeo wrote:

 On Sunday, 21 May 2017 at 20:25:14 UTC, Adam D. Ruppe wrote:

 Blah. Well, let's go ahead and formally propose the C++ 
 syntax, our library solutions are all fat.


 Are you going to create a DIP for this? I would be happy to 
 contribute, but I don't feel confident enough to create a DIP 
 on my own (I just started learning the language) :)


 The three of us could do it together through the magics of 
 github.


 I've created a very WIP draft here:
 https://github.com/SuperV1234/DIPs/blob/master/DIPs/DIP1009.md

 If you're interested in contributing, please let me know and 
 I'll add you as a collaborator.


As a matter of procedure, you don't want to assign a DIP number 
to it yourself. I'm pretty sure that is for the DIP manager.
May 24 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 May 2017 at 00:34:45 UTC, MysticZach wrote:


 I've created a very WIP draft here:
 https://github.com/SuperV1234/DIPs/blob/master/DIPs/DIP1009.md

 If you're interested in contributing, please let me know and 
 I'll add you as a collaborator.


 As a matter of procedure, you don't want to assign a DIP number 
 to it yourself. I'm pretty sure that is for the DIP manager.


We're well aware. The file name is not indicative of anything.
May 24 2017
parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 May 2017 at 01:13:08 UTC, Stanislav Blinov wrote:

 On Thursday, 25 May 2017 at 00:34:45 UTC, MysticZach wrote:



 As a matter of procedure, you don't want to assign a DIP 
 number to it yourself. I'm pretty sure that is for the DIP 
 manager.


 We're well aware. The file name is not indicative of anything.


The README outlines the procedure for DIP submission, including 
the format of the filename. The concern is that if you include 
number in the filename, it opens the door to people referring to 
it by that number. We want to avoid that.

[1] https://github.com/dlang/DIPs/blob/master/README.md
May 24 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 May 2017 at 01:18:22 UTC, Mike Parker wrote:

label:
if (self.bored) goto disclaimer;


 We're well aware. The file name is not indicative of anything.






 The README outlines the procedure for DIP submission, including 
 the format of the filename. The concern is that if you include 
 number in the filename, it opens the door to people referring 
 to it by that number. We want to avoid that.

 [1] https://github.com/dlang/DIPs/blob/master/README.md


// warning, loops until self.bored is true
goto label;

disclaimer:

Mike, I've read that README several times over. Yes, a mistake 
was made, a number self-assigned. Yes, that is not how it's done. 
Yes, we know. Yes. Thanks for the reminder. All is fixed.
May 24 2017
parent reply Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 May 2017 at 03:14:14 UTC, Stanislav Blinov wrote:


 Mike, I've read that README several times over. Yes, a mistake 
 was made, a number self-assigned. Yes, that is not how it's 
 done. Yes, we know. Yes. Thanks for the reminder. All is fixed.


Thanks. I wasn't criticizing. It wasn't fixed when I clicked the 
link, so I just wanted to make sure everyone's on the same page. 
I can't assume everyone has gone over the README.
May 24 2017
parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 May 2017 at 04:53:14 UTC, Mike Parker wrote:

 On Thursday, 25 May 2017 at 03:14:14 UTC, Stanislav Blinov 
 wrote:


 Mike, I've read that README several times over. Yes, a mistake 
 was made, a number self-assigned. Yes, that is not how it's 
 done. Yes, we know. Yes. Thanks for the reminder. All is fixed.


 Thanks. I wasn't criticizing. It wasn't fixed when I clicked 
 the link, so I just wanted to make sure everyone's on the same 
 page. I can't assume everyone has gone over the README.


Sigh... You're right. We should've renamed the file at the same 
time as removing the number from the contents, because 
"internets". And I shouldn't have snapped like that. I apologize.
May 24 2017
parent Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 May 2017 at 05:03:15 UTC, Stanislav Blinov wrote:


 Sigh... You're right. We should've renamed the file at the same 
 time as removing the number from the contents, because 
 "internets". And I shouldn't have snapped like that. I 
 apologize.


It's all good :-)
May 24 2017
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 24 May 2017 at 20:15:37 UTC, Vittorio Romeo wrote:

 If you're interested in contributing, please let me know and 
 I'll add you as a collaborator.


can i just edit it on the site?

but a few comments:

"Currently FunctionLiterals that capture their outer context 
(i.e. closures/delegates) require an allocation and the garbage 
collector. "


Not necessarily true, make sure you actually mention `scope 
delegate` and `alias` params that do not return it. Those capture 
but do not allocate.


"We will be proposing a syntax redundant w.r.t. current behavior 
(i.e. capture by reference), so maybe we should consider 
proposing deprecation?"

Don't do that.

"    // or simply, using existing syntax:
     //auto l = () => writeln("hello!");"

That existing one will actually make a function, not a delegate, 
since it doesn't access any locals.

"    struct anonymous_l"

That should be `static struct`.

"Capture by reference"

I'm against that, no need adding it and it complicates the whole 
thing. For example, "ref int _i;" as a struct member; there's no 
such thing in D. (the compiler could do it but still). And you'd 
have to explain the lifetime. Just no point doing this, the 
current behavior is completely fine for this.
May 24 2017
next sibling parent reply Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 May 2017 at 03:10:04 UTC, Adam D. Ruppe wrote:


 "Currently FunctionLiterals that capture their outer context 
 (i.e. closures/delegates) require an allocation and the garbage 
 collector. "


 Not necessarily true, make sure you actually mention `scope 
 delegate` and `alias` params that do not return it. Those 
 capture but do not allocate.


It's an infant document, wording is out there somewhere...


 "We will be proposing a syntax redundant w.r.t. current 
 behavior (i.e. capture by reference), so maybe we should 
 consider proposing deprecation?"

 Don't do that.


Why not? I, personally, have a simple, but solid, justification:

Captures, if any, should be explicit. That is all :)

That is solely my opinion, hence the cautious "maybe"...


 "Capture by reference"

 I'm against that, no need adding it and it complicates the 
 whole thing. For example, "ref int _i;" as a struct member; 
 there's no such thing in D. (the compiler could do it but 
 still). And you'd have to explain the lifetime. Just no point 
 doing this, the current behavior is completely fine for this.


By inference, adding capture syntax at all will also complicate 
the whole thing. IMHO, we either should have one, or the other, 
but not both. P.S. have you seen the TODO at the bottom of that 
section?
May 24 2017
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 25 May 2017 at 03:30:38 UTC, Stanislav Blinov wrote:

 Captures, if any, should be explicit. That is all :)


The current behavior is fine in most cases. The explicit by-value 
capture handles the remaining cases.


 By inference, adding capture syntax at all will also complicate 
 the whole thing.


Yeah, there's some complication in adding it but it is worth it 
because it gives something new without breaking anything old.


 P.S. have you seen the TODO at the bottom of that section?


Yeah, I'm EXTREMELY against removing the current behavior. I'd 
kill the whole thing just to avoid that.
May 24 2017
parent Stanislav Blinov <stanislav.blinov gmail.com> writes:
On Thursday, 25 May 2017 at 03:42:11 UTC, Adam D. Ruppe wrote:

 On Thursday, 25 May 2017 at 03:30:38 UTC, Stanislav Blinov 
 wrote:

 Captures, if any, should be explicit. That is all :)


 The current behavior is fine in most cases. The explicit 
 by-value capture handles the remaining cases.


In my opinion, it is not, because it's utterly silent unless you 
either stumble upon it via  nogc or a compiler switch.


 By inference, adding capture syntax at all will also 
 complicate the whole thing.


 Yeah, there's some complication in adding it but it is worth it 
 because it gives something new without breaking anything old.


 P.S. have you seen the TODO at the bottom of that section?


 Yeah, I'm EXTREMELY against removing the current behavior. I'd 
 kill the whole thing just to avoid that.


I understand perfectly, wasn't exactly easy contemplating that 
"maybe" either. In any case, we should not get ahead of 
ourselves. I'm not insisting on that particular change, I've 
brought it up for consideration, opened a discussion, and it 
looks like it worked :)
May 24 2017
prev sibling parent reply MakersF <makers.f.dev gmail.com> writes:
On Thursday, 25 May 2017 at 03:10:04 UTC, Adam D. Ruppe wrote:

 Snip


I think the discussion is going in two orthogonal directions, 
which are:

1) Capture by value. This is the ability to have a lambda which 
behaves as expected in the example Vittorio showed


   void delegate()[] arr;

    foreach(i; 0..5)
    {
        arr ~= () => writeln(i);
    }

    foreach(f; arr)
    {
        f();
    }

This is going to print "4 4 4 4"


2) Allocation of the lambda context. Whether this is performed by 
the GC, in the stack, or with another mechanism.

The reason these two points are orthogonal is that, as Adam 
showed, the ability to achieve one of the point is independent 
from the other (the capture by reference context allocation can 
be changed with the scope keyword, and from my understanding it's 
an implementation detail, it does not guarantee the use of the 
GC. And the value of capture by value could be allocated on the 
heap with the GC).


I see two weaknesses in this proposal:

1) The lambda is not compatible with a delegate anymore

2) There need to be a special notation for reference capture

Adam said

 I'm against that, no need adding it and it complicates the 
 whole thing. For example, "ref int _i;" as a struct member; 
 there's no such thing in D. (the compiler could do it but 
 still). And you'd have to explain the lifetime. Just no point 
 doing this, the current behavior is completely fine for this.




The other problem I see, which is easily noticeable when defining 
the new grammar, is that we are proposing a grammar which is a 
superset of the current one, but we are proposing it as an 
addition.


One of the assumption of D is that the GC is not a problem in the 
vast majority of the situations, and you want things to just work.
I think there is big value in the capture context not having to 
be explicit (even if usually I much prefer explicity over 
implicity).


Considered this, I'd propose an extension of the lambda syntax 
instead of an addition of a ValueFunctionLitteral.

Given the syntax

 [...] (...) ... {...}


which is composed of what I will call "capture", "parameters", 
"attributes", "body" (with capture and attributes optional)
the semantic would be:

- if no capture is provided the behavior is equivalent to today 
(automatic capturing by reference in a context allocated however 
the compiler prefers). This provides full backward compatibility.
Example
(int i) { writeln(i + j);}

j is captured by reference in a context allocated (possibly) on 
the GC. This is compatible with a delegate void(int).


- if capture is provided, then the lambda is rewritten as a 
struct, which the explicit variables in the capture list being 
captured by value, and the remaining being captured by reference 
(so there would still be implicit capture, which is considered 
convenient in many programming languages).
This is potentially unsafe since a reference could be copied when 
copying the struct, and the struct might outlive the referred 
object.
Example
[x] (int i) { writeln(i + j + x);}

x is captured by value in a local struct.
j is captured by reference in a local struct.
This is not a delegate anymore (different ABI).


- if "delegate" is used, then the capture context is allocated 
however the compile prefers

[x] delegate (int i) { writeln(i + j + x);}

x is captured by value in a (possibly) GC context.
j is captured by reference in a (possibly) GC context.
This is a delegate (compatible ABI).


This thus is an extension of the lambda syntax to allow capturing 
by value, but maintaining all the current properties and 
providing a way to remain compatible with the delegate functions.
You can also use the  nogc attribute to verify that the GC is not 
used (and in case also "delegate" is used "scope" will need to be 
used so that the compiler does not require the GC).

What are your opinions?
Please state if there are some misconceptions on the current 
situation.
May 29 2017
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 29 May 2017 at 21:26:11 UTC, MakersF wrote:

 1) Capture by value. This is the ability to have a lambda which 
 behaves as expected in the example Vittorio showed


That's a known bug in the implementation; it is supposed to work 
that way now.

(Though the bug has been open for so long I treat it as intended 
and tell people how to work with it, it isn't actually what the 
design says.)


 1) The lambda is not compatible with a delegate anymore

 2) There need to be a special notation for reference capture


That's true, but that's the whole point of the new thing. The 
auto-generated struct btw can be converted to a delegate with 
`&obj.opCall` or copied into a manually allocated buffer with 
`memcpy(buffer, &obj, obj.sizeof)`, just like the manually 
written struct to do the same. (and might have convenience 
methods, e.g. toDelegate and copyInto, to do these things too).



 - if capture is provided, then the lambda is rewritten as a 
 struct, which the explicit variables in the capture list being 
 captured by value, and the remaining being captured by 
 reference (so there would still be implicit capture, which is 
 considered convenient in many programming languages).


Eh, if you do any explicit, I think it should be ALL explicit. 
There's value in the simple proposals to syntax rewrite into a 
struct... but going beyond that I think is a negative and should 
not be in this proposal.
May 29 2017
prev sibling parent Francesco Zoffoli <lfc morz.com> writes:
On Sunday, 21 May 2017 at 00:43:58 UTC, Adam D. Ruppe wrote:

 On Sunday, 21 May 2017 at 00:33:30 UTC, Vittorio Romeo wrote:

 just manually writing out the struct functor being another 
 viable one. The c++ syntax sugar is nice, but I can live 
 without it.


Assembler is viable as well, but it's not nice to manually do 
what a computer can do faster and better, especially if it's an 
artificial limitation.

But I do agree that scope is a solution in this specific case.
There was also some discussion of a syntax to use allocators in 
closures (of course nothing even close to definitive) here
https://forum.dlang.org/thread/blvfxcbfzoyxowsfzlhn forum.dlang.org

Btw I'm happy to see you finally picked up D. You are going to 
enjoy it a lot!
May 20 2017
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 21 May 2017 at 00:33:30 UTC, Vittorio Romeo wrote:

 Hello everyone, I recently started learning D (I come from a 
 Modern C++ background) and I was curious about closures that 
 require GC allocation. I wrote this simple example:

     auto bar(T)(T x)  nogc
     {
         return x(10);
     }

     auto foo(int x)  nogc
     {
         return bar((int y) => x + y + 10);
     }

     int main()  nogc
     {
         return foo(10);
     }



 It doesn't compile with the following error:

     Error: function example.foo is  nogc yet allocates closures 
 with the GC
            example.foo.__lambda2 closes over variable x at [...]



 Live example on godbolt: https://godbolt.org/g/tECDh4



 I was wondering whether or not D could provide some syntax that 
 allowed the user to create a "value closure", similar to how 
 C++ lambdas work. How would you feel about something like:


     auto bar(T)(T x)  nogc
     {
         return x(10);
     }

     auto foo(int x)  nogc
     {
         return bar([x](int y) => x + y + 10);
     }

     int main()  nogc
     {
         return foo(10);
     }



 The syntax:

     [x](int y) => x + y + 10

 would mean "create a 'value closure' that captures `x` by value 
 inside it". It would be equivalent to the following program:

     struct AnonymousClosure
     {
         int captured_x;

         this(int x)  nogc
         {
             captured_x = x;
         }

         auto opCall(int y)  nogc
         {
             return captured_x + y + 10;
         }
     }

     auto foo(int x)  nogc
     {
         return bar(AnonymousClosure(x));
     }



 Which is very similar to how C++ lambdas work. This would allow 
 closures to be used in  nogc contexts with minimal syntactical 
 overhead over classical closures.

 Live example on godbolt: https://godbolt.org/g/ML2dlP

 What are your thoughts? Has something similar been proposed 
 before?


https://wiki.dlang.org/DIP30

Also, while no syntax is provided, this is how SDC works 
internally and this is how it can handle multiple context 
pointers.
May 30 2017
parent MysticZach <reachzach ggmail.com> writes:
On Tuesday, 30 May 2017 at 19:29:38 UTC, deadalnix wrote:

 What are your thoughts? Has something similar been proposed 
 before?


 https://wiki.dlang.org/DIP30

 Also, while no syntax is provided, this is how SDC works 
 internally and this is how it can handle multiple context 
 pointers.


FWIW, I think your three DIPs are elegant. Aside from questions 
of backward compatibility with current D, I like the way you 
designed functions and delegates there. Does this design 
integrate with the OP of this thread? It seems to. That's a 
question for the author of this thread too, if he reads this.
Jun 03 2017
prev sibling parent MakersF <makers.f.dev gmail.com> writes:
On Sunday, 21 May 2017 at 00:33:30 UTC, Vittorio Romeo wrote:

 Hello everyone, I recently started learning D (I come from a 
 Modern C++ background) and I was curious about closures that 
 require GC allocation. I wrote this simple example:

 [...]


I started to do some changes to the compiler.

https://github.com/MakersF/dmd/tree/valuelambda

It's my first time messing with DMD, it's mostly to familiarize 
myself with the compiler, so I don't expect to do everything the 
right way.
Let me know if you have inputs.
Jun 04 2017