digitalmars.D - I implemented delegates in D
- maik klein (25/25) Jun 09 2016 I am currently writing a task system and I have this case where I
- Alex Parrill (19/20) Jun 09 2016 Well, yes, the entire point of delegates is to be able to capture
- maik klein (4/25) Jun 09 2016 I meant, "has this been implement as a library before". I am well
- Mathias Lang via Digitalmars-d (7/39) Jun 09 2016 To avoid the delegate being GC allocated, use `scope foo = (int i) { ......
- Steven Schveighoffer (4/6) Jun 09 2016 Is that true? At one point in D's past, this ONLY worked if you passed a...
- Mathias Lang via Digitalmars-d (11/18) Jun 09 2016 It's even part of DMD testsuite:
- Martin Nowak (2/10) Jun 11 2016 I fixed that more than 4 years ago, https://github.com/dlang/dmd/pull/62...
- maik klein (15/62) Jun 09 2016 But that means that the closure will be allocated on the stack
- Mathias Lang via Digitalmars-d (31/44) Jun 09 2016 Yes it will be stack allocated.
- maik klein (11/18) Jun 11 2016 That is what I am currently doing and it at least appears to be
- BBasile (48/75) Jun 09 2016 What you have here is more functor, to the extent that you can
- maik klein (36/126) Jun 11 2016 The problem is I currently do this
I am currently writing a task system and I have this case where I want to send a delegate to a different thread but this delegate also captures a variable. I use this to implement a "future". Now as far as I know this delegate will allocate GC memory and I just wanted to avoid that, just for fun. Here is the code https://dpaste.dzfl.pl/cd77fce99a5b I have only worked on it a couple of hours and I am sure there are many problems with it. Basically the idea is that you can use normal lambda syntax. If you want a function that returns and int and takes an int, you can write it like this: (int i) => i If you want a function that returns an int, takes an int, but also captures and int you would write it like this (int i, int captured) => i + captured But you also have to declare the base function type without the captured variables beforehand. Fn!(FnType!(int, int), (int i, int captured) => i + captured)(42); That is how I know what the captured variables are. The only part that is currently missing are polymorphic delegates. They are not too useful if I can't pack them into the same array. I guess I have to do this with classes/interfaces. Thoughts? Has this been done before?
Jun 09 2016
On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:Has this been done before?Well, yes, the entire point of delegates is to be able to capture variables (as opposed to function pointers, which cannot). auto createADelegate(int captured) { return (int a) => captured + a; } void main() { auto dg1 = createADelegate(5); auto dg2 = createADelegate(32); assert(dg1(5) == 10); assert(dg1(10) == 15); assert(dg2(8) == 40); assert(dg2(32) == 64); } https://dpaste.dzfl.pl/90ebc29651f6 (Unfortunately template delegates, like the ones used with map, don't keep their captured variables alive after the captured variables go out of scope, but it doesn't sound like you need those)
Jun 09 2016
On Thursday, 9 June 2016 at 21:32:33 UTC, Alex Parrill wrote:On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:I meant, "has this been implement as a library before". I am well aware that delegates exist in the language but as far as I know you can not do manual allocation with delegates (to avoid the GC).Has this been done before?Well, yes, the entire point of delegates is to be able to capture variables (as opposed to function pointers, which cannot). auto createADelegate(int captured) { return (int a) => captured + a; } void main() { auto dg1 = createADelegate(5); auto dg2 = createADelegate(32); assert(dg1(5) == 10); assert(dg1(10) == 15); assert(dg2(8) == 40); assert(dg2(32) == 64); } https://dpaste.dzfl.pl/90ebc29651f6 (Unfortunately template delegates, like the ones used with map, don't keep their captured variables alive after the captured variables go out of scope, but it doesn't sound like you need those)
Jun 09 2016
To avoid the delegate being GC allocated, use `scope foo = (int i) { ... }`
at call site.
You can also make your function signature as `void func(scope void
delegate() dg)` in which case it won't allocate if you pass a literal
directly.
2016-06-09 23:57 GMT+02:00 maik klein via Digitalmars-d <
digitalmars-d puremagic.com>:
On Thursday, 9 June 2016 at 21:32:33 UTC, Alex Parrill wrote:
On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:
Has this been done before?
Well, yes, the entire point of delegates is to be able to capture
variables (as opposed to function pointers, which cannot).
auto createADelegate(int captured) {
return (int a) => captured + a;
}
void main() {
auto dg1 = createADelegate(5);
auto dg2 = createADelegate(32);
assert(dg1(5) == 10);
assert(dg1(10) == 15);
assert(dg2(8) == 40);
assert(dg2(32) == 64);
}
https://dpaste.dzfl.pl/90ebc29651f6
(Unfortunately template delegates, like the ones used with map, don't
keep their captured variables alive after the captured variables go out of
scope, but it doesn't sound like you need those)
I meant, "has this been implement as a library before". I am well aware
that delegates exist in the language but as far as I know you can not do
manual allocation with delegates (to avoid the GC).
Jun 09 2016
On 6/9/16 6:06 PM, Mathias Lang via Digitalmars-d wrote:
To avoid the delegate being GC allocated, use `scope foo = (int i) { ...
}` at call site.
Is that true? At one point in D's past, this ONLY worked if you passed a
delegate to a function accepting a scope delegate. Maybe it's been fixed.
-Steve
Jun 09 2016
Mathias Lang via Digitalmars-d <digitalmars-d puremagic.com> It's even part of DMD testsuite: https://github.com/dlang/dmd/blob/38c1bad2f41ad8e6bc7c08deae26e30c9c7d7704/test/runnable/nogc.d#L20 At DConf, Dicebot mention that he had some problems in the past and it doesn't work in some places. The Weka people seemed to have similar experience. I did some experiment and so far the only related issue I've found is https://issues.dlang.org/show_bug.cgi?id=16037 . If you happen to come accross a case where it doesn't work, I would very like to be CC'ed on that issue :) 2016-06-10 0:17 GMT+02:00 Steven Schveighoffer via Digitalmars-d < digitalmars-d puremagic.com>:On 6/9/16 6:06 PM, Mathias Lang via Digitalmars-d wrote:To avoid the delegate being GC allocated, use `scope foo = (int i) { ... }` at call site.Is that true? At one point in D's past, this ONLY worked if you passed a delegate to a function accepting a scope delegate. Maybe it's been fixed. -Steve
Jun 09 2016
On 06/10/2016 12:17 AM, Steven Schveighoffer wrote:On 6/9/16 6:06 PM, Mathias Lang via Digitalmars-d wrote:I fixed that more than 4 years ago, https://github.com/dlang/dmd/pull/625.To avoid the delegate being GC allocated, use `scope foo = (int i) { ... }` at call site.Is that true? At one point in D's past, this ONLY worked if you passed a delegate to a function accepting a scope delegate. Maybe it's been fixed. -Steve
Jun 11 2016
On Thursday, 9 June 2016 at 22:06:24 UTC, Mathias Lang wrote:
To avoid the delegate being GC allocated, use `scope foo = (int
i) { ... }`
at call site.
You can also make your function signature as `void func(scope
void
delegate() dg)` in which case it won't allocate if you pass a
literal
directly.
2016-06-09 23:57 GMT+02:00 maik klein via Digitalmars-d <
digitalmars-d puremagic.com>:
On Thursday, 9 June 2016 at 21:32:33 UTC, Alex Parrill wrote:
On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:
Has this been done before?
Well, yes, the entire point of delegates is to be able to
capture variables (as opposed to function pointers, which
cannot).
auto createADelegate(int captured) {
return (int a) => captured + a;
}
void main() {
auto dg1 = createADelegate(5);
auto dg2 = createADelegate(32);
assert(dg1(5) == 10);
assert(dg1(10) == 15);
assert(dg2(8) == 40);
assert(dg2(32) == 64);
}
https://dpaste.dzfl.pl/90ebc29651f6
(Unfortunately template delegates, like the ones used with
map, don't keep their captured variables alive after the
captured variables go out of scope, but it doesn't sound like
you need those)
I meant, "has this been implement as a library before". I am
well aware that delegates exist in the language but as far as
I know you can not do manual allocation with delegates (to
avoid the GC).
But that means that the closure will be allocated on the stack
right? What happens when I send it with
http://dpldocs.info/experimental-docs/std.concurrency.send.html
Will it copy the function or will it only send the pointer?
Also scope on local vars is marked to be deprecated, see
http://stackoverflow.com/a/4713064/944430
I don't think that I can use delegates (without the gc), what I
basically do is send a delegate to a thread, create a fiber on
that thread and put it in a thread local array.
The delegate contains a "future" that I can access on a different
thread. I use it as mechanism to share results. (Its synchronized
with atomics)
I mean currently I just use gc delegates, but I am exploring some
alternatives.
Jun 09 2016
2016-06-10 1:20 GMT+02:00 maik klein via Digitalmars-d < digitalmars-d puremagic.com>:But that means that the closure will be allocated on the stack right? What happens when I send it with http://dpldocs.info/experimental-docs/std.concurrency.send.html Will it copy the function or will it only send the pointer?Yes it will be stack allocated. But if you wish to `send` something, I think you're better off using a function in an object (disclaimer: never tried). Functions are not copied, so I guess you are refering to the context pointer ?Also scope on local vars is marked to be deprecated, see http://stackoverflow.com/a/4713064/944430Yeah and they are not going away anytime soon. There's a reason why the compiler won't even warn about them. They are extremely useful and there is currently no replacement for them.I don't think that I can use delegates (without the gc), what I basically do is send a delegate to a thread, create a fiber on that thread and put it in a thread local array.Did you try to send a native delegate ? I would be very surprised if you were allowed to do so.The delegate contains a "future" that I can access on a different thread. I use it as mechanism to share results. (Its synchronized with atomics) I mean currently I just use gc delegates, but I am exploring some alternatives.Note that not all delegates allocate. Function local delegate refering to variable do, but the one refering to aggregate never do. E.g. the following compiles and run: ``` struct Foobar { string toString() nogc { return "Foobar"; } } void func (string delegate() ts) nogc {} void main () nogc { scope c = new Object; Foobar s; func(&c.toString); // ptr=c funcptr=toString func(&s.toString); //ptr =s funcptr=toString string delegate() nogc dg; dg.funcptr = (&s.toString).funcptr; // There might be a better way to do that dg.ptr = &s; assert("Foobar" == dg()); } ```
Jun 09 2016
On Friday, 10 June 2016 at 00:23:13 UTC, Mathias Lang wrote:2016-06-10 1:20 GMT+02:00 maik klein via Digitalmars-d < digitalmars-d puremagic.com>:Did you try to send a native delegate ? I would be very surprised if you were allowed to do so.That is what I am currently doing and it at least appears to be working as intended.Note that not all delegates allocate. Function local delegate refering to variable do, but the one refering to aggregate never do.All my closures/delegated need to capture a local variable. They capture a pointer to a heap allocated "Cell". I use the cell as a primitive to write my result into. It has an internal atomic counter, if the counter as decremented to 0, I know that I can safely read from it. I am not sure how I would implement it differently. So that means that all my delegates allocate, and they allocate with the GC.
Jun 11 2016
On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:I am currently writing a task system and I have this case where I want to send a delegate to a different thread but this delegate also captures a variable. I use this to implement a "future". Now as far as I know this delegate will allocate GC memory and I just wanted to avoid that, just for fun. Here is the code https://dpaste.dzfl.pl/cd77fce99a5b I have only worked on it a couple of hours and I am sure there are many problems with it. Basically the idea is that you can use normal lambda syntax. If you want a function that returns and int and takes an int, you can write it like this: (int i) => i If you want a function that returns an int, takes an int, but also captures and int you would write it like this (int i, int captured) => i + captured But you also have to declare the base function type without the captured variables beforehand. Fn!(FnType!(int, int), (int i, int captured) => i + captured)(42); That is how I know what the captured variables are. The only part that is currently missing are polymorphic delegates. They are not too useful if I can't pack them into the same array. I guess I have to do this with classes/interfaces. Thoughts? Has this been done before?What you have here is more functor, to the extent that you can memorize the parameters. The problem is that it's not compatible with the delegates as defined in the language. Actual D delegates get collected when we take the address and the context of a member function with "&". To allocate the delegate itself in the non GC heap is easy (with a struct that contains two pointers). Your message has motivated me to make this, thanks to a union true nogc delegates are possible: ========================================= module runnable; union Delegate(FT) { // delegate layout as defined in the D ABI struct DgMembers { void* ptr; FT funcptr; } DgMembers members; // the 2nd member is "true" D delegate type import std.array: replace; mixin("alias DT = " ~ FT.stringof.replace("function", "delegate") ~ ";"); DT dg; } void main() nogc { static struct Foo { nogc string delegate() test; nogc string source() {return "test";} } import std.experimental.allocator.mallocator; import std.experimental.allocator; Foo foo; alias DelegateT = Delegate!(typeof(&Foo.source)); // &Foo.source takes the static address, i.e in the data segment DelegateT* dg = make!DelegateT(Mallocator.instance); dg.members.ptr = &foo; dg.members.funcptr = &Foo.source; foo.test = dg.dg; assert(foo.test() == "test"); } ========================================= Here you have a true D delegate that's conform with the language.
Jun 09 2016
On Thursday, 9 June 2016 at 23:53:33 UTC, BBasile wrote:On Thursday, 9 June 2016 at 21:02:26 UTC, maik klein wrote:The problem is I currently do this struct LocalTaskQueue{ ... Array!(Box!Fiber) work; Array!(Box!Fiber) queuedWork; ... } This is where I submit my delegates to, but before that I also wrap them inside a fiber. Btw "Box" is my version of Unique. I execute a fiber inside work and if it is done I want to delete it, because it is not needed anymore. If it is on hold it transfer it from "work" to queuedWork. That means I need be able to put my delegates inside a fiber, this is the constructor for a Fiber currently. this( void delegate() dg, size_t sz = PAGESIZE*4 ) nothrow in { assert( dg ); } body { allocStack( sz ); reset( dg ); } I mean I can easily convert my "Fn" version to delegate with &foo.opCall, but then I would not know how long my "Fn" should stay alive. I think what I can do is switch my struct Fn to a class and implement my own function polymorphism. I would then do Array!(Box!Fiber) work; Array!(Box!PolymorphicFn!(void, void)) rawFunction; and when I delete a fiber from work I can delete it from rawFunction. But that seems to be so much trouble just to avoid a bit of GC.I am currently writing a task system and I have this case where I want to send a delegate to a different thread but this delegate also captures a variable. I use this to implement a "future". Now as far as I know this delegate will allocate GC memory and I just wanted to avoid that, just for fun. Here is the code https://dpaste.dzfl.pl/cd77fce99a5b I have only worked on it a couple of hours and I am sure there are many problems with it. Basically the idea is that you can use normal lambda syntax. If you want a function that returns and int and takes an int, you can write it like this: (int i) => i If you want a function that returns an int, takes an int, but also captures and int you would write it like this (int i, int captured) => i + captured But you also have to declare the base function type without the captured variables beforehand. Fn!(FnType!(int, int), (int i, int captured) => i + captured)(42); That is how I know what the captured variables are. The only part that is currently missing are polymorphic delegates. They are not too useful if I can't pack them into the same array. I guess I have to do this with classes/interfaces. Thoughts? Has this been done before?What you have here is more functor, to the extent that you can memorize the parameters. The problem is that it's not compatible with the delegates as defined in the language. Actual D delegates get collected when we take the address and the context of a member function with "&". To allocate the delegate itself in the non GC heap is easy (with a struct that contains two pointers). Your message has motivated me to make this, thanks to a union true nogc delegates are possible: ========================================= module runnable; union Delegate(FT) { // delegate layout as defined in the D ABI struct DgMembers { void* ptr; FT funcptr; } DgMembers members; // the 2nd member is "true" D delegate type import std.array: replace; mixin("alias DT = " ~ FT.stringof.replace("function", "delegate") ~ ";"); DT dg; } void main() nogc { static struct Foo { nogc string delegate() test; nogc string source() {return "test";} } import std.experimental.allocator.mallocator; import std.experimental.allocator; Foo foo; alias DelegateT = Delegate!(typeof(&Foo.source)); // &Foo.source takes the static address, i.e in the data segment DelegateT* dg = make!DelegateT(Mallocator.instance); dg.members.ptr = &foo; dg.members.funcptr = &Foo.source; foo.test = dg.dg; assert(foo.test() == "test"); } ========================================= Here you have a true D delegate that's conform with the language.
Jun 11 2016