www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Combining Delegate and Functions

reply Jesse Phillips <jessekphillips gmail.com> writes:
So looking at a post on StackOverflow about D gatchas:


"functions that form closures or are attached to objects (ie. methods) are not
the same as regular functions, instead they are called delegates, and you must
be aware of the differences."

I seem to recall the distinction was going to be going away or minimized. Is
this already done? Will it be going into D2?
Jun 30 2009
next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Tue, Jun 30, 2009 at 11:38 AM, Jesse
Phillips<jessekphillips gmail.com> wrote:

 So looking at a post on StackOverflow about D gatchas:


 "functions that form closures or are attached to objects (ie. methods) are not
the same as regular functions, instead they are called delegates, and you must
be aware of the differences."

 I seem to recall the distinction was going to be going away or minimized. Is
this already done? Will it be going into D2?


Nothing's changed in that area.  Theoretically the compiler could
implicitly convert function pointers to delegates by creating thunks.
It shouldn't be *that* hard to implement..
Jun 30 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Jarrett Billingsley wrote:

 On Tue, Jun 30, 2009 at 11:38 AM, Jesse
 Phillips<jessekphillips gmail.com> wrote:

 So looking at a post on StackOverflow about D gatchas:


 "functions that form closures or are attached to objects (ie. methods) are not
the same as regular functions, instead they are called delegates, and you must
be aware of the differences."

 I seem to recall the distinction was going to be going away or minimized. Is
this already done? Will it be going into D2?


 
 Nothing's changed in that area.  Theoretically the compiler could
 implicitly convert function pointers to delegates by creating thunks.
 It shouldn't be *that* hard to implement..


Here's something I have in a project of mine.  I didn't come up with the
idea, but I forget who did.

Note that this is not generalised.  I started working on a general one,
but gave up when it proved too fiddly and difficult to correctly rebuild
a function's argument list.

/*
 * This is utterly evil, but also REALLY cool.
 *
 * What we do here is make a dummy struct, and give it a member
 * function.  The member function has a hidden argument (this).
 *
 * We then construct a delegate of the appropriate type, set its
 * funcptr to Wrap.call and its ptr to the function pointer.
 *
 * When dg is called, it is called as (dg.funcptr(dg.ptr, ...))
 * which is the same way the (this) argument of member functions
 * is passed.
 *
 * Thus, inside the member function, we can cast the this pointer
 * back to our function pointer and call it.
 */
struct Wrap
{
    void call(Context a, PullParserSlice b)
    {
        return (cast(void function(Context, PullParserSlice)) this)
            (a,b);
    }
}

ElementBinding.Handler dg;
Wrap wrap;

dg.ptr = handler;
dg.funcptr = cast(typeof(dg.funcptr)) &wrap.call;
Jun 30 2009
parent reply BCS <ao pathlink.com> writes:
Reply to Daniel,


 Jarrett Billingsley wrote:
 

 On Tue, Jun 30, 2009 at 11:38 AM, Jesse
 Phillips<jessekphillips gmail.com> wrote:

 So looking at a post on StackOverflow about D gatchas:
 http://stackoverflow.com/questions/743319/why-isnt-the-d-language-pi

 
 "functions that form closures or are attached to objects (ie.
 methods) are not the same as regular functions, instead they are
 called delegates, and you must be aware of the differences."
 
 I seem to recall the distinction was going to be going away or
 minimized. Is this already done? Will it be going into D2?
 


 Nothing's changed in that area.  Theoretically the compiler could
 implicitly convert function pointers to delegates by creating thunks.
 It shouldn't be *that* hard to implement..
 


 Here's something I have in a project of mine.  I didn't come up with
 the idea, but I forget who did.
 
 Note that this is not generalised.  I started working on a general
 one, but gave up when it proved too fiddly and difficult to correctly
 rebuild a function's argument list.
 
 /*
 * This is utterly evil, but also REALLY cool.
 *
 * What we do here is make a dummy struct, and give it a member
 * function.  The member function has a hidden argument (this).
 *
 * We then construct a delegate of the appropriate type, set its
 * funcptr to Wrap.call and its ptr to the function pointer.
 *
 * When dg is called, it is called as (dg.funcptr(dg.ptr, ...))
 * which is the same way the (this) argument of member functions
 * is passed.
 *
 * Thus, inside the member function, we can cast the this pointer
 * back to our function pointer and call it.
 */
 struct Wrap
 {
 void call(Context a, PullParserSlice b)
 {
 return (cast(void function(Context, PullParserSlice)) this)
 (a,b);
 }
 }
 ElementBinding.Handler dg;
 Wrap wrap;
 dg.ptr = handler;
 dg.funcptr = cast(typeof(dg.funcptr)) &wrap.call;


here's a general version that should work but dosn't:

struct Wraper(F)
{
    static if(is(F Fn == Fn*) && is(Fn A == function) && is(Fn R == return))
    {
        static assert(is(F == R function(A)));

        union U
        {
            Wraper* w;
            F f;
        }

        R Call(A a)  /// BUG: this drops ref and out on A
        {
            static assert(is(F == R function(A)));

            U u;
            u.w = this;
            return u.f(a);
        }

        alias R delegate(A) ret;
    }
    else
        static assert(false, "Can't use "~Fn.stringof);

}

Wraper!(F).ret Wrap(F)(F f)
{
    Wraper!(F).U u;
    u.f = f;
    return &u.w.Call;
}

int foo(ref int i, out float x, char[] c)
{
    x = 3.14;
    return i += c.length;
}
import std.stdio;
void main()
{
    auto dg = Wrap(&foo);
    float f = 2.717;
    int i = 3;
    char[] c = "Hello world";

    auto j = dg(i, f, c);
    writef("i=%s, j=%s, f=%s\n", i,j,f);
}
Jun 30 2009
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
BCS wrote:

 here's a general version that should work but dosn't:


The problem is that out and ref aren't types; they're storage classes,
and D's current metaprogramming system has no way of accessing these.

You CAN get at them with .stringof, but that's 100% undocumented and a
MASSIVE pain in the arse.  It also means you have to write horrible
parsing code every single time you want to wrap a function.

There *really* has to be a better way.
Jun 30 2009
parent reply BCS <none anon.com> writes:
Hello Daniel,


 BCS wrote:
 

 here's a general version that should work but dosn't:
 


 The problem is that out and ref aren't types; they're storage classes,
 and D's current metaprogramming system has no way of accessing these.
 
 You CAN get at them with .stringof, but that's 100% undocumented and a
 MASSIVE pain in the arse.  It also means you have to write horrible
 parsing code every single time you want to wrap a function.
 
 There *really* has to be a better way.
 


the really painful bit is that the tuple from is(T A == function) has the 
ref/out stuff in it as shown by the first static assert but it gets dropped 
when it's used as an argument list (as show my the second [identical!] assert).
Jun 30 2009
parent Christian Kamm <kamm-incasoftware removethis.de> writes:
BCS wrote:


 Hello Daniel,
 

 BCS wrote:
 

 here's a general version that should work but dosn't:
 


 The problem is that out and ref aren't types; they're storage classes,
 and D's current metaprogramming system has no way of accessing these.
 
 You CAN get at them with .stringof, but that's 100% undocumented and a
 MASSIVE pain in the arse.  It also means you have to write horrible
 parsing code every single time you want to wrap a function.
 
 There *really* has to be a better way.
 


 
 the really painful bit is that the tuple from is(T A == function) has the
 ref/out stuff in it as shown by the first static assert but it gets
 dropped when it's used as an argument list (as show my the second
 [identical!] assert).


This is one of my favorite issues. The bug list is here 
http://d.puremagic.com/issues/show_bug.cgi?id=3106 and an interesting recent 
development is that Andrei thinks this should work: 
http://d.puremagic.com/issues/show_bug.cgi?id=2913

Maybe it will if you make the parameter storage classes part of the type and 
define them to be ignored everywhere but in function parameter declarations?

Not pretty though.
Jun 30 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Jesse Phillips:

I seem to recall the distinction was going to be going away or minimized. Is
this already done? Will it be going into D2?<


D2 docs say:
http://www.digitalmars.com/d/2.0/function.html

Future directions: Function pointers and delegates may merge into a common
syntax and be interchangeable with each other.<


I think the situation isn't changed. It's one of the *many* details that look
forgotten. There's usually time to fix them later once the big things are in
place.

Bye,
bearophile
Jun 30 2009
parent reply Eric Poggel <dnewsgroup yage3d.net> writes:
bearophile wrote:

 Jesse Phillips:

 I seem to recall the distinction was going to be going away or minimized. Is
this already done? Will it be going into D2?<


 
 D2 docs say:
 http://www.digitalmars.com/d/2.0/function.html

 Future directions: Function pointers and delegates may merge into a common
syntax and be interchangeable with each other.<


 
 I think the situation isn't changed. It's one of the *many* details that look
forgotten. There's usually time to fix them later once the big things are in
place.
 
 Bye,
 bearophile


Perhaps "functions" should be eliminated except for being accessed via a 
   delegate's .funcptr property (which could also be used for c 
compatibility).  In addition,

int foo(int a) {};

would be the same as:

const int delegate(int) foo = delegate int(int a) {};

I don't have D2 installed, but the above fails to compile in d1 
(non-constant expression __dgliteral1).  I've also always found the 
above syntax confusing, on the left we have "int delegate" and on the 
right, "delegate int".  Perhaps it should always be "int delegate", and 
whether a function body is present determines whether the expression is 
a type or an anonymous function.  "int delegate" is also similar to how 
other functions are defined, since "delegate" simply replaces the 
function name, and it's similar to other languages, such as ECMA script 
v4 (ActionScript 3)

Perhaps also, functions should have available an arguments property that 
would be a struct of the functions arguments, of type 
ParamTypeTuple!(func), e.g.

void foo(int a)
{	bar(foo.arguments);
}
void bar(int a)
{	writefln(bar.arguments.a); // writes 3
	writefln(a); // writes 3
}
foo(3);

To me, this is much more straightforward than using std.stdarg when 
varargs come into play.  Perhaps foo.closure could reference any 
variables captured in a closure.

Finally, in D2, was a way ever figured out to differentiate between 
delegates and closures, so that delegates can be used w/o heap allocation?

Hopefully I didn't come across as dictating policy.  These are just my 
ideas for simplifying functions/delegates in D2 or beyond.
Jun 30 2009
next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Tue, Jun 30, 2009 at 9:21 PM, Eric Poggel<dnewsgroup yage3d.net> wrote:


 I don't have D2 installed, but the above fails to compile in d1
 (non-constant expression __dgliteral1). =A0I've also always found the abo=


ve

 syntax confusing, on the left we have "int delegate" and on the right,
 "delegate int". =A0Perhaps it should always be "int delegate", and whethe=


r a

 function body is present determines whether the expression is a type or a=


n

 anonymous function. =A0"int delegate" is also similar to how other functi=


ons

 are defined, since "delegate" simply replaces the function name, and it's
 similar to other languages, such as ECMA script v4 (ActionScript 3)


It would make parsing expressions prohibitively difficult.  Consider:

func(X); // X is just a name
func(X[]); // X[] is a whole-array slice
func(X[] delegate() { return new X[5]; }); // ughh

In the last case in particular, the compiler would already have parsed
X[] as a slice expression, and then bam!  'delegate'.  Shit, now it
has to rewrite or reparse the previous 'X[]' as a type instead of an
expression.

Furthermore, having 'delegate' come first in the literals lets you
skip the return type entirely with little change in the parsing:

func(delegate() { return new X[5]; });


 Perhaps also, functions should have available an arguments property that
 would be a struct of the functions arguments, of type ParamTypeTuple!(fun=


c),

 e.g.

 void foo(int a)
 { =A0 =A0 =A0 bar(foo.arguments);
 }
 void bar(int a)
 { =A0 =A0 =A0 writefln(bar.arguments.a); // writes 3
 =A0 =A0 =A0 =A0writefln(a); // writes 3
 }
 foo(3);

 To me, this is much more straightforward than using std.stdarg when varar=


gs

 come into play. =A0Perhaps foo.closure could reference any variables capt=


ured

 in a closure.


You're basically talking about variadic template args:

void foo(Args...)(Args args)
{
    writefln(args);
}

void bar(Args...)(Args args)
{
    foo(args);
}

bar(1, 2, 3); // prints 123


 Finally, in D2, was a way ever figured out to differentiate between
 delegates and closures, so that delegates can be used w/o heap allocation=


?

IIRC the 'scope' attribute on a delegate function parameter makes it
so that if you pass a delegate literal to that param, it won't be
allocated as a closure.  But that's still rather restrictive, since it
*forces* you to pass the closure as a parameter, which starts to get
really ugly if the delegate you're passing is large.
Jun 30 2009
parent reply Eric Poggel <dnewsgroup yage3d.net> writes:
Jarrett Billingsley wrote:

 On Tue, Jun 30, 2009 at 9:21 PM, Eric Poggel<dnewsgroup yage3d.net> wrote:
 

 I don't have D2 installed, but the above fails to compile in d1
 (non-constant expression __dgliteral1).  I've also always found the above
 syntax confusing, on the left we have "int delegate" and on the right,
 "delegate int".  Perhaps it should always be "int delegate", and whether a
 function body is present determines whether the expression is a type or an
 anonymous function.  "int delegate" is also similar to how other functions
 are defined, since "delegate" simply replaces the function name, and it's
 similar to other languages, such as ECMA script v4 (ActionScript 3)


 
 It would make parsing expressions prohibitively difficult.  Consider:
 
 func(X); // X is just a name
 func(X[]); // X[] is a whole-array slice
 func(X[] delegate() { return new X[5]; }); // ughh
 
 In the last case in particular, the compiler would already have parsed
 X[] as a slice expression, and then bam!  'delegate'.  Shit, now it
 has to rewrite or reparse the previous 'X[]' as a type instead of an
 expression.
 
 Furthermore, having 'delegate' come first in the literals lets you
 skip the return type entirely with little change in the parsing:
 
 func(delegate() { return new X[5]; });
 

 Perhaps also, functions should have available an arguments property that
 would be a struct of the functions arguments, of type ParamTypeTuple!(func),
 e.g.

 void foo(int a)
 {       bar(foo.arguments);
 }
 void bar(int a)
 {       writefln(bar.arguments.a); // writes 3
        writefln(a); // writes 3
 }
 foo(3);

 To me, this is much more straightforward than using std.stdarg when varargs
 come into play.  Perhaps foo.closure could reference any variables captured
 in a closure.


 
 You're basically talking about variadic template args:
 
 void foo(Args...)(Args args)
 {
     writefln(args);
 }
 
 void bar(Args...)(Args args)
 {
     foo(args);
 }
 
 bar(1, 2, 3); // prints 123
 

 Finally, in D2, was a way ever figured out to differentiate between
 delegates and closures, so that delegates can be used w/o heap allocation?


 
 IIRC the 'scope' attribute on a delegate function parameter makes it
 so that if you pass a delegate literal to that param, it won't be
 allocated as a closure.  But that's still rather restrictive, since it
 *forces* you to pass the closure as a parameter, which starts to get
 really ugly if the delegate you're passing is large.



I use variadic template args also, but templates are often confusing to 
new users, hence my suggestion.  Also, if you want to do something like 
what I habe below, I believe you have to resort to darker tuple magic.

void foo(int a, ...)
{       bar(foo.arguments); // also passes a
}
void bar(...)
{
}

Also, do you see any problems that would arise from replacing functions 
with delegates internally, so that no more conversions between them 
would be required?
Jul 03 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Eric Poggel wrote:

 ...
 
 Also, do you see any problems that would arise from replacing functions
 with delegates internally, so that no more conversions between them
 would be required?


Say goodbye to being able to call non-D libraries that take callbacks?
Jul 03 2009
parent reply Eric Poggel <dnewsgroup yage3d.net> writes:
Daniel Keep wrote:

 
 Eric Poggel wrote:

 ...

 Also, do you see any problems that would arise from replacing functions
 with delegates internally, so that no more conversions between them
 would be required?


 
 Say goodbye to being able to call non-D libraries that take callbacks?


cfunc(&foo.funcptr)

?
Jul 04 2009
next sibling parent BCS <none anon.com> writes:
Hello Eric,


 Daniel Keep wrote:
 

 Eric Poggel wrote:
 

 ...
 
 Also, do you see any problems that would arise from replacing
 functions with delegates internally, so that no more conversions
 between them would be required?
 


 Say goodbye to being able to call non-D libraries that take
 callbacks?
 


 cfunc(&foo.funcptr)
 
 ?
 


that would get you the address of a reference to the thunk. OTOH foo.ptr 
might work.
Jul 04 2009
prev sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Eric Poggel wrote:

 Daniel Keep wrote:

 Eric Poggel wrote:

 ...

 Also, do you see any problems that would arise from replacing functions
 with delegates internally, so that no more conversions between them
 would be required?


 Say goodbye to being able to call non-D libraries that take callbacks?


 
 cfunc(&foo.funcptr)
 
 ?


Umm, no.  Delegates have a different calling convention to function
pointers.  It will expect the context pointer in EAX.
Jul 04 2009
prev sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Fri, Jul 3, 2009 at 3:36 PM, Eric Poggel<dnewsgroup yage3d.net> wrote:

 I use variadic template args also, but templates are often confusing to n=


ew

 users, hence my suggestion.


I don't really buy that argument, only because if you're going to use
D without templates?  you might as well use Java pre-generics.
Really.  Generic programming is what makes D _D_.


 Also, if you want to do something like what I
 habe below, I believe you have to resort to darker tuple magic.

 void foo(int a, ...)
 { =A0 =A0 =A0 bar(foo.arguments); // also passes a
 }
 void bar(...)
 {
 }


void foo(T...)(int a, T args) { bar(a, args); }
void bar(T...)(T args) {}

Also, forwarding varargs with normal ...-style variadic functions is
currently - unexcusably - impossible.


 Also, do you see any problems that would arise from replacing functions w=


ith

 delegates internally, so that no more conversions between them would be
 required?


I would see no problem in allowing an implicit conversion from
function pointer to delegate.  All the compiler would do is insert a
thunk.  Then APIs would just always take delegates, and "raw" function
pointers would still be useful for cases where a delegate is
undesirable (i.e. when being passed to a library that the GC doesn't
scan, and for compatibility with the C ABI).
Jul 03 2009
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Finally got it hammered into shape.

http://gist.github.com/140507

Enjoy.


Incidentally, I released it under MIT because I've read some things
lately that have turned me off Public Domain dedications; specifically
that if you public domain something, that potentially leaves an avenue
for people to sue you if the code contains bugs and that some countries
don't actually allow for public domain dedications at all.

MIT was the least restrictive common license I could find, so I think
I'll just stick to that from now on for stuff like this.  That said, if
anyone wants to incorporate this into a larger library and needs to
apply a different license, let me know.
Jul 04 2009
prev sibling parent reply Jason House <jason.james.house gmail.com> writes:
Eric Poggel Wrote:

 I don't have D2 installed, but the above fails to compile in d1 
 (non-constant expression __dgliteral1).  



Here's the code I use in D2:

R delegate(T) toDelegate(R, T...)(R function(T) f){
   return delegate R(T t){return f(t);};
}
Jun 30 2009
parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Jason House wrote:

 Eric Poggel Wrote:

 I don't have D2 installed, but the above fails to compile in d1 
 (non-constant expression __dgliteral1).  


 
 
 Here's the code I use in D2:
 
 R delegate(T) toDelegate(R, T...)(R function(T) f){
    return delegate R(T t){return f(t);};
 }


What happens when you do this, though?

void foo(ref int v) { v = 42; }

void main()
{
    auto dg = toDelegate(&foo);
    int v; dg(v);
    writefln("%s", v);
}
Jun 30 2009
prev sibling parent reply Moritz Warning <moritzwarning web.de> writes:
On Tue, 30 Jun 2009 11:38:21 -0400, Jesse Phillips wrote:


 So looking at a post on StackOverflow about D gatchas:
 http://stackoverflow.com/questions/743319/why-isnt-the-d-language-




 
 "functions that form closures or are attached to objects (ie. methods)
 are not the same as regular functions, instead they are called
 delegates, and you must be aware of the differences."
 
 I seem to recall the distinction was going to be going away or
 minimized. Is this already done? Will it be going into D2?


fwiw, that's what I use to convert a function to a delegate (not my 
invention):

R delegate(T) toDg(R, T...)(R function(T) fp)
{
	struct dg
	{
		R opCall(T t)
		{
			return (cast(R function(T)) this) (t);
		}
	}
	
	R delegate(T) t;
	t.ptr = fp;
	t.funcptr = &dg.opCall;
	return t;
}

Note that this won't work for ref arguments.
Jun 30 2009
parent reply Eric Poggel <dnewsgroup yage3d.net> writes:
Moritz Warning wrote:

 On Tue, 30 Jun 2009 11:38:21 -0400, Jesse Phillips wrote:
 

 So looking at a post on StackOverflow about D gatchas:
 http://stackoverflow.com/questions/743319/why-isnt-the-d-language-




 "functions that form closures or are attached to objects (ie. methods)
 are not the same as regular functions, instead they are called
 delegates, and you must be aware of the differences."

 I seem to recall the distinction was going to be going away or
 minimized. Is this already done? Will it be going into D2?


 
 fwiw, that's what I use to convert a function to a delegate (not my 
 invention):
 
 R delegate(T) toDg(R, T...)(R function(T) fp)
 {
 	struct dg
 	{
 		R opCall(T t)
 		{
 			return (cast(R function(T)) this) (t);
 		}
 	}
 	
 	R delegate(T) t;
 	t.ptr = fp;
 	t.funcptr = &dg.opCall;
 	return t;
 }
 
 Note that this won't work for ref arguments.


I use this too, but my point was that it'd be nice if hacks like this 
weren't required, e.g. by doing away with functions except for C 
compatibility and making everything a delegate internally.
Jul 03 2009
parent Moritz Warning <moritzwarning web.de> writes:
On Fri, 03 Jul 2009 15:30:27 -0400, Eric Poggel wrote:


 Moritz Warning wrote:

 On Tue, 30 Jun 2009 11:38:21 -0400, Jesse Phillips wrote:
 

 So looking at a post on StackOverflow about D gatchas:
 http://stackoverflow.com/questions/743319/why-isnt-the-d-language-




 "functions that form closures or are attached to objects (ie. methods)
 are not the same as regular functions, instead they are called
 delegates, and you must be aware of the differences."

 I seem to recall the distinction was going to be going away or
 minimized. Is this already done? Will it be going into D2?


 
 fwiw, that's what I use to convert a function to a delegate (not my
 invention):
 
 R delegate(T) toDg(R, T...)(R function(T) fp) {
 	struct dg
 	{
 		R opCall(T t)
 		{
 			return (cast(R function(T)) this) (t);
 		}
 	}
 	
 	R delegate(T) t;
 	t.ptr = fp;
 	t.funcptr = &dg.opCall;
 	return t;
 }
 
 Note that this won't work for ref arguments.


 
 I use this too, but my point was that it'd be nice if hacks like this
 weren't required, e.g. by doing away with functions except for C
 compatibility and making everything a delegate internally.


I would like to omit the use of workarounds like this
and be able to implicitly convert functions to delegates.

I don't feel comfortable with dropping functions altogether.
Jul 03 2009