www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - One minute to twelve: last chance to fix D2 ABI?

reply Lionello Lunesu <lio lunesu.remove.com> writes:
The following has been discussed for years, and the biggest problem
(years ago) was that it was too late to change the D (1) ABI.

Is the D2 ABI compatible with D1? Should it be? If not, please consider
the following change:

alias void delegate() action_d;
void Callback() { }             // global
action_d = &Callback;           // should work

A function should be assignable to a delegate (but not vice versa*.)
This can work, since the code invoking the delegate will end up calling
the function with a context pointer ("this") in a register that's simply
ignored by the code in the function.

This is fairly easy too: the calling convention for functions and
delegates should be identical, except for the context pointer, which
should be stored in a register that's unused in the function calling
convention.

The reason for this change is that many times classes expose delegates
for callbacks, but it's impossible to bind a function to those
delegates. The code above can easily be made to compile:

action_d = { Callback(); }; //wrapped, works

but causes a dummy function to be called, only because the parameter
ABI's don't overlap:

_D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
        assume  CS:_D1t4mainFZi12__dgliteral1MFZi
L0:             push    EAX
                call    near ptr _D1t8FunctionFZi ;global function
                pop     ECX
                ret
_D1t4mainFZi12__dgliteral1MFZi  ends

Of course, all of this applies to extern "D" code only.

* Binding a delegate to a function pointer will always need a (dynamic)
thunk, because the function will have been invoked without any context
pointer and it's the thunk's responsibility to assign the correct
context pointer.
Mar 02 2010
next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:
 The following has been discussed for years, and the biggest problem
 (years ago) was that it was too late to change the D (1) ABI.

 Is the D2 ABI compatible with D1? Should it be? If not, please consider
 the following change:

 alias void delegate() action_d;
 void Callback() { }             // global
 action_d = &Callback;           // should work

 A function should be assignable to a delegate (but not vice versa*.)
 This can work, since the code invoking the delegate will end up calling
 the function with a context pointer ("this") in a register that's simply
 ignored by the code in the function.

Except this also effects member functions, nested functions, etc.
 This is fairly easy too: the calling convention for functions and
 delegates should be identical, except for the context pointer, which
 should be stored in a register that's unused in the function calling
 convention.

This breaks the synergy of returns in EAX and params in EAX.
 The reason for this change is that many times classes expose delegates
 for callbacks, but it's impossible to bind a function to those
 delegates. The code above can easily be made to compile:

 action_d = { Callback(); }; //wrapped, works

 but causes a dummy function to be called, only because the parameter
 ABI's don't overlap:

 _D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
         assume  CS:_D1t4mainFZi12__dgliteral1MFZi
 L0:             push    EAX
                 call    near ptr _D1t8FunctionFZi ;global function
                 pop     ECX
                 ret
 _D1t4mainFZi12__dgliteral1MFZi  ends

 Of course, all of this applies to extern "D" code only.

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.

It would be fairly easy for D (or anyone else) to implicitly define a simple adapter that stores the function pointer in the delegate's this pointer. The adapter function pointer points to one of two common stub functions which either calls the this pointer or pops a value off the stack into EAX and then calls the this pointer.
Mar 02 2010
parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 10:49, Robert Jacques wrote:
 On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu
 <lio lunesu.remove.com> wrote:
 The following has been discussed for years, and the biggest problem
 (years ago) was that it was too late to change the D (1) ABI.

 Is the D2 ABI compatible with D1? Should it be? If not, please consider
 the following change:

 alias void delegate() action_d;
 void Callback() { }             // global
 action_d = &Callback;           // should work

 A function should be assignable to a delegate (but not vice versa*.)
 This can work, since the code invoking the delegate will end up calling
 the function with a context pointer ("this") in a register that's simply
 ignored by the code in the function.

Except this also effects member functions, nested functions, etc.

Well, yes. That doesn't make it any harder or less possible.
 This is fairly easy too: the calling convention for functions and
 delegates should be identical, except for the context pointer, which
 should be stored in a register that's unused in the function calling
 convention.

This breaks the synergy of returns in EAX and params in EAX.

How exactly? I don't understand. What does the return value have to do with any of this?
 The reason for this change is that many times classes expose delegates
 for callbacks, but it's impossible to bind a function to those
 delegates. The code above can easily be made to compile:

 action_d = { Callback(); }; //wrapped, works

 but causes a dummy function to be called, only because the parameter
 ABI's don't overlap:

 _D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
         assume  CS:_D1t4mainFZi12__dgliteral1MFZi
 L0:             push    EAX
                 call    near ptr _D1t8FunctionFZi ;global function
                 pop     ECX
                 ret
 _D1t4mainFZi12__dgliteral1MFZi  ends

 Of course, all of this applies to extern "D" code only.

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.

It would be fairly easy for D (or anyone else) to implicitly define a simple adapter that stores the function pointer in the delegate's this pointer. The adapter function pointer points to one of two common stub functions which either calls the this pointer or pops a value off the stack into EAX and then calls the this pointer.

It is easy. Like I should, you could just enter a pair of {} and it would work. And to prevent context capturing (and heap allocation) you could use such an adapter function. But without changing the ABI, even in the best case you would have a wrapper function with an extra call operation, which simply should not be necessary. L.
Mar 03 2010
parent Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 23:51, Robert Jacques wrote:
 At the cost of forcing every single member function call to do an
 additional stack push? DMD doesn't pass args in ECX and EDX for a
 reason. Adapting free functions to delegates is a rare enough operation
 that making everything else slower to save one function call isn't worth
 it.

That's a great point. I hadn't thought about it. Thanks. L.
Mar 04 2010
prev sibling next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
That or just use something like this: http://gist.github.com/263824
Mar 02 2010
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
I consider the function-delegate thing a reasonably solved problem, starting in
the next release of Phobos.  See:

http://dsource.org/projects/phobos/changeset/1428
Mar 02 2010
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
dsimcha wrote:
 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:
 
 http://dsource.org/projects/phobos/changeset/1428

Nice. Btw, I'm guessing that pragma(msg) in line 562 should be commented out before release? :)
Mar 02 2010
prev sibling next sibling parent Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 11:51, dsimcha wrote:
 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:
 
 http://dsource.org/projects/phobos/changeset/1428

But there's still the unnecessary call that has to be made. My point was that it fixable by just adjusting the ABI a little. Perhaps the status quo isn't that bad if the compiler could use toDelegate transparently whenever a function pointer is assigned to a delegate. L.
Mar 03 2010
prev sibling parent Max Samukha <spambox d-coding.com> writes:
On 03.03.2010 5:51, dsimcha wrote:
 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:

 http://dsource.org/projects/phobos/changeset/1428

Your code works only if return/parameter types are accessible from std.functional. An implementation that handles any types is way more complex. You'll have to extract storage classes from the function pointer type and then reconstruct the delegate type using those extracted storage classes combined with the parameter types as elements of the parameter type tuple. An example demonstrating what is meant: auto toDelegate(F)(F fp) { ... alias ParameterTypeTuple!(F) Params; // CTFE generating the delegate type string static string genDgStr() { auto paramSCs = extractParamSCs!(F); auto dgStr = extractReturnSCs!(F) ~ " ReturnType!(F) delegate("; foreach (i, paramSC; paramSCs) { if (i) dgStr ~= ", "; dgStr ~= paramSCs[i] ~ " Params[" ~ to!string(i) ~ "]"; } return dgStr ~ ")"; } mixin ("alias " ~ genDgStr() ~ " Dg;"); ... } So, for the function type 'ref SomeAlienStruct function(ref SomeAlienStruct s)', the mixed-in delegate type alias will be: alias ref ReturnType!(F) delegate(ref Params[0]) Dg; Note that there are no stringof's involved in the alias declaration. Terrible.
Mar 03 2010
prev sibling next sibling parent reply Steve Teale <steve.teale britseyeview.com> writes:
 
 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.

At present, D's way of handling delegates requires patching GCC for the GDC implementation. If an ABI change was made it would be nice if it fitted into the GCC trampoline model. Steve
Mar 02 2010
next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale  
<steve.teale britseyeview.com> wrote:
 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.

At present, D's way of handling delegates requires patching GCC for the GDC implementation. If an ABI change was made it would be nice if it fitted into the GCC trampoline model. Steve

Um, how exactly? I just read the wikipedia page on GCC trampoline and it a) targeted only at nested functions b) utilizes self-modifying code which is disabled on any sensible platform c) would require dynamic allocation and d) actually uses more memory than D's method. Some time ago there was a link to an excellent article (by Walter? Don?) on different ways to do this. I remember d's way coming out the best, but I've seemed to have lost the link.
Mar 02 2010
prev sibling next sibling parent Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 03 Mar 2010 01:38:58 -0500, Robert Jacques wrote:

 On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale
 <steve.teale britseyeview.com> wrote:
 * Binding a delegate to a function pointer will always need a
 (dynamic) thunk, because the function will have been invoked without
 any context pointer and it's the thunk's responsibility to assign the
 correct context pointer.

At present, D's way of handling delegates requires patching GCC for the GDC implementation. If an ABI change was made it would be nice if it fitted into the GCC trampoline model. Steve

Um, how exactly? I just read the wikipedia page on GCC trampoline and it a) targeted only at nested functions b) utilizes self-modifying code which is disabled on any sensible platform c) would require dynamic allocation and d) actually uses more memory than D's method. Some time ago there was a link to an excellent article (by Walter? Don?) on different ways to do this. I remember d's way coming out the best, but I've seemed to have lost the link.

I'm just quoting from something I noticed a couple of days ago. In David Friedman's make file for GDC there is: # Enables correct implementation of delegates. This requires patching GCC. D_EXTRA_DEFINES += -DD_NO_TRAMPOLINES=1 Steve
Mar 03 2010
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 10:34:48 -0500, Steve Teale  
<steve.teale britseyeview.com> wrote:

 On Wed, 03 Mar 2010 01:38:58 -0500, Robert Jacques wrote:

 On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale
 <steve.teale britseyeview.com> wrote:
 * Binding a delegate to a function pointer will always need a
 (dynamic) thunk, because the function will have been invoked without
 any context pointer and it's the thunk's responsibility to assign the
 correct context pointer.

At present, D's way of handling delegates requires patching GCC for the GDC implementation. If an ABI change was made it would be nice if it fitted into the GCC trampoline model. Steve

Um, how exactly? I just read the wikipedia page on GCC trampoline and it a) targeted only at nested functions b) utilizes self-modifying code which is disabled on any sensible platform c) would require dynamic allocation and d) actually uses more memory than D's method. Some time ago there was a link to an excellent article (by Walter? Don?) on different ways to do this. I remember d's way coming out the best, but I've seemed to have lost the link.

I'm just quoting from something I noticed a couple of days ago. In David Friedman's make file for GDC there is: # Enables correct implementation of delegates. This requires patching GCC. D_EXTRA_DEFINES += -DD_NO_TRAMPOLINES=1 Steve

I think that's because GCC's internal nested function AST construct would normally use trampolines, but D defines them as delegates (IIRC). As this hack is around GCC's AST handling and not D's ABI, it would be required regardless. BTW nested function support is a language extension specific to GCC.
Mar 03 2010
prev sibling next sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 05:28:46 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:
 On 3-3-2010 10:49, Robert Jacques wrote:
 On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu


 Except this also effects member functions, nested functions, etc.

Well, yes. That doesn't make it any harder or less possible.

It greatly increases the scope of the change, which does increase the implementation difficulty. [snip]
 This breaks the synergy of returns in EAX and params in EAX.

How exactly? I don't understand. What does the return value have to do with any of this?

The ABI has been designed so that there is a synergy between the return value of a function (EAX) and where the last parameter to the next function is (also EAX). This allows function chaining to occur without mucking with register moves, etc.
 It would be fairly easy for D (or anyone else) to implicitly define a
 simple adapter that stores the function pointer in the delegate's this
 pointer. The adapter function pointer points to one of two common stub
 functions which either calls the this pointer or pops a value off the
 stack into EAX and then calls the this pointer.

It is easy. Like I should, you could just enter a pair of {} and it would work. And to prevent context capturing (and heap allocation) you could use such an adapter function. But without changing the ABI, even in the best case you would have a wrapper function with an extra call operation, which simply should not be necessary. L.

At the cost of forcing every single member function call to do an additional stack push? DMD doesn't pass args in ECX and EDX for a reason. Adapting free functions to delegates is a rare enough operation that making everything else slower to save one function call isn't worth it.
Mar 03 2010
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 04 Mar 2010 05:00:46 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:

 On 3-3-2010 23:51, Robert Jacques wrote:
 At the cost of forcing every single member function call to do an
 additional stack push? DMD doesn't pass args in ECX and EDX for a
 reason. Adapting free functions to delegates is a rare enough operation
 that making everything else slower to save one function call isn't worth
 it.

That's a great point. I hadn't thought about it. Thanks. L.

No problem. I'd never have thought about it and think it through without you raising your point. The really odd thing is that 2-word structs are returned in register, but not sent in registers. Maybe this will change in x86.
Mar 04 2010