www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Convert a delegate to a function (i.e. make a thunk)

reply "Mehrdad" <wfunction hotmail.com> writes:
It would be nice if there was a way to convert delegates to 
functions/thunks, because certain annoying tasks (e.g. 'WndProc's in 
Windows) would become a heck of a lot easier.

Is there any way to already do this? If not, how about adding a toFunction() 
method in std.functional?

(This would of course require making data executable, so that's why it's 
painful to get right. ATL uses a pool of some sort, I think... so if 
toFunction() did this internally, with a separate pool, it would reduce a 
lot of memory management burden from the programmer.)
Apr 25 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, April 25, 2012 14:02:53 Mehrdad wrote:

 It would be nice if there was a way to convert delegates to
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in
 Windows) would become a heck of a lot easier.
 
 Is there any way to already do this? If not, how about adding a toFunction()
 method in std.functional?
 
 (This would of course require making data executable, so that's why it's
 painful to get right. ATL uses a pool of some sort, I think... so if
 toFunction() did this internally, with a separate pool, it would reduce a
 lot of memory management burden from the programmer.)


You could have a function which took a delegate and called it (which could 
probably be done with void* if the function has to be an extern(C) function), 
or if the function didn't need to be pure, then you could have a function 
which called a module-level variable which the delegate had been assigned to. 
But you can't convert a delegate to a function without passing the delegate to 
the function one way or another, because the function doesn't have a context 
(not being a delegate).

- Jonathan M Davis
Apr 25 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 25-04-2012 23:27, Jonathan M Davis wrote:

 On Wednesday, April 25, 2012 14:02:53 Mehrdad wrote:

 It would be nice if there was a way to convert delegates to
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in
 Windows) would become a heck of a lot easier.

 Is there any way to already do this? If not, how about adding a toFunction()
 method in std.functional?

 (This would of course require making data executable, so that's why it's
 painful to get right. ATL uses a pool of some sort, I think... so if
 toFunction() did this internally, with a separate pool, it would reduce a
 lot of memory management burden from the programmer.)


 You could have a function which took a delegate and called it (which could
 probably be done with void* if the function has to be an extern(C) function),
 or if the function didn't need to be pure, then you could have a function
 which called a module-level variable which the delegate had been assigned to.
 But you can't convert a delegate to a function without passing the delegate to
 the function one way or another, because the function doesn't have a context
 (not being a delegate).

 - Jonathan M Davis


http://en.wikipedia.org/wiki/Thunk_(functional_programming)
http://catb.org/jargon/html/T/thunk.html

You may be able to do something like this with libffi.

(See https://github.com/lycus/libffi-d for a D binding.)

-- 
- Alex
Apr 25 2012
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis 
wrote:

 But you can't convert a delegate to a function without passing 
 the delegate to
 the function one way or another, because the function doesn't 
 have a context
 (not being a delegate).


That's the whole point of the question - you _can_ pull this off 
if you create a small code fragment somewhere in memory which 
calls the delegate with that specific context argument, and then 
set your function pointer to its address (similar to thunks in 
most vtable implementations or SEH, if you are familiar with 
these). Now, the only problem is that executing data is not quite 
easy, mostly because of security mechanisms (W^X, NX and friends).

As for the original question, I didn't need the feature so far 
(many C libraries provide a void* bit of data to use at the 
client's discretion), but if you need inspiration on how to 
implement it, I'd look at projects like Mono and FFI libraries 
which likely need to provide similar functionality.

Would probably be a good addition to Phobos.

David
Apr 25 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 26-04-2012 00:06, David Nadlinger wrote:

 On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis wrote:

 But you can't convert a delegate to a function without passing the
 delegate to
 the function one way or another, because the function doesn't have a
 context
 (not being a delegate).


 That's the whole point of the question - you _can_ pull this off if you
 create a small code fragment somewhere in memory which calls the
 delegate with that specific context argument, and then set your function
 pointer to its address (similar to thunks in most vtable implementations
 or SEH, if you are familiar with these). Now, the only problem is that
 executing data is not quite easy, mostly because of security mechanisms
 (W^X, NX and friends).

 As for the original question, I didn't need the feature so far (many C
 libraries provide a void* bit of data to use at the client's
 discretion), but if you need inspiration on how to implement it, I'd
 look at projects like Mono and FFI libraries which likely need to
 provide similar functionality.

 Would probably be a good addition to Phobos.

 David


libffi can do it. There's a reason I wanted std.ffi. ;) (Unfortunately 
interest in such a module seemed low earlier.)

-- 
- Alex
Apr 25 2012
parent "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 22:25:10 UTC, Alex Rønne Petersen 
wrote:

 libffi can do it. There's a reason I wanted std.ffi. ;) 
 (Unfortunately interest in such a module seemed low earlier.)


Ah, you beat me to it - didn't see your answer while I was at the 
reply screen. ;)

David
Apr 25 2012
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 26, 2012 at 12:06:54AM +0200, David Nadlinger wrote:

 On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis wrote:

But you can't convert a delegate to a function without passing the
delegate to the function one way or another, because the function
doesn't have a context (not being a delegate).


 
 That's the whole point of the question - you _can_ pull this off if
 you create a small code fragment somewhere in memory which calls the
 delegate with that specific context argument, and then set your
 function pointer to its address (similar to thunks in most vtable
 implementations or SEH, if you are familiar with these). Now, the only
 problem is that executing data is not quite easy, mostly because of
 security mechanisms (W^X, NX and friends).


This is like GCC's implementation of nested function pointers using
trampolines:

	http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html

These nested function pointers have access to their containing lexical
scope, even though they are just regular, non-fat pointers.

Something similar can be used here, though for full-fledged D delegates
the trampoline would need to be in executable heap memory. I believe
nowadays heap memory is non-executable due to security concerns with
heap overflow exploits, right? So this may require special treatment.


T

-- 
PNP = Plug 'N' Pray
Apr 25 2012
next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 22:31:59 UTC, H. S. Teoh wrote:

 I believe
 nowadays heap memory is non-executable due to security concerns 
 with
 heap overflow exploits, right? So this may require special 
 treatment.


Yes; it usually boils down to allocating a dedicated page and 
then setting the executable flag after you are done with setting 
up your code (e.g. using VirtualAlloc/VirtualProtect on Windows).

David
Apr 25 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 26, 2012 at 12:47:42AM +0200, David Nadlinger wrote:

 On Wednesday, 25 April 2012 at 22:31:59 UTC, H. S. Teoh wrote:

I believe nowadays heap memory is non-executable due to security
concerns with heap overflow exploits, right? So this may require
special treatment.


 
 Yes; it usually boils down to allocating a dedicated page and then
 setting the executable flag after you are done with setting up your
 code (e.g. using VirtualAlloc/VirtualProtect on Windows).


[...]

Does turning on the executable flag also make it read-only?  If so, we'd
need to allocate a page per wrapper function, which seems a bit
excessive. Or is it possible to turn it off temporarily, add more code,
and turn it back on? (Seems like that should be prohibited, since
otherwise it would just be too easy to exploit, and would defeat the
purpose of protecting it in the first place.)


T

-- 
Arise, you prisoners of Windows
Arise, you slaves of Redmond, Wash,
The day and hour soon are coming
When all the IT folks say "Gosh!"
It isn't from a clever lawsuit
That Windowsland will finally fall,
But thousands writing open source code
Like mice who nibble through a wall.
-- The Linux-nationale by Greg Baker
Apr 25 2012
prev sibling parent David Brown <davidb davidb.org> writes:
On 2012-04-25, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:


 This is like GCC's implementation of nested function pointers using
 trampolines:

 	http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html

 These nested function pointers have access to their containing lexical
 scope, even though they are just regular, non-fat pointers.

 Something similar can be used here, though for full-fledged D delegates
 the trampoline would need to be in executable heap memory. I believe
 nowadays heap memory is non-executable due to security concerns with
 heap overflow exploits, right? So this may require special treatment.


Gcc (at least on Linux) does a few things.  The trampoline is on
the stack, but the generated code:

  - Creates a special section

    .section .note.GNU-stack,"x", progbits

    in the assembly that indicates to the linker that the resulting
    elf file needs to allow the stack to be executable.

  - Depending on the CPU, it may call a libgcc function
    __clear_cache.  x86 is coherent, so this isn't needed, but it is
    in the general case.

David
Apr 26 2012
prev sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Mehrdad" <wfunction hotmail.com> wrote in message 
news:jn9ops$13bs$1 digitalmars.com...

 It would be nice if there was a way to convert delegates to 
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in 
 Windows) would become a heck of a lot easier.

 Is there any way to already do this? If not, how about adding a 
 toFunction() method in std.functional?


Well, there's this: http://pastebin.com/XwEb2cpm

But it's pre-3797, and probably doesn't work anywhere except win32.
Apr 26 2012