www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Convert a delegate to a function (i.e. make a thunk)

reply "Mehrdad" <wfunction hotmail.com> writes:
It would be nice if there was a way to convert delegates to 
functions/thunks, because certain annoying tasks (e.g. 'WndProc's in 
Windows) would become a heck of a lot easier.

Is there any way to already do this? If not, how about adding a toFunction() 
method in std.functional?

(This would of course require making data executable, so that's why it's 
painful to get right. ATL uses a pool of some sort, I think... so if 
toFunction() did this internally, with a separate pool, it would reduce a 
lot of memory management burden from the programmer.) 
Apr 25 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, April 25, 2012 14:02:53 Mehrdad wrote:
 It would be nice if there was a way to convert delegates to
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in
 Windows) would become a heck of a lot easier.
 
 Is there any way to already do this? If not, how about adding a toFunction()
 method in std.functional?
 
 (This would of course require making data executable, so that's why it's
 painful to get right. ATL uses a pool of some sort, I think... so if
 toFunction() did this internally, with a separate pool, it would reduce a
 lot of memory management burden from the programmer.)

You could have a function which took a delegate and called it (which could probably be done with void* if the function has to be an extern(C) function), or if the function didn't need to be pure, then you could have a function which called a module-level variable which the delegate had been assigned to. But you can't convert a delegate to a function without passing the delegate to the function one way or another, because the function doesn't have a context (not being a delegate). - Jonathan M Davis
Apr 25 2012
next sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 25-04-2012 23:27, Jonathan M Davis wrote:
 On Wednesday, April 25, 2012 14:02:53 Mehrdad wrote:
 It would be nice if there was a way to convert delegates to
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in
 Windows) would become a heck of a lot easier.

 Is there any way to already do this? If not, how about adding a toFunction()
 method in std.functional?

 (This would of course require making data executable, so that's why it's
 painful to get right. ATL uses a pool of some sort, I think... so if
 toFunction() did this internally, with a separate pool, it would reduce a
 lot of memory management burden from the programmer.)

You could have a function which took a delegate and called it (which could probably be done with void* if the function has to be an extern(C) function), or if the function didn't need to be pure, then you could have a function which called a module-level variable which the delegate had been assigned to. But you can't convert a delegate to a function without passing the delegate to the function one way or another, because the function doesn't have a context (not being a delegate). - Jonathan M Davis

http://en.wikipedia.org/wiki/Thunk_(functional_programming) http://catb.org/jargon/html/T/thunk.html You may be able to do something like this with libffi. (See https://github.com/lycus/libffi-d for a D binding.) -- - Alex
Apr 25 2012
prev sibling parent =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com> writes:
On 26-04-2012 00:06, David Nadlinger wrote:
 On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis wrote:
 But you can't convert a delegate to a function without passing the
 delegate to
 the function one way or another, because the function doesn't have a
 context
 (not being a delegate).

That's the whole point of the question - you _can_ pull this off if you create a small code fragment somewhere in memory which calls the delegate with that specific context argument, and then set your function pointer to its address (similar to thunks in most vtable implementations or SEH, if you are familiar with these). Now, the only problem is that executing data is not quite easy, mostly because of security mechanisms (W^X, NX and friends). As for the original question, I didn't need the feature so far (many C libraries provide a void* bit of data to use at the client's discretion), but if you need inspiration on how to implement it, I'd look at projects like Mono and FFI libraries which likely need to provide similar functionality. Would probably be a good addition to Phobos. David

libffi can do it. There's a reason I wanted std.ffi. ;) (Unfortunately interest in such a module seemed low earlier.) -- - Alex
Apr 25 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis 
wrote:
 But you can't convert a delegate to a function without passing 
 the delegate to
 the function one way or another, because the function doesn't 
 have a context
 (not being a delegate).

That's the whole point of the question - you _can_ pull this off if you create a small code fragment somewhere in memory which calls the delegate with that specific context argument, and then set your function pointer to its address (similar to thunks in most vtable implementations or SEH, if you are familiar with these). Now, the only problem is that executing data is not quite easy, mostly because of security mechanisms (W^X, NX and friends). As for the original question, I didn't need the feature so far (many C libraries provide a void* bit of data to use at the client's discretion), but if you need inspiration on how to implement it, I'd look at projects like Mono and FFI libraries which likely need to provide similar functionality. Would probably be a good addition to Phobos. David
Apr 25 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 22:25:10 UTC, Alex Rønne Petersen 
wrote:
 libffi can do it. There's a reason I wanted std.ffi. ;) 
 (Unfortunately interest in such a module seemed low earlier.)

Ah, you beat me to it - didn't see your answer while I was at the reply screen. ;) David
Apr 25 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 26, 2012 at 12:06:54AM +0200, David Nadlinger wrote:
 On Wednesday, 25 April 2012 at 21:27:30 UTC, Jonathan M Davis wrote:
But you can't convert a delegate to a function without passing the
delegate to the function one way or another, because the function
doesn't have a context (not being a delegate).

That's the whole point of the question - you _can_ pull this off if you create a small code fragment somewhere in memory which calls the delegate with that specific context argument, and then set your function pointer to its address (similar to thunks in most vtable implementations or SEH, if you are familiar with these). Now, the only problem is that executing data is not quite easy, mostly because of security mechanisms (W^X, NX and friends).

This is like GCC's implementation of nested function pointers using trampolines: http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html These nested function pointers have access to their containing lexical scope, even though they are just regular, non-fat pointers. Something similar can be used here, though for full-fledged D delegates the trampoline would need to be in executable heap memory. I believe nowadays heap memory is non-executable due to security concerns with heap overflow exploits, right? So this may require special treatment. T -- PNP = Plug 'N' Pray
Apr 25 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 25 April 2012 at 22:31:59 UTC, H. S. Teoh wrote:
 I believe
 nowadays heap memory is non-executable due to security concerns 
 with
 heap overflow exploits, right? So this may require special 
 treatment.

Yes; it usually boils down to allocating a dedicated page and then setting the executable flag after you are done with setting up your code (e.g. using VirtualAlloc/VirtualProtect on Windows). David
Apr 25 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 26, 2012 at 12:47:42AM +0200, David Nadlinger wrote:
 On Wednesday, 25 April 2012 at 22:31:59 UTC, H. S. Teoh wrote:
I believe nowadays heap memory is non-executable due to security
concerns with heap overflow exploits, right? So this may require
special treatment.

Yes; it usually boils down to allocating a dedicated page and then setting the executable flag after you are done with setting up your code (e.g. using VirtualAlloc/VirtualProtect on Windows).

Does turning on the executable flag also make it read-only? If so, we'd need to allocate a page per wrapper function, which seems a bit excessive. Or is it possible to turn it off temporarily, add more code, and turn it back on? (Seems like that should be prohibited, since otherwise it would just be too easy to exploit, and would defeat the purpose of protecting it in the first place.) T -- Arise, you prisoners of Windows Arise, you slaves of Redmond, Wash, The day and hour soon are coming When all the IT folks say "Gosh!" It isn't from a clever lawsuit That Windowsland will finally fall, But thousands writing open source code Like mice who nibble through a wall. -- The Linux-nationale by Greg Baker
Apr 25 2012
prev sibling next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Mehrdad" <wfunction hotmail.com> wrote in message 
news:jn9ops$13bs$1 digitalmars.com...
 It would be nice if there was a way to convert delegates to 
 functions/thunks, because certain annoying tasks (e.g. 'WndProc's in 
 Windows) would become a heck of a lot easier.

 Is there any way to already do this? If not, how about adding a 
 toFunction() method in std.functional?

Well, there's this: http://pastebin.com/XwEb2cpm But it's pre-3797, and probably doesn't work anywhere except win32.
Apr 26 2012
prev sibling parent David Brown <davidb davidb.org> writes:
On 2012-04-25, H. S. Teoh <hsteoh quickfur.ath.cx> wrote:

 This is like GCC's implementation of nested function pointers using
 trampolines:

 	http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html

 These nested function pointers have access to their containing lexical
 scope, even though they are just regular, non-fat pointers.

 Something similar can be used here, though for full-fledged D delegates
 the trampoline would need to be in executable heap memory. I believe
 nowadays heap memory is non-executable due to security concerns with
 heap overflow exploits, right? So this may require special treatment.

Gcc (at least on Linux) does a few things. The trampoline is on the stack, but the generated code: - Creates a special section .section .note.GNU-stack,"x", progbits in the assembly that indicates to the linker that the resulting elf file needs to allow the stack to be executable. - Depending on the CPU, it may call a libgcc function __clear_cache. x86 is coherent, so this isn't needed, but it is in the general case. David
Apr 26 2012