www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [Win32] Remotely execute functions of a D program

reply "alex" <info alexanderbothe.com> writes:
Hi everyone,

To keeps things short: There shall be a extended debugging 
feature integrated into Mono-D / VisualD later on. As you may see 
on 
http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads
2012/09/2ylpkqg.jpg 
, there already is debugging functionality possible for windows 
programs (when it's arrived a pretty stable status it'll be 
released as a second D addin, that's for sure)

Anyway, we'd like to replace all those 'struct main helper' value 
strings with actual values returned by the object's toString 
function.
So the debug engine shall execute the toString() 
function/overload of an object
1) via a D DLL that has been injected into the program run-time or
2) directly via CreateRemoteThread(), whereas it should be 
possible to allocate some code memory and write binary code right 
into it

I kept experimenting with all the injection, assembler and 
program hacking stuff quite a while, and these are my primary 
perceptions:

- It seems that one cannot inject D DLLs into D programs without 
crashing the actual program (it's always an exception thrown by 
RTLMultiPool::SelectFree), whereas:
     --One may write a main() function instead of the DllMain() 
callback
     and then start a normal WindowMessage-loop in order to
     prevent both DLL and Program unload/crash - but that's not 
really it, because it's just caught in a loop, and nothing less.
     --It's possible to call LoadLibrary with the DLLs file path 
inside the DllMain() to hook into the program without letting it 
crash - but then it seems impossible to access the dll from the 
outside (probably via named pipes, then)
     --It doesn't seem to make sense to load in a C dll - because 
from there, it's practically impossible to call D functions.

- I've created an export toString(int pointerToObject) method 
inside the
D program - and it's not possible to invoke it via 
CreateRemoteThread().
So even if I did it to successfully inject a D Dll into the D 
program,
there's no guarantee that it's possible to call that toString 
function even in the D Dll.

extern(C) export string toSt(int p)
{
	return (cast(Object)cast(void*)p).toString();	
}
the 'p' paramter comes from the debugger engine then - so it 
knows the object address.

- Another approach was to put in raw assembler code into the 
program's
virtual memory space and try to execute it from there - so 'just'
put the assembler code (I've built it already lol) into the
program run-time, and execute it somehow. But I definitely do not 
know how to create real working assembler etc.

Or: I've tried Winject yesterday, too, and there it worked to 
load in the DLL just at launching the program - this is something 
which could be realized with the debug engine, I guess.
But then again the question of having execution access to the 
exported functions of the client dll .. named pipes?

Okay, these are my explenanations so far - and I think it would 
be really interesting to have such debugger-debugee communication 
in D.
1) So to anyone who's got richer experiences in programming 
assembler and hacking/'debugging' programs than I - how would you 
do it?
2) And why can't I inject a D DLL right into the program? I tried 
it with a C DLL, it's working with that one, but then I don't 
have access to D-specific functions..
Looking at that, would it make a difference to use dmc to 
build/link a dll as a D/C++ hybrid or something?

Thanks in advance for any ideas, recommendations etc.!

Oh and the debugger addin project: 
https://github.com/aBothe/monodevelop-win32-debugger
Sep 14 2012
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> wrote:

 1) So to anyone who's got richer experiences in programming assembler =

 2) And why can't I inject a D DLL right into the program? I tried it =

to D-specific functions..
 Looking at that, would it make a difference to use dmc to build/link a =

You might want to check the madCodeHook library. It works well, and = there's a version that includes source code. Though now that I check = the site, it doesn't look like you can get the source-included one for = free any more.=
Sep 14 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-14 20:53, alex wrote:

 The primary injection routine and how it works is not the problem - I
 can successfully insert a MSVC++ Dll without any problems into a D
 program. (I've used this https://github.com/jeffora/extemory library btw
 - it's for c# :-))
 Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
 crashes or unloads the dll when returning false in the DllMain()

 So how to insert a D DLL into a D program?

I don't know the status on Windows but in general D dynamic libraries don't properly work. The problems are TLS, exception handing tables and module infos. There might be other problems as well. -- /Jacob Carlborg
Sep 15 2012
parent Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
15.09.2012 16:03, Jacob Carlborg пишет:
 On 2012-09-14 20:53, alex wrote:

 The primary injection routine and how it works is not the problem - I
 can successfully insert a MSVC++ Dll without any problems into a D
 program. (I've used this https://github.com/jeffora/extemory library btw
 - it's for c# :-))
 Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
 crashes or unloads the dll when returning false in the DllMain()

 So how to insert a D DLL into a D program?

I don't know the status on Windows but in general D dynamic libraries don't properly work. The problems are TLS, exception handing tables and module infos. There might be other problems as well.

TLS is the problem on Windows 5.x (XP, Server 2003), not 6.x. D DLL will use incomplete ugly hacks to fix it and will be unable to unload because of hacks incompleteness. By the way I have created a complete hack version to solve TLS problems in XP forever for all programs independently of its source availability that injects itself into running process, fixes everything and injects itself to all it's children, etc. Sorry for such a long story about my just another project nobody needs. Lets return to the toppic. The common practice of DLL-s injection is creating a new thread in remote process and calling LoadLibrary from there but it is incorrect because: * it wastes resources (requires a copy of TLS local variables, calls of module thread local constructors (for D) and all loading DLL-s DllMains to attach new thread (and then to detach of course)) * injecting DLL's DllMain will be called not in main thread but the DLL may expect to be in main one * for D DLL it will just fail because of Digital Mars C runtime library (honestly I'm not sure here but everything looks like it is so) Again, Digital Mars C runtime library is the problem for everything in D language including DLL-s. -- Денис В. Шеломовский Denis V. Shelomovskij
Sep 15 2012
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 14.09.2012 20:53, alex wrote:
 On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:
 On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> wrote:

 1) So to anyone who's got richer experiences in programming assembler
 and hacking/'debugging' programs than I - how would you do it?
 2) And why can't I inject a D DLL right into the program? I tried it
 with a C DLL, it's working with that one, but then I don't have
 access to D-specific functions..
 Looking at that, would it make a difference to use dmc to build/link
 a dll as a D/C++ hybrid or something?

You might want to check the madCodeHook library. It works well, and there's a version that includes source code. Though now that I check the site, it doesn't look like you can get the source-included one for free any more.

The primary injection routine and how it works is not the problem - I can successfully insert a MSVC++ Dll without any problems into a D program. (I've used this https://github.com/jeffora/extemory library btw - it's for c# :-)) Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately crashes or unloads the dll when returning false in the DllMain() So how to insert a D DLL into a D program?

I recently implemented an injection DLL aswell, in D, but without any runtime library: https://github.com/rainers/visuald/blob/master/tools/filemonitor.d The command line to build it is dmd -offilemonitor.dll -defaultlib=user32.lib -L/ENTRY:_DllMain 12 filemonitor.d It also crashed when using the standard DLL framework, my suspicion is that the druntime-initialization isn't properly run for the injected DLL. If you are creating the process suspended, the main thread cannot run, so initialization might have to be done differently. But I haven't investigated any further, I was fine with just using the Windows API. For C/C++ the VS debugger can already call functions in the debuggee as a side effect to watch expressions, so it should work for D aswell. It might be a little complicated to specify the correct symbols, though. To improve debugging experience in VS I think the better approach would be to extend the capabilities of mago, the debug engine explicitely built for dmd generated executables. Unfortunately development of it has stalled.
Sep 15 2012
prev sibling next sibling parent "alex" <info alexanderbothe.com> writes:
On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:
 On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> 
 wrote:

 1) So to anyone who's got richer experiences in programming 
 assembler and hacking/'debugging' programs than I - how would 
 you do it?
 2) And why can't I inject a D DLL right into the program? I 
 tried it with a C DLL, it's working with that one, but then I 
 don't have access to D-specific functions..
 Looking at that, would it make a difference to use dmc to 
 build/link a dll as a D/C++ hybrid or something?

You might want to check the madCodeHook library. It works well, and there's a version that includes source code. Though now that I check the site, it doesn't look like you can get the source-included one for free any more.

The primary injection routine and how it works is not the problem - I can successfully insert a MSVC++ Dll without any problems into a D program. (I've used this https://github.com/jeffora/extemory library btw - it's for c# :-)) Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately crashes or unloads the dll when returning false in the DllMain() So how to insert a D DLL into a D program?
Sep 14 2012
prev sibling next sibling parent "alex" <info alexanderbothe.com> writes:
On Saturday, 15 September 2012 at 13:02:32 UTC, Denis 
Shelomovskij wrote:
....
 Again, Digital Mars C runtime library is the problem for 
 everything in D language including DLL-s.

Lol okay I think I've also seen it. I've tried to build a hybrid dll with mixed C and D code (just compiled with dmc+dmd), and it's just not working, even if it's raw C exclusively.. I'll try an other approach now that is probably way more elegant and doesn't need any LoadLibrary calls: I simply create a code cave in the debuggee and inject some assembler into it. The method I'll be executing then takes a variable address (that has been stored in an other space), makes an object pointer out of it, and calls the virtual toString() overload - whereas the pointer to that function is stored at a fixed offset, fortunately. The returned string struct/pointer whatever will be stored to the variable address then (so I don't have to allocate another variable space), and the debug engine will finally read out the string. That's my theory so far, I hope it'll work somehow :) Rainer I'll try my method first, and if that's not working at all, or if it's working, I'll contact you ;)
Sep 15 2012
prev sibling next sibling parent "alex" <info alexanderbothe.com> writes:
On Saturday, 15 September 2012 at 13:34:02 UTC, alex wrote:
 On Saturday, 15 September 2012 at 13:02:32 UTC, Denis 
 Shelomovskij wrote:
....
 Again, Digital Mars C runtime library is the problem for 
 everything in D language including DLL-s.

Lol okay I think I've also seen it. I've tried to build a hybrid dll with mixed C and D code (just compiled with dmc+dmd), and it's just not working, even if it's raw C exclusively.. I'll try an other approach now that is probably way more elegant and doesn't need any LoadLibrary calls: I simply create a code cave in the debuggee and inject some assembler into it. The method I'll be executing then takes a variable address (that has been stored in an other space), makes an object pointer out of it, and calls the virtual toString() overload - whereas the pointer to that function is stored at a fixed offset, fortunately. The returned string struct/pointer whatever will be stored to the variable address then (so I don't have to allocate another variable space), and the debug engine will finally read out the string. That's my theory so far, I hope it'll work somehow :) Rainer I'll try my method first, and if that's not working at all, or if it's working, I'll contact you ;)

It's absolutely frickin' awesome - it works! I could inject the assembler code, call the object's toString() method, do everything as I've just explained...it's awesome!! FUCK YEAH I did it..now I can go to sleep :D Rainer I'll share it so we both may integrate it into the debuggers then :)
Sep 15 2012
prev sibling next sibling parent "alex" <info alexanderbothe.com> writes:
Ah sorry that I've forgotten to post a link to the final project 
- I was just too excited by the fact that it worked ;)

https://github.com/aBothe/monodevelop-win32-debugger/tree/master/DInject
(The most interesting part is located in Inject.cs)
Sep 16 2012
prev sibling next sibling parent "Chang Long" <changlon gmail.com> writes:
It is a problem cause by snn.lib , you can edit it with a hex 
editor

find :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 53 e8
replace to :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 eb 07



On Friday, 14 September 2012 at 17:34:37 UTC, alex wrote:
 Hi everyone,

 To keeps things short: There shall be a extended debugging 
 feature integrated into Mono-D / VisualD later on. As you may 
 see on 
 http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads
2012/09/2ylpkqg.jpg 
 , there already is debugging functionality possible for windows 
 programs (when it's arrived a pretty stable status it'll be 
 released as a second D addin, that's for sure)

 Anyway, we'd like to replace all those 'struct main helper' 
 value strings with actual values returned by the object's 
 toString function.
 So the debug engine shall execute the toString() 
 function/overload of an object
 1) via a D DLL that has been injected into the program run-time 
 or
 2) directly via CreateRemoteThread(), whereas it should be 
 possible to allocate some code memory and write binary code 
 right into it

 I kept experimenting with all the injection, assembler and 
 program hacking stuff quite a while, and these are my primary 
 perceptions:

 - It seems that one cannot inject D DLLs into D programs 
 without crashing the actual program (it's always an exception 
 thrown by RTLMultiPool::SelectFree), whereas:
     --One may write a main() function instead of the DllMain() 
 callback
     and then start a normal WindowMessage-loop in order to
     prevent both DLL and Program unload/crash - but that's not 
 really it, because it's just caught in a loop, and nothing less.
     --It's possible to call LoadLibrary with the DLLs file path 
 inside the DllMain() to hook into the program without letting 
 it crash - but then it seems impossible to access the dll from 
 the outside (probably via named pipes, then)
     --It doesn't seem to make sense to load in a C dll - 
 because from there, it's practically impossible to call D 
 functions.

 - I've created an export toString(int pointerToObject) method 
 inside the
 D program - and it's not possible to invoke it via 
 CreateRemoteThread().
 So even if I did it to successfully inject a D Dll into the D 
 program,
 there's no guarantee that it's possible to call that toString 
 function even in the D Dll.

 extern(C) export string toSt(int p)
 {
 	return (cast(Object)cast(void*)p).toString();	
 }
 the 'p' paramter comes from the debugger engine then - so it 
 knows the object address.

 - Another approach was to put in raw assembler code into the 
 program's
 virtual memory space and try to execute it from there - so 
 'just'
 put the assembler code (I've built it already lol) into the
 program run-time, and execute it somehow. But I definitely do 
 not know how to create real working assembler etc.

 Or: I've tried Winject yesterday, too, and there it worked to 
 load in the DLL just at launching the program - this is 
 something which could be realized with the debug engine, I 
 guess.
 But then again the question of having execution access to the 
 exported functions of the client dll .. named pipes?

 Okay, these are my explenanations so far - and I think it would 
 be really interesting to have such debugger-debugee 
 communication in D.
 1) So to anyone who's got richer experiences in programming 
 assembler and hacking/'debugging' programs than I - how would 
 you do it?
 2) And why can't I inject a D DLL right into the program? I 
 tried it with a C DLL, it's working with that one, but then I 
 don't have access to D-specific functions..
 Looking at that, would it make a difference to use dmc to 
 build/link a dll as a D/C++ hybrid or something?

 Thanks in advance for any ideas, recommendations etc.!

 Oh and the debugger addin project: 
 https://github.com/aBothe/monodevelop-win32-debugger

Sep 16 2012
prev sibling parent "alex" <info alexanderbothe.com> writes:
On Sunday, 16 September 2012 at 16:13:18 UTC, Chang Long wrote:
 It is a problem cause by snn.lib , you can edit it with a hex 
 editor

 find :
 83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 53 e8
 replace to :
 83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 eb 07

That's the cause why a D DLL crashes a D program when being injected? Wow, thanks! Though I've got a solution I'll try this anyway
Sep 16 2012