www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [Win32] Remotely execute functions of a D program

reply "alex" <info alexanderbothe.com> writes:
Hi everyone,

To keeps things short: There shall be a extended debugging 
feature integrated into Mono-D / VisualD later on. As you may see 
on 
http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads
2012/09/2ylpkqg.jpg 
, there already is debugging functionality possible for windows 
programs (when it's arrived a pretty stable status it'll be 
released as a second D addin, that's for sure)

Anyway, we'd like to replace all those 'struct main helper' value 
strings with actual values returned by the object's toString 
function.
So the debug engine shall execute the toString() 
function/overload of an object
1) via a D DLL that has been injected into the program run-time or
2) directly via CreateRemoteThread(), whereas it should be 
possible to allocate some code memory and write binary code right 
into it

I kept experimenting with all the injection, assembler and 
program hacking stuff quite a while, and these are my primary 
perceptions:

- It seems that one cannot inject D DLLs into D programs without 
crashing the actual program (it's always an exception thrown by 
RTLMultiPool::SelectFree), whereas:
     --One may write a main() function instead of the DllMain() 
callback
     and then start a normal WindowMessage-loop in order to
     prevent both DLL and Program unload/crash - but that's not 
really it, because it's just caught in a loop, and nothing less.
     --It's possible to call LoadLibrary with the DLLs file path 
inside the DllMain() to hook into the program without letting it 
crash - but then it seems impossible to access the dll from the 
outside (probably via named pipes, then)
     --It doesn't seem to make sense to load in a C dll - because 
from there, it's practically impossible to call D functions.

- I've created an export toString(int pointerToObject) method 
inside the
D program - and it's not possible to invoke it via 
CreateRemoteThread().
So even if I did it to successfully inject a D Dll into the D 
program,
there's no guarantee that it's possible to call that toString 
function even in the D Dll.

extern(C) export string toSt(int p)
{
	return (cast(Object)cast(void*)p).toString();	
}
the 'p' paramter comes from the debugger engine then - so it 
knows the object address.

- Another approach was to put in raw assembler code into the 
program's
virtual memory space and try to execute it from there - so 'just'
put the assembler code (I've built it already lol) into the
program run-time, and execute it somehow. But I definitely do not 
know how to create real working assembler etc.

Or: I've tried Winject yesterday, too, and there it worked to 
load in the DLL just at launching the program - this is something 
which could be realized with the debug engine, I guess.
But then again the question of having execution access to the 
exported functions of the client dll .. named pipes?

Okay, these are my explenanations so far - and I think it would 
be really interesting to have such debugger-debugee communication 
in D.
1) So to anyone who's got richer experiences in programming 
assembler and hacking/'debugging' programs than I - how would you 
do it?
2) And why can't I inject a D DLL right into the program? I tried 
it with a C DLL, it's working with that one, but then I don't 
have access to D-specific functions..
Looking at that, would it make a difference to use dmc to 
build/link a dll as a D/C++ hybrid or something?

Thanks in advance for any ideas, recommendations etc.!

Oh and the debugger addin project: 
https://github.com/aBothe/monodevelop-win32-debugger
Sep 14 2012
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> wrote:


 1) So to anyone who's got richer experiences in programming assembler =


and hacking/'debugging' programs than I - how would you do it?

 2) And why can't I inject a D DLL right into the program? I tried it =


with a C DLL, it's working with that one, but then I don't have access =
to D-specific functions..

 Looking at that, would it make a difference to use dmc to build/link a =


dll as a D/C++ hybrid or something?

You might want to check the madCodeHook library.  It works well, and =
there's a version that includes source code.  Though now that I check =
the site, it doesn't look like you can get the source-included one for =
free any more.=
Sep 14 2012
parent reply "alex" <info alexanderbothe.com> writes:
On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:

 On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> 
 wrote:


 1) So to anyone who's got richer experiences in programming 
 assembler and hacking/'debugging' programs than I - how would 
 you do it?
 2) And why can't I inject a D DLL right into the program? I 
 tried it with a C DLL, it's working with that one, but then I 
 don't have access to D-specific functions..
 Looking at that, would it make a difference to use dmc to 
 build/link a dll as a D/C++ hybrid or something?


 You might want to check the madCodeHook library.  It works 
 well, and there's a version that includes source code.  Though 
 now that I check the site, it doesn't look like you can get the 
 source-included one for free any more.


The primary injection routine and how it works is not the problem 
- I can successfully insert a MSVC++ Dll without any problems 
into a D program. (I've used this 
https://github.com/jeffora/extemory
Anyway, on a DLL built with dmc or dmd 1/2, the main program 
immediately crashes or unloads the dll when returning false in 
the DllMain()

So how to insert a D DLL into a D program?
Sep 14 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-14 20:53, alex wrote:


 The primary injection routine and how it works is not the problem - I
 can successfully insert a MSVC++ Dll without any problems into a D
 program. (I've used this https://github.com/jeffora/extemory library btw

 Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
 crashes or unloads the dll when returning false in the DllMain()

 So how to insert a D DLL into a D program?


I don't know the status on Windows but in general D dynamic libraries 
don't properly work. The problems are TLS, exception handing tables and 
module infos. There might be other problems as well.

-- 
/Jacob Carlborg
Sep 15 2012
parent reply Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
15.09.2012 16:03, Jacob Carlborg пишет:

 On 2012-09-14 20:53, alex wrote:


 The primary injection routine and how it works is not the problem - I
 can successfully insert a MSVC++ Dll without any problems into a D
 program. (I've used this https://github.com/jeffora/extemory library btw

 Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
 crashes or unloads the dll when returning false in the DllMain()

 So how to insert a D DLL into a D program?


 I don't know the status on Windows but in general D dynamic libraries
 don't properly work. The problems are TLS, exception handing tables and
 module infos. There might be other problems as well.


TLS is the problem on Windows 5.x (XP, Server 2003), not 6.x. D DLL will 
use incomplete ugly hacks to fix it and will be unable to unload because 
of hacks incompleteness. By the way I have created a complete hack 
version to solve TLS problems in XP forever for all programs 
independently of its source availability that injects itself into 
running process, fixes everything and injects itself to all it's 
children, etc. Sorry for such a long story about my just another project 
nobody needs.

Lets return to the toppic.

The common practice of DLL-s injection is creating a new thread in 
remote process and calling LoadLibrary from there but it is incorrect 
because:
* it wastes resources (requires a copy of TLS local variables, calls of 
module thread local constructors (for D) and all loading DLL-s DllMains 
to attach new thread (and then to detach of course))
* injecting DLL's DllMain will be called not in main thread but the DLL 
may expect to be in main one
* for D DLL it will just fail because of Digital Mars C runtime library 
(honestly I'm not sure here but everything looks like it is so)

Again, Digital Mars C runtime library is the problem for everything in D 
language including DLL-s.

-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Sep 15 2012
parent reply "alex" <info alexanderbothe.com> writes:
On Saturday, 15 September 2012 at 13:02:32 UTC, Denis 
Shelomovskij wrote:

....
 Again, Digital Mars C runtime library is the problem for 
 everything in D language including DLL-s.


Lol okay I think I've also seen it. I've tried to build a hybrid 
dll with mixed C and D code (just compiled with dmc+dmd), and 
it's just not working, even if it's raw C exclusively..


I'll try an other approach now that is probably way more elegant 
and doesn't need any LoadLibrary calls:
I simply create a code cave in the debuggee and inject some 
assembler into it.
The method I'll be executing then takes a variable address (that 
has been stored in an other space), makes an object pointer out 
of it, and calls the virtual toString() overload - whereas the 
pointer to that function is stored at a fixed offset, 
fortunately. The returned string struct/pointer whatever will be 
stored to the variable address then (so I don't have to allocate 
another variable space), and the debug engine will finally read 
out the string.
That's my theory so far, I hope it'll work somehow :)

 Rainer I'll try my method first, and if that's not working at 
all, or if it's working, I'll contact you ;)
Sep 15 2012
parent reply "alex" <info alexanderbothe.com> writes:
On Saturday, 15 September 2012 at 13:34:02 UTC, alex wrote:

 On Saturday, 15 September 2012 at 13:02:32 UTC, Denis 
 Shelomovskij wrote:

....
 Again, Digital Mars C runtime library is the problem for 
 everything in D language including DLL-s.


 Lol okay I think I've also seen it. I've tried to build a 
 hybrid dll with mixed C and D code (just compiled with 
 dmc+dmd), and it's just not working, even if it's raw C 
 exclusively..


 I'll try an other approach now that is probably way more 
 elegant and doesn't need any LoadLibrary calls:
 I simply create a code cave in the debuggee and inject some 
 assembler into it.
 The method I'll be executing then takes a variable address 
 (that has been stored in an other space), makes an object 
 pointer out of it, and calls the virtual toString() overload - 
 whereas the pointer to that function is stored at a fixed 
 offset, fortunately. The returned string struct/pointer 
 whatever will be stored to the variable address then (so I 
 don't have to allocate another variable space), and the debug 
 engine will finally read out the string.
 That's my theory so far, I hope it'll work somehow :)

  Rainer I'll try my method first, and if that's not working at 
 all, or if it's working, I'll contact you ;)


It's absolutely frickin' awesome - it works! I could inject the 
assembler code, call the object's toString() method, do 
everything as I've just explained...it's awesome!! FUCK YEAH I 
did it..now I can go to sleep :D

 Rainer I'll share it so we both may integrate it into the 
debuggers then :)
Sep 15 2012
parent "alex" <info alexanderbothe.com> writes:
Ah sorry that I've forgotten to post a link to the final project 
- I was just too excited by the fact that it worked ;)

https://github.com/aBothe/monodevelop-win32-debugger/tree/master/DInject
(The most interesting part is located in Inject.cs)
Sep 16 2012
prev sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 14.09.2012 20:53, alex wrote:

 On Friday, 14 September 2012 at 18:35:53 UTC, Sean Kelly wrote:

 On Sep 14, 2012, at 10:34 AM, alex <info alexanderbothe.com> wrote:


 1) So to anyone who's got richer experiences in programming assembler
 and hacking/'debugging' programs than I - how would you do it?
 2) And why can't I inject a D DLL right into the program? I tried it
 with a C DLL, it's working with that one, but then I don't have
 access to D-specific functions..
 Looking at that, would it make a difference to use dmc to build/link
 a dll as a D/C++ hybrid or something?


 You might want to check the madCodeHook library.  It works well, and
 there's a version that includes source code.  Though now that I check
 the site, it doesn't look like you can get the source-included one for
 free any more.


 The primary injection routine and how it works is not the problem - I
 can successfully insert a MSVC++ Dll without any problems into a D
 program. (I've used this https://github.com/jeffora/extemory library btw

 Anyway, on a DLL built with dmc or dmd 1/2, the main program immediately
 crashes or unloads the dll when returning false in the DllMain()

 So how to insert a D DLL into a D program?


I recently implemented an injection DLL aswell, in D, but without any 
runtime library: 
https://github.com/rainers/visuald/blob/master/tools/filemonitor.d

The command line to build it is
dmd -offilemonitor.dll -defaultlib=user32.lib -L/ENTRY:_DllMain 12 
filemonitor.d

It also crashed when using the standard DLL framework, my suspicion is 
that the druntime-initialization isn't properly run for the injected 
DLL. If you are creating the process suspended, the main thread cannot 
run, so initialization might have to be done differently. But I haven't 
investigated any further, I was fine with just using the Windows API.

For C/C++ the VS debugger can already call functions in the debuggee as 
a side effect to watch expressions, so it should work for D aswell. It 
might be a little complicated to specify the correct symbols, though.

To improve debugging experience in VS I think the better approach would 
be to extend the capabilities of mago, the debug engine explicitely 
built for dmd generated executables. Unfortunately development of it has 
stalled.
Sep 15 2012
prev sibling parent reply "Chang Long" <changlon gmail.com> writes:
It is a problem cause by snn.lib , you can edit it with a hex 
editor

find :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 53 e8
replace to :
83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 eb 07



On Friday, 14 September 2012 at 17:34:37 UTC, alex wrote:

 Hi everyone,

 To keeps things short: There shall be a extended debugging 
 feature integrated into Mono-D / VisualD later on. As you may 
 see on 
 http://mono-d.alexanderbothe.com/wordpress/wp-content/uploads
2012/09/2ylpkqg.jpg 
 , there already is debugging functionality possible for windows 
 programs (when it's arrived a pretty stable status it'll be 
 released as a second D addin, that's for sure)

 Anyway, we'd like to replace all those 'struct main helper' 
 value strings with actual values returned by the object's 
 toString function.
 So the debug engine shall execute the toString() 
 function/overload of an object
 1) via a D DLL that has been injected into the program run-time 
 or
 2) directly via CreateRemoteThread(), whereas it should be 
 possible to allocate some code memory and write binary code 
 right into it

 I kept experimenting with all the injection, assembler and 
 program hacking stuff quite a while, and these are my primary 
 perceptions:

 - It seems that one cannot inject D DLLs into D programs 
 without crashing the actual program (it's always an exception 
 thrown by RTLMultiPool::SelectFree), whereas:
     --One may write a main() function instead of the DllMain() 
 callback
     and then start a normal WindowMessage-loop in order to
     prevent both DLL and Program unload/crash - but that's not 
 really it, because it's just caught in a loop, and nothing less.
     --It's possible to call LoadLibrary with the DLLs file path 
 inside the DllMain() to hook into the program without letting 
 it crash - but then it seems impossible to access the dll from 
 the outside (probably via named pipes, then)
     --It doesn't seem to make sense to load in a C dll - 
 because from there, it's practically impossible to call D 
 functions.

 - I've created an export toString(int pointerToObject) method 
 inside the
 D program - and it's not possible to invoke it via 
 CreateRemoteThread().
 So even if I did it to successfully inject a D Dll into the D 
 program,
 there's no guarantee that it's possible to call that toString 
 function even in the D Dll.

 extern(C) export string toSt(int p)
 {
 	return (cast(Object)cast(void*)p).toString();	
 }
 the 'p' paramter comes from the debugger engine then - so it 
 knows the object address.

 - Another approach was to put in raw assembler code into the 
 program's
 virtual memory space and try to execute it from there - so 
 'just'
 put the assembler code (I've built it already lol) into the
 program run-time, and execute it somehow. But I definitely do 
 not know how to create real working assembler etc.

 Or: I've tried Winject yesterday, too, and there it worked to 
 load in the DLL just at launching the program - this is 
 something which could be realized with the debug engine, I 
 guess.
 But then again the question of having execution access to the 
 exported functions of the client dll .. named pipes?

 Okay, these are my explenanations so far - and I think it would 
 be really interesting to have such debugger-debugee 
 communication in D.
 1) So to anyone who's got richer experiences in programming 
 assembler and hacking/'debugging' programs than I - how would 
 you do it?
 2) And why can't I inject a D DLL right into the program? I 
 tried it with a C DLL, it's working with that one, but then I 
 don't have access to D-specific functions..
 Looking at that, would it make a difference to use dmc to 
 build/link a dll as a D/C++ hybrid or something?

 Thanks in advance for any ideas, recommendations etc.!

 Oh and the debugger addin project: 
 https://github.com/aBothe/monodevelop-win32-debugger
Sep 16 2012
parent "alex" <info alexanderbothe.com> writes:
On Sunday, 16 September 2012 at 16:13:18 UTC, Chang Long wrote:

 It is a problem cause by snn.lib , you can edit it with a hex 
 editor

 find :
 83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 53 e8
 replace to :
 83 C4 04 83 7B 1C 00 74 09 FF 73 1C FF 15 00 00 00 00 eb 07


That's the cause why a D DLL crashes a D program when being 
injected? Wow, thanks! Though I've got a solution I'll try this 
anyway
Sep 16 2012