www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Debugging D DLL from C# app with C linkage for native Unity 5 plugin

reply Thalamus <norobots foo.com> writes:
Apologies if this has been discussed before, but I wasn't able to 
find anything similar on the forums or web. I can't seem to 
figure out how to debug a D DLL from a C# EXE. (My actual purpose 
here is to use D to build native plugins for Unity 5, but Unity 
and Mono aren't necessary to repro the problem I'm running into.)

The D DLL and C# are both built as 64-bit. (C# is not set to 
AnyCPU). Using VS 2015 and Visual D, I can drive the D DLL from 
the C# EXE without any problems, but the debugger doesn't load 
the D DLL symbols. The DLL and PDB are in the same folder as the 
C# EXE. Everything behaves the same if I link to the DLL 
statically using P/Invoke or dynamically using LoadLibrary.

Everything from the D DLL is exposed as extern(C), listed in the 
.def, etc. When I drive the DLL from a D EXE, I can break into 
the debugger easily, whether I attach at launch or attach to the 
process when it's already running.

I'm building the DLL using:

dmd <obj files omitted for brevity> dllmain.d dll.def -w -wi -g 
-map -ofLogic.dll <res and lib files omitted for brevity> -m64 
-debug -shared

Anyone know what I should try next? Am I missing something 
simple? :)

thanks!
Thalamus
Mar 29 2016
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 30.03.2016 01:41, Thalamus wrote:
 Apologies if this has been discussed before, but I wasn't able to find
 anything similar on the forums or web. I can't seem to figure out how to
 debug a D DLL from a C# EXE. (My actual purpose here is to use D to
 build native plugins for Unity 5, but Unity and Mono aren't necessary to
 repro the problem I'm running into.)

 The D DLL and C# are both built as 64-bit. (C# is not set to AnyCPU).
 Using VS 2015 and Visual D, I can drive the D DLL from the C# EXE
 without any problems, but the debugger doesn't load the D DLL symbols.
 The DLL and PDB are in the same folder as the C# EXE. Everything behaves
 the same if I link to the DLL statically using P/Invoke or dynamically
 using LoadLibrary.

 Everything from the D DLL is exposed as extern(C), listed in the .def,
 etc. When I drive the DLL from a D EXE, I can break into the debugger
 easily, whether I attach at launch or attach to the process when it's
 already running.

 I'm building the DLL using:

 dmd <obj files omitted for brevity> dllmain.d dll.def -w -wi -g -map
 -ofLogic.dll <res and lib files omitted for brevity> -m64 -debug -shared

 Anyone know what I should try next? Am I missing something simple? :)

 thanks!
 Thalamus
I haven't tried debugging a C# application, but you might have to disable "Just My Code" in the global debugger options. Also make sure to allow native debugging (JIT options). When debugging Visual D (a D DLL), I set VS (devenv.exe, a mixed C++/C# app) as the program to debug in the project debugger options. This ensures a native debugger engine is used and is the simplest way to use the mago debugger. With the "Mixed mode" engine, you can debug both C# and D/C++ in the same session.
Mar 29 2016
prev sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
On Tuesday, 29 March 2016 at 23:41:28 UTC, Thalamus wrote:
 dmd <obj files omitted for brevity> dllmain.d dll.def -w -wi -g 
 -map -ofLogic.dll <res and lib files omitted for brevity> -m64 
 -debug -shared

 Anyone know what I should try next? Am I missing something 
 simple? :)

 thanks!
 Thalamus
You should be using "-gc" instead of "-g" when building 64-bit D programs that should be debugged with visual studio. Otherwise the visual studio debugger might get confused over some of the symbol names. (Because they contain '.')
Mar 30 2016
parent Thalamus <norobots foo.com> writes:
On Wednesday, 30 March 2016 at 07:38:07 UTC, Benjamin Thaut wrote:
 On Tuesday, 29 March 2016 at 23:41:28 UTC, Thalamus wrote:
 dmd <obj files omitted for brevity> dllmain.d dll.def -w -wi 
 -g -map -ofLogic.dll <res and lib files omitted for brevity> 
 -m64 -debug -shared

 Anyone know what I should try next? Am I missing something 
 simple? :)

 thanks!
 Thalamus
You should be using "-gc" instead of "-g" when building 64-bit D programs that should be debugged with visual studio. Otherwise the visual studio debugger might get confused over some of the symbol names. (Because they contain '.')
Thanks Benjamin! I changed over to -gc. I spent another couple of hours on this and finally figured it out. As it turns out, it wasn't necessary to change Just My Code. Enabling Mixed Mode debugging didn't work, but that's what set me down the path where I was able to find the answer. Unity is an odd duck in a lot of ways. They use Mono to provide cross-platform portability, and that decision led them to use their own custom subset of .NET 3.5. Although I had eliminated Unity and Mono from the repro, I had accidentally left that subset on the C# EXE. Because of this, VS uses a different managed code debugger (v3.5, v3.0, v2.0 instead of v4.6, v4.5, v4.0). This version doesn't play nicely with the Native debugger. By default, VS determines the debuggers to use automatically. So in this scenario, if you launch the EXE and then attach, it only loads the managed debugger (without telling you). If you select the old Managed and the Native debuggers in the Attach to Process dialog's Attach to: drop down list, an "Interop debugging is not supported" error pops up when you click Attach. (This is the same error you get if you select "Enable native code debugging" on the C# EXE's Debug property page.) The solution is to select only Native or the older Managed debugger, but never both. That means that you can't hit F5 to launch the EXE in debug mode and then step through native code, which is inconvenient. But for my purposes debugging through Unity, attaching to an already running process is the only scenario I really need to work anyway. The C# layer is a very thin interop and marshaling layer between the C++ (hopefully soon D) core and Unity, so 99% of the time I'll need to debug only the native code anyway. The transition between the two is the only thing that can't be stepped through, and there isn't a whole lot to that. As it turns out, I never saw this with the C++ version of the core logic because there were no C# projects in the solution, so VS automatically chose the Native one without my knowledge. Hope this helps someone else in the future! thanks, Gene
Mar 30 2016