www.digitalmars.com         C & C++   DMDScript  

D.gnu - Can GDC make DLLs?

reply Ben Davis <entheh cantab.net> writes:
Hi,

Has anyone had any success using GDC to make DLLs to be called from C/C++?

The reason I ask is, for me, the following snippet inside dll.d / 
dll_fixTLS() seems to have compiled to a call to abort():

         void** peb;
         asm
         {
             mov EAX,FS:[0x30];
             mov peb, EAX;
         }

and thus dll_process_attach() crashes the process.

It seems like a bug that would affect more people than just me, yet I 
couldn't find any evidence of other people hitting it. Have I got it 
right what's happening, or is something else at work?

If I'm right, then I'm just wondering if anyone has any ideas on whether 
it could be fixed, and how?

Also, I found some discussion about D-style inline asm being problematic 
and worthy of removal, but didn't find any explanation as to what those 
problems were. I'm curious :)

Thanks in advance if anyone can help. This is my first post in this 
group, so I hope it meets the necessary standards :)

Ben :)
Feb 23 2013
parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net> wrote:
 Hi,

 Has anyone had any success using GDC to make DLLs to be called from C/C++?

 The reason I ask is, for me, the following snippet inside dll.d /

dll_fixTLS() seems to have compiled to a call to abort():
         void** peb;
         asm
         {
             mov EAX,FS:[0x30];
             mov peb, EAX;
         }

 and thus dll_process_attach() crashes the process.

 It seems like a bug that would affect more people than just me, yet I

couldn't find any evidence of other people hitting it. Have I got it right what's happening, or is something else at work?
 If I'm right, then I'm just wondering if anyone has any ideas on whether

it could be fixed, and how?
 Also, I found some discussion about D-style inline asm being problematic

and worthy of removal, but didn't find any explanation as to what those problems were. I'm curious :)

Only because shared libraries requires PIC, and quite a few of the IASM routines clobber the pic register. Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 23 2013
parent reply Ben Davis <entheh cantab.net> writes:
On 23/02/2013 18:38, Iain Buclaw wrote:
 On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net
 <mailto:entheh cantab.net>> wrote:
  >
  > Hi,
  >
  > Has anyone had any success using GDC to make DLLs to be called from
 C/C++?
  >
  > The reason I ask is, for me, the following snippet inside dll.d /
 dll_fixTLS() seems to have compiled to a call to abort():
  >
  >         void** peb;
  >         asm
  >         {
  >             mov EAX,FS:[0x30];
  >             mov peb, EAX;
  >         }
  >
  > and thus dll_process_attach() crashes the process.
  >
  > It seems like a bug that would affect more people than just me, yet I
 couldn't find any evidence of other people hitting it. Have I got it
 right what's happening, or is something else at work?
  >
  > If I'm right, then I'm just wondering if anyone has any ideas on
 whether it could be fixed, and how?
  >
  > Also, I found some discussion about D-style inline asm being
 problematic and worthy of removal, but didn't find any explanation as to
 what those problems were. I'm curious :)
  >

 Only because shared libraries requires PIC, and quite a few of the IASM
 routines clobber the pic register.

I see. :) I've managed to build my DLL with a replacement for dll.d, using GCC-style inline asm for the offending block. Now I've run into another problem. This code crashes: static bool tlsCtorRun; static this() { tlsCtorRun = true; } 64841F0F 8B 15 D4 E2 95 64 mov edx,dword ptr ds:[6495E2D4h] 64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch] 64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4] 64841F1E C6 80 1C 50 96 64 01 mov byte ptr _tls_start+4 (6496501Ch)[eax],1 <-- this line crashes 64841F25 5D pop ebp 64841F26 C3 ret It works when built with DMD. The code is functionally identical, and gets identical values (give or take some randomless with exact memory layout), with one exception: the offending line effectively says "byte ptr [eax+10h]" instead. So the constant displacement in the instruction is 10h instead of 6496501Ch. I'm no expert on how DLLs are loaded, but I think I've ruled out any load-time code offset adjustment - because I found the exact sequence of code bytes in the DLL file on disk, including that number 6496501Ch and all the surrounding code mnemonics. Is this a compiler TLS codegen bug?
Feb 23 2013
parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
On 23 Feb 2013 22:00, "Ben Davis" <entheh cantab.net> wrote:
 On 23/02/2013 18:38, Iain Buclaw wrote:
 On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net
 <mailto:entheh cantab.net>> wrote:
  >
  > Hi,
  >
  > Has anyone had any success using GDC to make DLLs to be called from
 C/C++?
  >
  > The reason I ask is, for me, the following snippet inside dll.d /
 dll_fixTLS() seems to have compiled to a call to abort():
  >
  >         void** peb;
  >         asm
  >         {
  >             mov EAX,FS:[0x30];
  >             mov peb, EAX;
  >         }
  >
  > and thus dll_process_attach() crashes the process.
  >
  > It seems like a bug that would affect more people than just me, yet I
 couldn't find any evidence of other people hitting it. Have I got it
 right what's happening, or is something else at work?
  >
  > If I'm right, then I'm just wondering if anyone has any ideas on
 whether it could be fixed, and how?
  >
  > Also, I found some discussion about D-style inline asm being
 problematic and worthy of removal, but didn't find any explanation as to
 what those problems were. I'm curious :)
  >

 Only because shared libraries requires PIC, and quite a few of the IASM
 routines clobber the pic register.

I see. :) I've managed to build my DLL with a replacement for dll.d, using

GCC-style inline asm for the offending block. Now I've run into another problem. This code crashes:
     static bool tlsCtorRun;
     static this() { tlsCtorRun = true; }
 64841F0F 8B 15 D4 E2 95 64    mov         edx,dword ptr ds:[6495E2D4h]
 64841F15 64 A1 2C 00 00 00    mov         eax,dword ptr fs:[0000002Ch]
 64841F1B 8B 04 90             mov         eax,dword ptr [eax+edx*4]
 64841F1E C6 80 1C 50 96 64 01 mov         byte ptr _tls_start+4

(6496501Ch)[eax],1 <-- this line crashes
 64841F25 5D                   pop         ebp
 64841F26 C3                   ret

 It works when built with DMD. The code is functionally identical, and

gets identical values (give or take some randomless with exact memory layout), with one exception: the offending line effectively says "byte ptr [eax+10h]" instead. So the constant displacement in the instruction is 10h instead of 6496501Ch.
 I'm no expert on how DLLs are loaded, but I think I've ruled out any

load-time code offset adjustment - because I found the exact sequence of code bytes in the DLL file on disk, including that number 6496501Ch and all the surrounding code mnemonics.
 Is this a compiler TLS codegen bug?

GDC TLS is differrent to whatever DMD uses, so assembly code that works for DMD may not necessarily be correct for GDC. Regards ---- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 24 2013
parent reply Ben Davis <entheh cantab.net> writes:
On 24/02/2013 09:38, Iain Buclaw wrote:
 On 23 Feb 2013 22:00, "Ben Davis" <entheh cantab.net
 <mailto:entheh cantab.net>> wrote:
  > I've managed to build my DLL with a replacement for dll.d, using
 GCC-style inline asm for the offending block. Now I've run into another
 problem. This code crashes:
  >
  >     static bool tlsCtorRun;
  >     static this() { tlsCtorRun = true; }
  > 64841F0F 8B 15 D4 E2 95 64    mov         edx,dword ptr ds:[6495E2D4h]
  > 64841F15 64 A1 2C 00 00 00    mov         eax,dword ptr fs:[0000002Ch]
  > 64841F1B 8B 04 90             mov         eax,dword ptr [eax+edx*4]
  > 64841F1E C6 80 1C 50 96 64 01 mov         byte ptr _tls_start+4
 (6496501Ch)[eax],1     <-- this line crashes
  > 64841F25 5D                   pop         ebp
  > 64841F26 C3                   ret
  >
  > It works when built with DMD. The code is functionally identical, and
 gets identical values (give or take some randomless with exact memory
 layout), with one exception: the offending line effectively says "byte
 ptr [eax+10h]" instead. So the constant displacement in the instruction
 is 10h instead of 6496501Ch.
  >
  > I'm no expert on how DLLs are loaded, but I think I've ruled out any
 load-time code offset adjustment - because I found the exact sequence of
 code bytes in the DLL file on disk, including that number 6496501Ch and
 all the surrounding code mnemonics.
  >
  > Is this a compiler TLS codegen bug?

 GDC TLS is differrent to whatever DMD uses, so assembly code that works
 for DMD may not necessarily be correct for GDC.

That may be true, but what GDC is generating is wrong nevertheless. Otherwise it wouldn't crash! There should be enough evidence here for someone to investigate the bug if they want to. I can also report that the bug only exists in the latest version: gcc-4.6.1-tdm64-1-gdc-20130108-D2.060.7z - buggy gcc-4.6.1-tdm64-1-gdc-20121117-D2.060.7z - works And I suspect it only affects 32-bit code. With the older build, the codegen looks like this: static bool tlsCtorRun; static this() { tlsCtorRun = true; } 64841F0F 8B 15 D4 C2 94 64 mov edx,dword ptr ds:[6494C2D4h] 64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch] 64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4] 64841F1E C6 80 04 00 00 00 01 mov byte ptr [eax+4],1 64841F25 5D pop ebp 64841F26 C3 ret And it doesn't crash. My DLL needs to be 32-bit - but while I had the broken version installed, I also tried a simple hello world exe (Visual D's default) in both 32-bit and 64-bit. The 32-bit one crashed somewhere inside the GC thread, and the 64-bit one worked. When I switched to the non-broken version, they both worked. I'm hitting more problems getting my DLL to start up, but I'll post about those separately later if necessary. Hope that helps :)
Feb 24 2013
parent reply Ben Davis <entheh cantab.net> writes:
On 24/02/2013 15:13, Ben Davis wrote:
 I'm hitting more problems getting my DLL to start up, but I'll post
 about those separately later if necessary.

That's weird. Whatever further problem I thought I was hitting, I'm not hitting it any more. I'll put it down to user error on my part. :) Yay! GDC built my DLL and it works! I'll post here a summary of what I had to do, in case it helps anyone else who wants to build a DLL with GDC. Also in case you want to fix the bugs I'm posting a workaround for. I get the feeling you guys are already too busy, but it's here if you want it. :) So, to summarise - in order for 32-bit DLLs to work, you need to: - Not use the 20130108 release because of the 32-bit TLS bug; - Replace dll.d and threadaux.d (both in core/sys/windows, containing D-style asm) with versions that contain GCC-style asm instead (my versions below). Because the user's DllMain is responsible for calling down into these, it's easy to declare one's own copies with the fixes applied, and call those instead. I put mine in a subdirectory (does D call those packages?) called 'gdcdll', and then imported them instead of the Windows ones. (They're Boost-licensed so it looks as if this is OK no matter what end-user licence your own code has. I'll put my changes under Boost too; why not? So you guys can use them.) Here are all the specific changes I made for MinGW, along with the old (DMD) versions. You'll find the 'else' parts already in the source. In dll.d somewhere near the top - I think this change already exists somewhere but the only place I found it was in a .di file somewhere: version (MinGW) { extern __gshared void* _tls_start; extern __gshared void* _tls_end; extern __gshared void* __xl_a; alias _tls_start _tlsstart; alias _tls_end _tlsend; alias __xl_a _tls_callbacks_a; } else { extern __gshared void* _tlsstart; extern __gshared void* _tlsend; extern __gshared void* _tls_callbacks_a; } extern __gshared int _tls_index; //This is unchanged Further down in dll.d: void** peb; version (MinGW) { asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x30]\n" ".att_syntax noprefix\n" :"=a"(peb); } } else { asm { mov EAX,FS:[0x30]; mov peb, EAX; } } Somewhere in the middle of threadaux.d: version(MinGW) { void** teb; asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x18]\n" "ret\n" ".att_syntax noprefix\n" :"=a"(teb); } return teb; } else version(Win32) { asm { naked; mov EAX,FS:[0x18]; ret; } } else version(Win64) { snip }
Feb 24 2013
parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
On Feb 24, 2013 3:50 PM, "Ben Davis" <entheh cantab.net> wrote:
 On 24/02/2013 15:13, Ben Davis wrote:
 I'm hitting more problems getting my DLL to start up, but I'll post
 about those separately later if necessary.

That's weird. Whatever further problem I thought I was hitting, I'm not

hitting it any more. I'll put it down to user error on my part. :)
 Yay! GDC built my DLL and it works!

 I'll post here a summary of what I had to do, in case it helps anyone

else who wants to build a DLL with GDC. Also in case you want to fix the bugs I'm posting a workaround for. I get the feeling you guys are already too busy, but it's here if you want it. :)

I leave monger to Daniel, and don't really affiliate any modifications to gcc-proper any part if gdc. However that's not to say that I don't think his work is invaluable. :)
 So, to summarise - in order for 32-bit DLLs to work, you need to:

 - Not use the 20130108 release because of the 32-bit TLS bug;

 - Replace dll.d and threadaux.d (both in core/sys/windows, containing

D-style asm) with versions that contain GCC-style asm instead (my versions below). Because the user's DllMain is responsible for calling down into these, it's easy to declare one's own copies with the fixes applied, and call those instead. I put mine in a subdirectory (does D call those packages?) called 'gdcdll', and then imported them instead of the Windows ones. (They're Boost-licensed so it looks as if this is OK no matter what end-user licence your own code has. I'll put my changes under Boost too; why not? So you guys can use them.)
 Here are all the specific changes I made for MinGW, along with the old

(DMD) versions. You'll find the 'else' parts already in the source.
 In dll.d somewhere near the top - I think this change already exists

somewhere but the only place I found it was in a .di file somewhere:
 version (MinGW)
 {
         extern __gshared void* _tls_start;
         extern __gshared void* _tls_end;
         extern __gshared void* __xl_a;

         alias _tls_start _tlsstart;
         alias _tls_end _tlsend;
         alias __xl_a _tls_callbacks_a;
 }
 else
 {
         extern __gshared void* _tlsstart;
         extern __gshared void* _tlsend;
         extern __gshared void* _tls_callbacks_a;
 }
 extern __gshared int   _tls_index;      //This is unchanged


 Further down in dll.d:

 void** peb;
 version (MinGW) {
         asm
         {
                 ".intel_syntax noprefix\n"
                 "mov EAX,FS:[0x30]\n"
                 ".att_syntax noprefix\n"
                 :"=a"(peb);
         }
 }
 else {

         asm
         {
                 mov EAX,FS:[0x30];
                 mov peb, EAX;
         }
 }


 Somewhere in the middle of threadaux.d:

 version(MinGW)
 {
         void** teb;
         asm
         {
                 ".intel_syntax noprefix\n"
                 "mov EAX,FS:[0x18]\n"
                 "ret\n"
                 ".att_syntax noprefix\n"
                 :"=a"(teb);
         }

I think it's horrible that you use .intel_syntax. \n shouldn't be required either if you just have it as one string. ;) Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 24 2013
parent reply Ben Davis <entheh cantab.net> writes:
On 25/02/2013 00:53, Iain Buclaw wrote:
 I leave monger to Daniel, and don't really affiliate any modifications
 to gcc-proper any part if gdc.   However that's not to say that I don't
 think his work is invaluable. :)

Can't say I know what monger is, but OK :)
 I think it's horrible that you use .intel_syntax. \n shouldn't be
 required either if you just have it as one string.  ;)

Hey, I just wanted to get it working as quickly as possible - Googling how to use Intel syntax with GCC's inline asm was faster than trying to learn the absolute %hideous &satan-$born that(%is,$abomination,%syntax),%at&t;. ;) About the string though, do you mean a "multiline string like this"? I guess I've been using Java too long, where those aren't allowed. Good to be reminded that's possible. :) As it happens, I ran into some of the more general problems with GDC which I'm sure people are actively working on - e.g. missing stack traces, failure to generate catchable throwables in response to null pointer access violations and the like - and also, my DLL stopped working when I turned debug info off (I didn't investigate why beyond checking that the export table contained the required entries, which it did) - so after a whole weekend of it, I decided to cut my losses and shell out for the DMC source so I could fix the bug that led me to GDC in the first place. (I also noticed that the GDC build was over 6 MB and the DMD build was less than 1 MB.) I do hope to take advantage of GCC's alleged faster code one day, but my current project is hardly going to tax any CPUs, so I'll go with more stuff working for now. But definitely keep up the good work :)
Feb 24 2013
parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On Feb 25, 2013 3:00 AM, "Ben Davis" <entheh cantab.net> wrote:
 On 25/02/2013 00:53, Iain Buclaw wrote:
 I leave monger to Daniel, and don't really affiliate any modifications
 to gcc-proper any part if gdc.   However that's not to say that I don't
 think his work is invaluable. :)

Can't say I know what monger is, but OK :)

Autocomplete fail on tablet. MingGW port.
 I think it's horrible that you use .intel_syntax. \n shouldn't be
 required either if you just have it as one string.  ;)

Hey, I just wanted to get it working as quickly as possible - Googling

how to use Intel syntax with GCC's inline asm was faster than trying to learn the absolute %hideous &satan-$born that(%is,$abomination,%syntax),%at&t;. ;)
 About the string though, do you mean a "multiline
 string like
 this"? I guess I've been using Java too long, where those aren't allowed.

Good to be reminded that's possible. :)

Yes.
 As it happens, I ran into some of the more general problems with GDC

which I'm sure people are actively working on - e.g. missing stack traces, failure to generate catchable throwables in response to null pointer access violations and the like - and also, my DLL stopped working when I turned debug info off (I didn't investigate why beyond checking that the export table contained the required entries, which it did) - so after a whole weekend of it, I decided to cut my losses and shell out for the DMC source so I could fix the bug that led me to GDC in the first place. (I also noticed that the GDC build was over 6 MB and the DMD build was less than 1 MB.)
 I do hope to take advantage of GCC's alleged faster code one day, but my

current project is hardly going to tax any CPUs, so I'll go with more stuff working for now.
 But definitely keep up the good work :)

Feb 24 2013