www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Exported Classes and DLLs. Working Exceptions from Dlls

reply Venix <venix cbpu.com> writes:
After a little bit of time and effort spent working with DLLs, D, 
exceptions, and classes.  I believe I understand the basics to the 
problems that exist.

Exceptions crash.
Annoying. One solution was to wrap all dll bodies with try/catch, which 
would be going back to the C method. So i'm going to present a solution 
that works.

The problem here is that Exceptions are static. Meaning both your DLL 
and program get a copy. So if your using the posted method of syncing 
your garbage collection, this is trivial to do. Just add a void*[] 
argument to you Load function. Then when you call it. Call
   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
Now within Load. You have to copy the supplied vtbl over the internal 
Exception vtbl.
     for (int i=0; i<vtbl.length;++i)
         Exception.classinfo.vtbl[i] = vtbl[i];

Now exceptions should work like a charm

http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)


Minor issues(Can be worked around with a simple D parser):
1: new cannot be used on an imported class.
2: Classes/Interfaces cannot be inherited from a DLL

Both those stem from the class name not be exported. Quoting D's specs. 
"A class can be exported, which means its name and all its non-private 
members are exposed externally to the DLL or EXE." which i have found 
not to be true with the current compiler.

Solution:
I compiled the D front end. Modified func.c 
FunctionDeclaration::isExport to return true if the class is exported. 
Also i added isExport to AggregateDeclaration.  This allowed me to 
export all exported symbols. Class/Interfaces/etc need to be doctored 
with _Class_ _interface, and the _ from functions need to be removed.

That allows me to inherit and theoretically call new on functions. Then 
I run into access violations.

Major Issues(I didn't find a solution):
1: Inheritance doesn't work. Access violation on method calls.
2: new causes access violation.

After a few hours of fiddling. I came to the following conclusions.
If I make a static library. It imports 2 more symbols for every class 
_init_ & _vtbl_. So I thought if I exported em it may work.
Wrong. The linker/compiler never uses those for exported classes.

In fact I believe all access violations, with dlls, are caused by that 
issue.  Upon inspecting the imported class within its dll. ClassInfo 
resolves into something useful. Outside the class, ClassInfo is garbage, 
just try to print baseclass.classinfo.name.  I have tried to fix this 
improper Classinfo structure by copying over the real with the bad, but 
i got nothing but access violations. So I'll assume that this ClassInfo 
structure is not initialized and I'm just writing to bad memory.

I don't know how that structure is made. I would guess maybe by _init_ 
or _vtbl_. But I hope this information may help fix the issues.

I'm going to play around with the GDC compiler in hopes that I can get a 
working solution with that. I believe once DLL classes are supported 
phobos/ares can be can be compiled as a dll and then the exception issue 
will resolve itself. Along with the garbage collection. Allowing Dlls to 
be used without a load function.
Nov 17 2005
next sibling parent pragma <pragma_member pathlink.com> writes:
If it helps, I documented these and a few other drawbacks to DLL's in D a while
back:

http://www.prowiki.org/wiki4d/wiki.cgi?BestPractices/DLL

It prompted me to start this:

http://trac.dsource.org/projects/ddl

Which is an outright replacement to operating with DDLs by borrowing a lot from
other (existing) models of dynamic module usage.  While the project is still
alpha/beta, it should be pretty clear what's been accomplished so far without
needing to hack the compiler or use a modified frontend.

If you're interested, the current proof-of-concept for dynamic linking is
described here:



- EricAnderton at yahoo


In article <dlievm$1lne$1 digitaldaemon.com>, Venix says...
After a little bit of time and effort spent working with DLLs, D, 
exceptions, and classes.  I believe I understand the basics to the 
problems that exist.

Exceptions crash.
Annoying. One solution was to wrap all dll bodies with try/catch, which 
would be going back to the C method. So i'm going to present a solution 
that works.

The problem here is that Exceptions are static. Meaning both your DLL 
and program get a copy. So if your using the posted method of syncing 
your garbage collection, this is trivial to do. Just add a void*[] 
argument to you Load function. Then when you call it. Call
   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
Now within Load. You have to copy the supplied vtbl over the internal 
Exception vtbl.
     for (int i=0; i<vtbl.length;++i)
         Exception.classinfo.vtbl[i] = vtbl[i];

Now exceptions should work like a charm

http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)


Minor issues(Can be worked around with a simple D parser):
1: new cannot be used on an imported class.
2: Classes/Interfaces cannot be inherited from a DLL

Both those stem from the class name not be exported. Quoting D's specs. 
"A class can be exported, which means its name and all its non-private 
members are exposed externally to the DLL or EXE." which i have found 
not to be true with the current compiler.

Solution:
I compiled the D front end. Modified func.c 
FunctionDeclaration::isExport to return true if the class is exported. 
Also i added isExport to AggregateDeclaration.  This allowed me to 
export all exported symbols. Class/Interfaces/etc need to be doctored 
with _Class_ _interface, and the _ from functions need to be removed.

That allows me to inherit and theoretically call new on functions. Then 
I run into access violations.

Major Issues(I didn't find a solution):
1: Inheritance doesn't work. Access violation on method calls.
2: new causes access violation.

After a few hours of fiddling. I came to the following conclusions.
If I make a static library. It imports 2 more symbols for every class 
_init_ & _vtbl_. So I thought if I exported em it may work.
Wrong. The linker/compiler never uses those for exported classes.

In fact I believe all access violations, with dlls, are caused by that 
issue.  Upon inspecting the imported class within its dll. ClassInfo 
resolves into something useful. Outside the class, ClassInfo is garbage, 
just try to print baseclass.classinfo.name.  I have tried to fix this 
improper Classinfo structure by copying over the real with the bad, but 
i got nothing but access violations. So I'll assume that this ClassInfo 
structure is not initialized and I'm just writing to bad memory.

I don't know how that structure is made. I would guess maybe by _init_ 
or _vtbl_. But I hope this information may help fix the issues.

I'm going to play around with the GDC compiler in hopes that I can get a 
working solution with that. I believe once DLL classes are supported 
phobos/ares can be can be compiled as a dll and then the exception issue 
will resolve itself. Along with the garbage collection. Allowing Dlls to 
be used without a load function.
Nov 17 2005
prev sibling parent reply Venix <venix cbpu.com> writes:
After playing around with GDC.  I finally got it compiled with mingw and 
msys.  With a few simple modifications GDC can have full support for 
windows DLLs.  Inheritance and calling.  I haven't tested this 
extensively. My simple test case I had for DMD compiled and ran with 
GDC.  I will be playing with them more in the upcoming weeks, so any 
subtle errors will most likely show themselves.

One rather minor issue, is my exception fix no longer works. As I 
believe GDC makes vtbl's readonly. Once again making phobos a dll would 
fix that.  I'll try the a2dll tool that comes with mingw and see how 
that works.

Venix wrote:
 After a little bit of time and effort spent working with DLLs, D, 
 exceptions, and classes.  I believe I understand the basics to the 
 problems that exist.
 
 Exceptions crash.
 Annoying. One solution was to wrap all dll bodies with try/catch, which 
 would be going back to the C method. So i'm going to present a solution 
 that works.
 
 The problem here is that Exceptions are static. Meaning both your DLL 
 and program get a copy. So if your using the posted method of syncing 
 your garbage collection, this is trivial to do. Just add a void*[] 
 argument to you Load function. Then when you call it. Call
   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
 Now within Load. You have to copy the supplied vtbl over the internal 
 Exception vtbl.
     for (int i=0; i<vtbl.length;++i)
         Exception.classinfo.vtbl[i] = vtbl[i];
 
 Now exceptions should work like a charm
 
 http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)
 
 
 Minor issues(Can be worked around with a simple D parser):
 1: new cannot be used on an imported class.
 2: Classes/Interfaces cannot be inherited from a DLL
 
 Both those stem from the class name not be exported. Quoting D's specs. 
 "A class can be exported, which means its name and all its non-private 
 members are exposed externally to the DLL or EXE." which i have found 
 not to be true with the current compiler.
 
 Solution:
 I compiled the D front end. Modified func.c 
 FunctionDeclaration::isExport to return true if the class is exported. 
 Also i added isExport to AggregateDeclaration.  This allowed me to 
 export all exported symbols. Class/Interfaces/etc need to be doctored 
 with _Class_ _interface, and the _ from functions need to be removed.
 
 That allows me to inherit and theoretically call new on functions. Then 
 I run into access violations.
 
 Major Issues(I didn't find a solution):
 1: Inheritance doesn't work. Access violation on method calls.
 2: new causes access violation.
 
 After a few hours of fiddling. I came to the following conclusions.
 If I make a static library. It imports 2 more symbols for every class 
 _init_ & _vtbl_. So I thought if I exported em it may work.
 Wrong. The linker/compiler never uses those for exported classes.
 
 In fact I believe all access violations, with dlls, are caused by that 
 issue.  Upon inspecting the imported class within its dll. ClassInfo 
 resolves into something useful. Outside the class, ClassInfo is garbage, 
 just try to print baseclass.classinfo.name.  I have tried to fix this 
 improper Classinfo structure by copying over the real with the bad, but 
 i got nothing but access violations. So I'll assume that this ClassInfo 
 structure is not initialized and I'm just writing to bad memory.
 
 I don't know how that structure is made. I would guess maybe by _init_ 
 or _vtbl_. But I hope this information may help fix the issues.
 
 I'm going to play around with the GDC compiler in hopes that I can get a 
 working solution with that. I believe once DLL classes are supported 
 phobos/ares can be can be compiled as a dll and then the exception issue 
 will resolve itself. Along with the garbage collection. Allowing Dlls to 
 be used without a load function.
Nov 18 2005
parent reply Venix <venix cbpu.com> writes:
Phobos DLL
a2dll failed. So i did the old fashioned trick of exploding the library 
and remaking it a dll. Tha gave me link errors. So I then decided to 
move all the problemsome functions into a folder called lib.  Once i got 
a clean compile. I had a Phobos dll.

Disclaimer:
For the record. I have little internal knowledge on dlls or even 
libraries for that matter. Just lots of accumlated random knowledge. So 
I can't guarntee that the dll produced will always work for every 
situation.  I can't even guaranteee it fixes the GC DLL issue.  All I 
can say is that it didn't crash and handles the Exceptions problem.

With that said, Here's the procedure: It does require the modifed GDC to 
suport full DLLs.

1. Create a directory called gphobos. Enter.
2. Execute "ar x libgphobos.a". That will extrain all the obj files.
3. Create a lib directory.
4. move cmain.o com.o cstream.o dgccmain2.o file.o iunknown.o loader.o 
mmfile.o moduleinit.o recls.o recls_api.o recls_api_win32.o rundmain.o 
socket.o socketstream.o stream.o zip.o zlib.o Into lib
5. In the lib directory type "ar r libg.a *.o". Return to gphobos.
6. Execute "gcc -shared -o gphobos.dll *.o  lib/libg.a 
-Wl,--out-implib=libgphobos.a"
7. Re-enter lib. type "ar x ../libgphobos.a". Once again this explodes 
the library.We do it to get the function stubs.
8. Execute "ar r libgphobos.a *.o". Now copy over your orignal 
libgphobos in your gdc/lib directory.

Notes:
Multiple files may produce duplicate symbols. moduleinit.o will conflict 
with D00XXX.o when you first try to make your program. I do not know how 
exactly to determine this before creating the library. So when it tells 
you a name. Delete the dxxxx.o file and redo step 8.

Once again I guarantee none of this. I can try to help if you want to 
follow my footsteps.

If someone can tell me how to use patch. I can release the gdc dll 
changes. If someone can tell me how to determine which obj files will 
conflict before linking. I can try to create a script that will auto 
convert phobos to a dll.

Venix wrote:
 After playing around with GDC.  I finally got it compiled with mingw and 
 msys.  With a few simple modifications GDC can have full support for 
 windows DLLs.  Inheritance and calling.  I haven't tested this 
 extensively. My simple test case I had for DMD compiled and ran with 
 GDC.  I will be playing with them more in the upcoming weeks, so any 
 subtle errors will most likely show themselves.
 
 One rather minor issue, is my exception fix no longer works. As I 
 believe GDC makes vtbl's readonly. Once again making phobos a dll would 
 fix that.  I'll try the a2dll tool that comes with mingw and see how 
 that works.
 
 Venix wrote:
 
 After a little bit of time and effort spent working with DLLs, D, 
 exceptions, and classes.  I believe I understand the basics to the 
 problems that exist.

 Exceptions crash.
 Annoying. One solution was to wrap all dll bodies with try/catch, 
 which would be going back to the C method. So i'm going to present a 
 solution that works.

 The problem here is that Exceptions are static. Meaning both your DLL 
 and program get a copy. So if your using the posted method of syncing 
 your garbage collection, this is trivial to do. Just add a void*[] 
 argument to you Load function. Then when you call it. Call
   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
 Now within Load. You have to copy the supplied vtbl over the internal 
 Exception vtbl.
     for (int i=0; i<vtbl.length;++i)
         Exception.classinfo.vtbl[i] = vtbl[i];

 Now exceptions should work like a charm

 http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)


 Minor issues(Can be worked around with a simple D parser):
 1: new cannot be used on an imported class.
 2: Classes/Interfaces cannot be inherited from a DLL

 Both those stem from the class name not be exported. Quoting D's 
 specs. "A class can be exported, which means its name and all its 
 non-private members are exposed externally to the DLL or EXE." which i 
 have found not to be true with the current compiler.

 Solution:
 I compiled the D front end. Modified func.c 
 FunctionDeclaration::isExport to return true if the class is exported. 
 Also i added isExport to AggregateDeclaration.  This allowed me to 
 export all exported symbols. Class/Interfaces/etc need to be doctored 
 with _Class_ _interface, and the _ from functions need to be removed.

 That allows me to inherit and theoretically call new on functions. 
 Then I run into access violations.

 Major Issues(I didn't find a solution):
 1: Inheritance doesn't work. Access violation on method calls.
 2: new causes access violation.

 After a few hours of fiddling. I came to the following conclusions.
 If I make a static library. It imports 2 more symbols for every class 
 _init_ & _vtbl_. So I thought if I exported em it may work.
 Wrong. The linker/compiler never uses those for exported classes.

 In fact I believe all access violations, with dlls, are caused by that 
 issue.  Upon inspecting the imported class within its dll. ClassInfo 
 resolves into something useful. Outside the class, ClassInfo is 
 garbage, just try to print baseclass.classinfo.name.  I have tried to 
 fix this improper Classinfo structure by copying over the real with 
 the bad, but i got nothing but access violations. So I'll assume that 
 this ClassInfo structure is not initialized and I'm just writing to 
 bad memory.

 I don't know how that structure is made. I would guess maybe by _init_ 
 or _vtbl_. But I hope this information may help fix the issues.

 I'm going to play around with the GDC compiler in hopes that I can get 
 a working solution with that. I believe once DLL classes are supported 
 phobos/ares can be can be compiled as a dll and then the exception 
 issue will resolve itself. Along with the garbage collection. Allowing 
 Dlls to be used without a load function.
Nov 18 2005
parent Venix <venix cbpu.com> writes:
Due to some issues with the first method I described. I've generated a 
script which will do all the work for you.  One oddity in the script tho 
is I let moduleinit.o get included twice.  This is because I need to 
static link it. Yet it's required by the majority of the code. The 
reason for the static link is that _moduleDtor crashes on program exit. 
   Statically linking it fixes it.

Script should be ran with msys.

Create a directory called gphobos.
Put the script in it. Put libgphobos.a in it.
Run the script. gphobos.dll can be found in the root. libgphobos.a can 
be found in the ./lib/

Just copy the new libgphobos.a over the old. And make sure you include 
gphobos.dll with the binaries.
Nov 18 2005