www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Quiz of the day: Why does this not work?

reply TomD <t_demmer nospam.web.de> writes:
Hi,
this is about dmd and DLLs.

Given a simple object hierachy in myclasses.d:
module myclasses;
class base{ char[] toString(){ return "I am base";} }
class c1: base{ char[] toString(){ return "I am c1";} }
class c2: base{ char[] toString(){ return "I am c2";} }

and a main file that first does some sanity checks, and then loads and
calls a function in a DLL:

mymain.d:
import tango.sys.SharedLib;

import myclasses;

void main(){
  base[] instances;
  // populate instances
  instances.length=3;
  instances[0] = new base;
  instances[1] = new c1;
  instances[2] = new c2;
  // no problem
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  SharedLib lib = SharedLib.load(`mydll.dll`);
  assert( lib !is null);
  
  extern(C) void function(base[]) my_c_check;
  
  void*  ptr;
  void** point;
  ptr = lib.getSymbol("my_c_check");
  point = cast(void** ) &my_c_check;
  *point = ptr;

  my_c_check( instances );
}

Finally, a DLL that is supposed to work on instances, mydll.d:
import myclasses;
import tango.sys.win32.Types;
import tango.util.log.Trace;

// The core DLL init code, taken from tango wiki.
extern (C) bool  rt_init( void delegate( Exception ) dg = null );
extern (C) bool  rt_term( void delegate( Exception ) dg = null ); 

HINSTANCE g_hInst;

extern (Windows) BOOL 
DllMain(HINSTANCE hInstance, ULONG ulReason, LPVOID pvReserved){
  switch (ulReason){
  case DLL_PROCESS_ATTACH:
    rt_init();
    break;
    
  case DLL_PROCESS_DETACH:
    rt_term();
    break;
    
  case DLL_THREAD_ATTACH:
  case DLL_THREAD_DETACH:
    // Multiple threads not supported yet
    return false;
  }
  g_hInst=hInstance;
  return true;
}
// End of core DLL Init

export extern(C) int my_c_check( base[] instances){
  Trace.formatln("into my_c_check");
  assert( instances.length ==3 );
  Trace.formatln("length check OK");
  assert( instances[0] !is null);
  assert( instances[1] !is null);
  assert( instances[2] !is null);
  Trace.formatln("instances check OK");
  Trace.formatln("instances[0] says: {}", instances[0].toString() );
  Trace.formatln("instances[1] says: {}", instances[1].toString() );
  Trace.formatln("instances[2] says: {}", instances[2].toString() );
  Trace.formatln("instances[0] is: {}", instances[0].classinfo.name );
  Trace.formatln("instances[1] is: {}", instances[1].classinfo.name );
  Trace.formatln("instances[2] is: {}", instances[2].classinfo.name );
  // Boom!
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  return 0;
}

This is a real show stopper for using D with/for dynamic libraries.
Is there anything simple to fix this?

Ciao
Tom
Nov 06 2008
next sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 3:14 PM, TomD <t_demmer nospam.web.de> wrote:

 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }

 and a main file that first does some sanity checks, and then loads and
 calls a function in a DLL:

 mymain.d:
 import tango.sys.SharedLib;

 import myclasses;

 void main(){
  base[] instances;
  // populate instances
  instances.length=3;
  instances[0] = new base;
  instances[1] = new c1;
  instances[2] = new c2;
  // no problem
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  SharedLib lib = SharedLib.load(`mydll.dll`);
  assert( lib !is null);

  extern(C) void function(base[]) my_c_check;

  void*  ptr;
  void** point;
  ptr = lib.getSymbol("my_c_check");
  point = cast(void** ) &my_c_check;
  *point = ptr;

  my_c_check( instances );
 }

 Finally, a DLL that is supposed to work on instances, mydll.d:
 import myclasses;
 import tango.sys.win32.Types;
 import tango.util.log.Trace;

 // The core DLL init code, taken from tango wiki.
 extern (C) bool  rt_init( void delegate( Exception ) dg = null );
 extern (C) bool  rt_term( void delegate( Exception ) dg = null );

 HINSTANCE g_hInst;

 extern (Windows) BOOL
 DllMain(HINSTANCE hInstance, ULONG ulReason, LPVOID pvReserved){
  switch (ulReason){
  case DLL_PROCESS_ATTACH:
    rt_init();
    break;

  case DLL_PROCESS_DETACH:
    rt_term();
    break;

  case DLL_THREAD_ATTACH:
  case DLL_THREAD_DETACH:
    // Multiple threads not supported yet
    return false;
  }
  g_hInst=hInstance;
  return true;
 }
 // End of core DLL Init

 export extern(C) int my_c_check( base[] instances){
  Trace.formatln("into my_c_check");
  assert( instances.length ==3 );
  Trace.formatln("length check OK");
  assert( instances[0] !is null);
  assert( instances[1] !is null);
  assert( instances[2] !is null);
  Trace.formatln("instances check OK");
  Trace.formatln("instances[0] says: {}", instances[0].toString() );
  Trace.formatln("instances[1] says: {}", instances[1].toString() );
  Trace.formatln("instances[2] says: {}", instances[2].toString() );
  Trace.formatln("instances[0] is: {}", instances[0].classinfo.name );
  Trace.formatln("instances[1] is: {}", instances[1].classinfo.name );
  Trace.formatln("instances[2] is: {}", instances[2].classinfo.name );
  // Boom!
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  return 0;
 }

 This is a real show stopper for using D with/for dynamic libraries.
 Is there anything simple to fix this?

 Ciao
 Tom


DLLs are unusable.  It's not actually D's fault, it's the fault of
Microsoft using such a poorly-capable dynamic library loading system.
Basically, the problem with DLLs is that they cannot load symbols from
the "host" application at load-time.  This means that information
contained in the host is not available to the DLL unless the host
passes that data to the DLL after it has been loaded.  In the case of
D, there is type info embedded into your applications and libraries
that is required for things like class casting (as you've seen here)
to work correctly.  What happens currently is that when you build your
application and then you build the DLL, they each end up _with their
own copies of the same type info_, meaning that even though a "host"
c1 is the same thing as a DLL c1, they are in effect two different
classes as they have two different type infos.

The way around this?  Don't use DLLs.  Use DDLs.
http://www.dsource.org/projects/ddl
Nov 06 2008
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
Jarrett Billingsley wrote:

 
 DLLs are unusable.  It's not actually D's fault, it's the fault of
 Microsoft using such a poorly-capable dynamic library loading system.
 Basically, the problem with DLLs is that they cannot load symbols from
 the "host" application at load-time.  This means that information
 contained in the host is not available to the DLL unless the host
 passes that data to the DLL after it has been loaded.


Darnit, I forgot that bit.  So some pretty complex compiler / runtime 
magic would be required for this to work.  What a mess.


Sean
Nov 06 2008
prev sibling parent TomD <t_demmer nospam.web.de> writes:
Jarrett Billingsley Wrote:

[...]

 
 DLLs are unusable.  It's not actually D's fault, it's the fault of
 Microsoft using such a poorly-capable dynamic library loading system.
 Basically, the problem with DLLs is that they cannot load symbols from
 the "host" application at load-time.  This means that information
 contained in the host is not available to the DLL unless the host
 passes that data to the DLL after it has been loaded.  In the case of
 D, there is type info embedded into your applications and libraries
 that is required for things like class casting (as you've seen here)
 to work correctly.  What happens currently is that when you build your
 application and then you build the DLL, they each end up _with their
 own copies of the same type info_, meaning that even though a "host"
 c1 is the same thing as a DLL c1, they are in effect two different
 classes as they have two different type infos.
 


Understood. I have seen a similar (or, as I realize now: the
same) problem on Cygwin, where you have the need to resolve
externals at link time, not at load time.

 The way around this?  Don't use DLLs.  Use DDLs.
 http://www.dsource.org/projects/ddl


Last time I tried that I did not manage to get that running,
but I'll try harder.

Thanks for all the help and insight,

Ciao
Tom
Nov 06 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
TomD wrote:

 Hi,
 this is about dmd and DLLs.
 
 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }


...

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);
 
   return 0;
 }


I think this is a bug in the runtime.  Look like 116 of:

http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d

In your example, I'm pretty sure that the DLLs ClassInfo instance of c1 
will be passed into _d_isbaseof2 while the ClassInfo obtained from the 
object to be cast will come from the object's memory space (ie. from the 
app).  Since an 'is' comparison is taking place and these are distinct 
objects, the cast will fail.  Try changing the 'is' comparisons at lines 
105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see 
if that does the trick.


Sean
Nov 06 2008
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Sean Kelly" wrote

 TomD wrote:

 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }


 ...

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);

   return 0;
 }


 I think this is a bug in the runtime.  Look like 116 of:

 http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d

 In your example, I'm pretty sure that the DLLs ClassInfo instance of c1 
 will be passed into _d_isbaseof2 while the ClassInfo obtained from the 
 object to be cast will come from the object's memory space (ie. from the 
 app).  Since an 'is' comparison is taking place and these are distinct 
 objects, the cast will fail.  Try changing the 'is' comparisons at lines 
 105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see 
 if that does the trick.


Won't make a difference.  ClassInfo does not override the default opEquals, 
which does an is compare.

-Steve
Nov 06 2008
parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 4:23 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:

 "Sean Kelly" wrote

 TomD wrote:

 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }


 ...

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);

   return 0;
 }


 I think this is a bug in the runtime.  Look like 116 of:

 http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d

 In your example, I'm pretty sure that the DLLs ClassInfo instance of c1
 will be passed into _d_isbaseof2 while the ClassInfo obtained from the
 object to be cast will come from the object's memory space (ie. from the
 app).  Since an 'is' comparison is taking place and these are distinct
 objects, the cast will fail.  Try changing the 'is' comparisons at lines
 105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see
 if that does the trick.


 Won't make a difference.  ClassInfo does not override the default opEquals,
 which does an is compare.


Even if it did, it's not really enough to do something like a name
compare to see if the two typeinfos are the same.  (Actually now that
I think about it, I think classinfo does that.. or did at some point
in the past.)  You would have to check and make sure that every piece
of the type - every member, every method, all the bases - were the
same in order for the types to be "equal".  This is obviously a
nontrivial operation and not something you want to have happen every
time you do a cast.

Having duplicated typeinfos is always bad news.  The best solution -
the one that DDLs and SOs provide - is the only sane, correct one: use
the same damn typeinfo for the same types.
Nov 06 2008
parent reply Sean Kelly <sean invisibleduck.org> writes:
Jarrett Billingsley wrote:

 On Thu, Nov 6, 2008 at 4:23 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:

 Won't make a difference.  ClassInfo does not override the default opEquals,
 which does an is compare.


 
 Even if it did, it's not really enough to do something like a name
 compare to see if the two typeinfos are the same.  (Actually now that
 I think about it, I think classinfo does that.. or did at some point
 in the past.)


Oops, you're right.  TypeInfo does a name comparison but ClassInfo does 
an identity comparison.


 You would have to check and make sure that every piece
 of the type - every member, every method, all the bases - were the
 same in order for the types to be "equal".  This is obviously a
 nontrivial operation and not something you want to have happen every
 time you do a cast.


I'd expect a name comparison to be sufficient so long as the name is the 
fully qualified name of the class.  At least within a single binary. 
There's obviously the rare chance that different class implementations 
could have the same name between an app and a DLL.  So my suggestion 
wouldn't have worked in every case anyway.


 Having duplicated typeinfos is always bad news.  The best solution -
 the one that DDLs and SOs provide - is the only sane, correct one: use
 the same damn typeinfo for the same types.


Agreed.  It would be nice to gain some traction here.


Sean
Nov 06 2008
parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 5:34 PM, Sean Kelly <sean invisibleduck.org> wrote:

 of the type - every member, every method, all the bases - were the
 same in order for the types to be "equal".  This is obviously a
 nontrivial operation and not something you want to have happen every
 time you do a cast.


 I'd expect a name comparison to be sufficient so long as the name is the
 fully qualified name of the class.  At least within a single binary. There's
 obviously the rare chance that different class implementations could have
 the same name between an app and a DLL.  So my suggestion wouldn't have
 worked in every case anyway.


Using identity/name within a single binary is fine.  I was mostly
considering the case of the host app and the plugin being compiled
against different/modified versions of the source.
Nov 06 2008