www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Quiz of the day: Why does this not work?

reply TomD <t_demmer nospam.web.de> writes:
Hi,
this is about dmd and DLLs.

Given a simple object hierachy in myclasses.d:
module myclasses;
class base{ char[] toString(){ return "I am base";} }
class c1: base{ char[] toString(){ return "I am c1";} }
class c2: base{ char[] toString(){ return "I am c2";} }

and a main file that first does some sanity checks, and then loads and
calls a function in a DLL:

mymain.d:
import tango.sys.SharedLib;

import myclasses;

void main(){
  base[] instances;
  // populate instances
  instances.length=3;
  instances[0] = new base;
  instances[1] = new c1;
  instances[2] = new c2;
  // no problem
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  SharedLib lib = SharedLib.load(`mydll.dll`);
  assert( lib !is null);
  
  extern(C) void function(base[]) my_c_check;
  
  void*  ptr;
  void** point;
  ptr = lib.getSymbol("my_c_check");
  point = cast(void** ) &my_c_check;
  *point = ptr;

  my_c_check( instances );
}

Finally, a DLL that is supposed to work on instances, mydll.d:
import myclasses;
import tango.sys.win32.Types;
import tango.util.log.Trace;

// The core DLL init code, taken from tango wiki.
extern (C) bool  rt_init( void delegate( Exception ) dg = null );
extern (C) bool  rt_term( void delegate( Exception ) dg = null ); 

HINSTANCE g_hInst;

extern (Windows) BOOL 
DllMain(HINSTANCE hInstance, ULONG ulReason, LPVOID pvReserved){
  switch (ulReason){
  case DLL_PROCESS_ATTACH:
    rt_init();
    break;
    
  case DLL_PROCESS_DETACH:
    rt_term();
    break;
    
  case DLL_THREAD_ATTACH:
  case DLL_THREAD_DETACH:
    // Multiple threads not supported yet
    return false;
  }
  g_hInst=hInstance;
  return true;
}
// End of core DLL Init

export extern(C) int my_c_check( base[] instances){
  Trace.formatln("into my_c_check");
  assert( instances.length ==3 );
  Trace.formatln("length check OK");
  assert( instances[0] !is null);
  assert( instances[1] !is null);
  assert( instances[2] !is null);
  Trace.formatln("instances check OK");
  Trace.formatln("instances[0] says: {}", instances[0].toString() );
  Trace.formatln("instances[1] says: {}", instances[1].toString() );
  Trace.formatln("instances[2] says: {}", instances[2].toString() );
  Trace.formatln("instances[0] is: {}", instances[0].classinfo.name );
  Trace.formatln("instances[1] is: {}", instances[1].classinfo.name );
  Trace.formatln("instances[2] is: {}", instances[2].classinfo.name );
  // Boom!
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  return 0;
}

This is a real show stopper for using D with/for dynamic libraries.
Is there anything simple to fix this?

Ciao
Tom
Nov 06 2008
next sibling parent reply "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 3:14 PM, TomD <t_demmer nospam.web.de> wrote:
 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }

 and a main file that first does some sanity checks, and then loads and
 calls a function in a DLL:

 mymain.d:
 import tango.sys.SharedLib;

 import myclasses;

 void main(){
  base[] instances;
  // populate instances
  instances.length=3;
  instances[0] = new base;
  instances[1] = new c1;
  instances[2] = new c2;
  // no problem
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  SharedLib lib = SharedLib.load(`mydll.dll`);
  assert( lib !is null);

  extern(C) void function(base[]) my_c_check;

  void*  ptr;
  void** point;
  ptr = lib.getSymbol("my_c_check");
  point = cast(void** ) &my_c_check;
  *point = ptr;

  my_c_check( instances );
 }

 Finally, a DLL that is supposed to work on instances, mydll.d:
 import myclasses;
 import tango.sys.win32.Types;
 import tango.util.log.Trace;

 // The core DLL init code, taken from tango wiki.
 extern (C) bool  rt_init( void delegate( Exception ) dg = null );
 extern (C) bool  rt_term( void delegate( Exception ) dg = null );

 HINSTANCE g_hInst;

 extern (Windows) BOOL
 DllMain(HINSTANCE hInstance, ULONG ulReason, LPVOID pvReserved){
  switch (ulReason){
  case DLL_PROCESS_ATTACH:
    rt_init();
    break;

  case DLL_PROCESS_DETACH:
    rt_term();
    break;

  case DLL_THREAD_ATTACH:
  case DLL_THREAD_DETACH:
    // Multiple threads not supported yet
    return false;
  }
  g_hInst=hInstance;
  return true;
 }
 // End of core DLL Init

 export extern(C) int my_c_check( base[] instances){
  Trace.formatln("into my_c_check");
  assert( instances.length ==3 );
  Trace.formatln("length check OK");
  assert( instances[0] !is null);
  assert( instances[1] !is null);
  assert( instances[2] !is null);
  Trace.formatln("instances check OK");
  Trace.formatln("instances[0] says: {}", instances[0].toString() );
  Trace.formatln("instances[1] says: {}", instances[1].toString() );
  Trace.formatln("instances[2] says: {}", instances[2].toString() );
  Trace.formatln("instances[0] is: {}", instances[0].classinfo.name );
  Trace.formatln("instances[1] is: {}", instances[1].classinfo.name );
  Trace.formatln("instances[2] is: {}", instances[2].classinfo.name );
  // Boom!
  assert( cast(c1) instances[1] !is null);
  assert( cast(c2) instances[2] !is null);

  return 0;
 }

 This is a real show stopper for using D with/for dynamic libraries.
 Is there anything simple to fix this?

 Ciao
 Tom

DLLs are unusable. It's not actually D's fault, it's the fault of Microsoft using such a poorly-capable dynamic library loading system. Basically, the problem with DLLs is that they cannot load symbols from the "host" application at load-time. This means that information contained in the host is not available to the DLL unless the host passes that data to the DLL after it has been loaded. In the case of D, there is type info embedded into your applications and libraries that is required for things like class casting (as you've seen here) to work correctly. What happens currently is that when you build your application and then you build the DLL, they each end up _with their own copies of the same type info_, meaning that even though a "host" c1 is the same thing as a DLL c1, they are in effect two different classes as they have two different type infos. The way around this? Don't use DLLs. Use DDLs. http://www.dsource.org/projects/ddl
Nov 06 2008
next sibling parent Sean Kelly <sean invisibleduck.org> writes:
Jarrett Billingsley wrote:
 
 DLLs are unusable.  It's not actually D's fault, it's the fault of
 Microsoft using such a poorly-capable dynamic library loading system.
 Basically, the problem with DLLs is that they cannot load symbols from
 the "host" application at load-time.  This means that information
 contained in the host is not available to the DLL unless the host
 passes that data to the DLL after it has been loaded.

Darnit, I forgot that bit. So some pretty complex compiler / runtime magic would be required for this to work. What a mess. Sean
Nov 06 2008
prev sibling parent TomD <t_demmer nospam.web.de> writes:
Jarrett Billingsley Wrote:

[...]
 
 DLLs are unusable.  It's not actually D's fault, it's the fault of
 Microsoft using such a poorly-capable dynamic library loading system.
 Basically, the problem with DLLs is that they cannot load symbols from
 the "host" application at load-time.  This means that information
 contained in the host is not available to the DLL unless the host
 passes that data to the DLL after it has been loaded.  In the case of
 D, there is type info embedded into your applications and libraries
 that is required for things like class casting (as you've seen here)
 to work correctly.  What happens currently is that when you build your
 application and then you build the DLL, they each end up _with their
 own copies of the same type info_, meaning that even though a "host"
 c1 is the same thing as a DLL c1, they are in effect two different
 classes as they have two different type infos.
 

same) problem on Cygwin, where you have the need to resolve externals at link time, not at load time.
 The way around this?  Don't use DLLs.  Use DDLs.
 http://www.dsource.org/projects/ddl

Last time I tried that I did not manage to get that running, but I'll try harder. Thanks for all the help and insight, Ciao Tom
Nov 06 2008
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
TomD wrote:
 Hi,
 this is about dmd and DLLs.
 
 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);
 
   return 0;
 }

I think this is a bug in the runtime. Look like 116 of: http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d In your example, I'm pretty sure that the DLLs ClassInfo instance of c1 will be passed into _d_isbaseof2 while the ClassInfo obtained from the object to be cast will come from the object's memory space (ie. from the app). Since an 'is' comparison is taking place and these are distinct objects, the cast will fail. Try changing the 'is' comparisons at lines 105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see if that does the trick. Sean
Nov 06 2008
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
"Sean Kelly" wrote
 TomD wrote:
 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);

   return 0;
 }

I think this is a bug in the runtime. Look like 116 of: http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d In your example, I'm pretty sure that the DLLs ClassInfo instance of c1 will be passed into _d_isbaseof2 while the ClassInfo obtained from the object to be cast will come from the object's memory space (ie. from the app). Since an 'is' comparison is taking place and these are distinct objects, the cast will fail. Try changing the 'is' comparisons at lines 105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see if that does the trick.

Won't make a difference. ClassInfo does not override the default opEquals, which does an is compare. -Steve
Nov 06 2008
parent Sean Kelly <sean invisibleduck.org> writes:
Jarrett Billingsley wrote:
 On Thu, Nov 6, 2008 at 4:23 PM, Steven Schveighoffer
 <schveiguy yahoo.com> wrote:
 Won't make a difference.  ClassInfo does not override the default opEquals,
 which does an is compare.

Even if it did, it's not really enough to do something like a name compare to see if the two typeinfos are the same. (Actually now that I think about it, I think classinfo does that.. or did at some point in the past.)

Oops, you're right. TypeInfo does a name comparison but ClassInfo does an identity comparison.
 You would have to check and make sure that every piece
 of the type - every member, every method, all the bases - were the
 same in order for the types to be "equal".  This is obviously a
 nontrivial operation and not something you want to have happen every
 time you do a cast.

I'd expect a name comparison to be sufficient so long as the name is the fully qualified name of the class. At least within a single binary. There's obviously the rare chance that different class implementations could have the same name between an app and a DLL. So my suggestion wouldn't have worked in every case anyway.
 Having duplicated typeinfos is always bad news.  The best solution -
 the one that DDLs and SOs provide - is the only sane, correct one: use
 the same damn typeinfo for the same types.

Agreed. It would be nice to gain some traction here. Sean
Nov 06 2008
prev sibling next sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 4:23 PM, Steven Schveighoffer
<schveiguy yahoo.com> wrote:
 "Sean Kelly" wrote
 TomD wrote:
 Hi,
 this is about dmd and DLLs.

 Given a simple object hierachy in myclasses.d:
 module myclasses;
 class base{ char[] toString(){ return "I am base";} }
 class c1: base{ char[] toString(){ return "I am c1";} }
 class c2: base{ char[] toString(){ return "I am c2";} }

   assert( cast(c1) instances[1] !is null);
   assert( cast(c2) instances[2] !is null);

   return 0;
 }

I think this is a bug in the runtime. Look like 116 of: http://dsource.org/projects/tango/browser/trunk/lib/compiler/dmd/cast.d In your example, I'm pretty sure that the DLLs ClassInfo instance of c1 will be passed into _d_isbaseof2 while the ClassInfo obtained from the object to be cast will come from the object's memory space (ie. from the app). Since an 'is' comparison is taking place and these are distinct objects, the cast will fail. Try changing the 'is' comparisons at lines 105, 109, 116, 150, and 176 (I think that's all of them) to '==' and see if that does the trick.

Won't make a difference. ClassInfo does not override the default opEquals, which does an is compare.

Even if it did, it's not really enough to do something like a name compare to see if the two typeinfos are the same. (Actually now that I think about it, I think classinfo does that.. or did at some point in the past.) You would have to check and make sure that every piece of the type - every member, every method, all the bases - were the same in order for the types to be "equal". This is obviously a nontrivial operation and not something you want to have happen every time you do a cast. Having duplicated typeinfos is always bad news. The best solution - the one that DDLs and SOs provide - is the only sane, correct one: use the same damn typeinfo for the same types.
Nov 06 2008
prev sibling parent "Jarrett Billingsley" <jarrett.billingsley gmail.com> writes:
On Thu, Nov 6, 2008 at 5:34 PM, Sean Kelly <sean invisibleduck.org> wrote:
 of the type - every member, every method, all the bases - were the
 same in order for the types to be "equal".  This is obviously a
 nontrivial operation and not something you want to have happen every
 time you do a cast.

I'd expect a name comparison to be sufficient so long as the name is the fully qualified name of the class. At least within a single binary. There's obviously the rare chance that different class implementations could have the same name between an app and a DLL. So my suggestion wouldn't have worked in every case anyway.

Using identity/name within a single binary is fine. I was mostly considering the case of the host app and the plugin being compiled against different/modified versions of the source.
Nov 06 2008