www.digitalmars.com         C & C++   DMDScript  

D - Runtime linking question

reply "chris jones" <flak clara.co.uk> writes:
Hi, im not sure what the corect terminoly is but i have a question. Many
programs are utilising plugins, scanned for and linked at runtime, and the
ones i know of are all based on dlls. But dlls are limited to a C based
interface so if you want to use OO code then you have bodge it together at
either end. So i think it would be very usefull if there was some way to
have D object code that could link at runtime, or even a DDLL that could
have an object orientated interface. I guese that there would need to be a
compatible class declaration in both the host and the plugin and the
compatibilty would need to be check when th plugin is loaded, but that
shouldnt be to hard to do? Or is this posible already, if it is where should
i be looking for some info?

thanks,

chris
Sep 25 2002
parent reply Mac Reiter <Mac_member pathlink.com> writes:
In article <amsh1o$2oup$1 digitaldaemon.com>, chris jones says...
Hi, im not sure what the corect terminoly is but i have a question. Many
programs are utilising plugins, scanned for and linked at runtime, and the
ones i know of are all based on dlls. But dlls are limited to a C based
interface so if you want to use OO code then you have bodge it together at
either end. So i think it would be very usefull if there was some way to
have D object code that could link at runtime, or even a DDLL that could
have an object orientated interface. I guese that there would need to be a
compatible class declaration in both the host and the plugin and the
compatibilty would need to be check when th plugin is loaded, but that
shouldnt be to hard to do? Or is this posible already, if it is where should
i be looking for some info?

thanks,
chris

Not sure if this exactly fits your question, but thought I'd throw it out there anyway. Nothing about DLLs or other runtime linkable libraries precludes the ability to have OO linkage. The problem comes from non-standardized name mangling. ANSI did not specify any standard method of name mangling to support overloaded functions and other C++ issues, so each compiler developer made up their own. Since linkage occurs by names, and since each compiler has its own method of coming up with names, that means that the linkage between components made by different compilers won't work. In straight C, there is no name mangling (or, more accurately, very minimal name mangling that is standardized so that everyone does it the same way), so a MSVC .EXE can link in a Borland DLL because the used the same names. At least in theory (I have seen articles on this, but have never actually tried it) it is possible to have classes in DLLs. You just have to use the same compiler for your client as you did for the DLL. This potentially means not only the same manufacturer, but the same version and maybe even the same patch level. The second issue involves truly dynamic loading. If the plugin is being found through a directory search or other runtime system, the LoadLibrary (Win32) and GetProcAddress calls must be made, and I'm not sure exactly which procedures you would have to get the address of to properly support classes. I think all of the articles I've seen on DLL based classes used the .LIB stub library approach, which is more of a shared library than a dynamic library methodology. Dunno if that provided any useful info, or if I just wasted a couple of K of bandwidth... Hopefully the former. Mac
Sep 25 2002
next sibling parent reply "chris jones" <flak clara.co.uk> writes:
"Mac Reiter" <Mac_member pathlink.com> wrote in message
news:amsung$c85$1 digitaldaemon.com...
 In article <amsh1o$2oup$1 digitaldaemon.com>, chris jones says...

Not sure if this exactly fits your question, but thought I'd throw it out

 anyway.  Nothing about DLLs or other runtime linkable libraries precludes

 ability to have OO linkage.  The problem comes from non-standardized name
 mangling.  ANSI did not specify any standard method of name mangling to

 overloaded functions and other C++ issues, so each compiler developer made

 their own.  Since linkage occurs by names, and since each compiler has its

 method of coming up with names, that means that the linkage between

 made by different compilers won't work.  In straight C, there is no name
 mangling (or, more accurately, very minimal name mangling that is

 so that everyone does it the same way), so a MSVC .EXE can link in a

 DLL because the used the same names.

 At least in theory (I have seen articles on this, but have never actually

 it) it is possible to have classes in DLLs.  You just have to use the same
 compiler for your client as you did for the DLL.  This potentially means

 only the same manufacturer, but the same version and maybe even the same

 level.

Its not an apraoch that would be suitable for an open format, i mean its ok to do this if you are the programer of both host and plugin, but if lots of diferant people are writing plugins it would be troublesome. The way VST plugins work all the functions are routed to the object through static functions via pointers. Its quite messy. Theres about twice as much code just for c interface as there is for the class it is interfacing.
 The second issue involves truly dynamic loading.  If the plugin is being

 through a directory search or other runtime system, the LoadLibrary

 GetProcAddress calls must be made, and I'm not sure exactly which

 would have to get the address of to properly support classes.  I think all

 the articles I've seen on DLL based classes used the .LIB stub library

 which is more of a shared library than a dynamic library methodology.

I'm not sure what the .LIB stub apraoch is. I am thinking that a specifation for dynamicly loadable code with an OO interface would be very valuable, although i have little understanding of the complexities it would involve. For it to be effective the actualy memory structure of the object would need to defined indenticly in plugin and host. Which may be practical if the same compiler is used for both. The more i think about the more complicated it seems. Mabey its not even a D issue but an OS issue? chris
Sep 25 2002
parent reply "Walter" <walter digitalmars.com> writes:
D can access DLLs written that expose functions with a C interface. It can
also interface with COM objects, which is the more usual approach to putting
a class interface on a DLL.
Sep 26 2002
parent reply Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:
 D can access DLLs written that expose functions with a C interface. It can
 also interface with COM objects, which is the more usual approach to putting
 a class interface on a DLL.

Stonewheel handles classes in loaded objects. I'm going to revisit it again someday.
Sep 26 2002
parent reply "chris jones" <flak clara.co.uk> writes:
"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an0jho$ukv$1 digitaldaemon.com...
 Walter wrote:
 D can access DLLs written that expose functions with a C interface. It


 also interface with COM objects, which is the more usual approach to


 a class interface on a DLL.

Stonewheel handles classes in loaded objects. I'm going to revisit it again someday.

Eh? Does that mean COM is very slow? chris
Sep 27 2002
next sibling parent "Walter" <walter digitalmars.com> writes:
"chris jones" <flak clara.co.uk> wrote in message
news:an2roc$319j$1 digitaldaemon.com...
 Eh? Does that mean COM is very slow?

COM has its share of problems, but efficiency isn't one of them.
Sep 28 2002
prev sibling parent reply Burton Radons <loth users.sourceforge.net> writes:
chris jones wrote:
 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an0jho$ukv$1 digitaldaemon.com...
 
Walter wrote:

D can access DLLs written that expose functions with a C interface. It


can
also interface with COM objects, which is the more usual approach to


putting
a class interface on a DLL.

Stonewheel handles classes in loaded objects. I'm going to revisit it again someday.

Eh? Does that mean COM is very slow?

No - it goes through a normal vtable, so it's as fast as a completely virtual object, although it's twice as large on stack (if I understand the object model properly - interfaces aren't in my port yet). Which I appear to not, looking at DMD. Walter, is there a search involved in calling a method on any interface reference? If there is, it'll be significantly slower. If there isn't, it'll be slightly faster depending on how the cache is feeling. But anyway. Most classes aren't fully virtual; with a normal class, involving public fields and final and static methods, Stonewheel will be far faster, as it's just like linking the module into the executable.
Sep 29 2002
next sibling parent "chris jones" <flak clara.co.uk> writes:
"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an827i$2hc4$1 digitaldaemon.com...
 chris jones wrote:
 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an0jho$ukv$1 digitaldaemon.com...

But anyway. Most classes aren't fully virtual; with a normal class, involving public fields and final and static methods, Stonewheel will be far faster, as it's just like linking the module into the executable.

Its the 'Stonewheel' that has thown me, i thought it was a joke as in 'stone wheels' would be very slow :o) chris
Sep 29 2002
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Burton Radons" <loth users.sourceforge.net> wrote in message
news:an827i$2hc4$1 digitaldaemon.com...
 No - it goes through a normal vtable, so it's as fast as a completely
 virtual object, although it's twice as large on stack (if I understand
 the object model properly - interfaces aren't in my port yet).  Which I
 appear to not, looking at DMD.  Walter, is there a search involved in
 calling a method on any interface reference?  If there is, it'll be
 significantly slower.  If there isn't, it'll be slightly faster
 depending on how the cache is feeling.

No, no search. It uses vtbl[]s and adjustor thunks, and so is as efficient as MI dispatch in C++.
Oct 05 2002
parent reply Burton Radons <loth users.sourceforge.net> writes:
Walter wrote:
 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:an827i$2hc4$1 digitaldaemon.com...
 
No - it goes through a normal vtable, so it's as fast as a completely
virtual object, although it's twice as large on stack (if I understand
the object model properly - interfaces aren't in my port yet).  Which I
appear to not, looking at DMD.  Walter, is there a search involved in
calling a method on any interface reference?  If there is, it'll be
significantly slower.  If there isn't, it'll be slightly faster
depending on how the cache is feeling.

No, no search. It uses vtbl[]s and adjustor thunks, and so is as efficient as MI dispatch in C++.

Know of a link describing memory behaviour here? Glancing over C++'s spec, I don't see them describing it. Although to be honest, I can't look too hard at it for fear of becoming blind or insane.
Oct 05 2002
parent reply "Walter" <walter digitalmars.com> writes:
"Burton Radons" <loth users.sourceforge.net> wrote in message
news:annomt$1ujr$2 digitaldaemon.com...
 Walter wrote:
 No, no search. It uses vtbl[]s and adjustor thunks, and so is as


 as MI dispatch in C++.

spec, I don't see them describing it. Although to be honest, I can't look too hard at it for fear of becoming blind or insane.

It wouldn't be described in the C++ spec, as it's an implementation detail. I think Bjarne Stroustrup wrote an article on a scheme to do this long ago, but I can't recall for sure. Check his web site.
Oct 05 2002
parent "Walter" <walter digitalmars.com> writes:
Thanks!
Oct 12 2002
prev sibling next sibling parent Antti Sykari <jsykari cc.hut.fi> writes:
"Walter" <walter digitalmars.com> writes:

 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:annomt$1ujr$2 digitaldaemon.com...
 Walter wrote:
 No, no search. It uses vtbl[]s and adjustor thunks, and so is as


 as MI dispatch in C++.

spec, I don't see them describing it. Although to be honest, I can't look too hard at it for fear of becoming blind or insane.

It wouldn't be described in the C++ spec, as it's an implementation detail. I think Bjarne Stroustrup wrote an article on a scheme to do this long ago, but I can't recall for sure. Check his web site.

Stroustrup's paper "Multiple Inheritance in C++" is available at: http://citeseer.nj.nec.com/stroustrup99multiple.html Antti.
Oct 12 2002
prev sibling parent Antti Sykari <jsykari cc.hut.fi> writes:
By the way,

here's another paper I bumped into. It claims to do faster MI virtual
function call dispatch than the traditional C++ compilers, also
enabling separate compilation of base classes:

http://citeseer.nj.nec.com/myers95bidirectional.html

The abstract:

  Bidirectional Object Layout for Separate Compilation

  Existing schemes for object layout and dispatch in the presence of
  multiple inheritance and separate compilation waste space and are
  slower than systems with single inheritance. This paper describes
  the bidirectional object layout, a new scheme for object layout that
  produces smaller objects and faster method invocations than existing
  schemes by automatically optimizing particular uses of multiple
  inheritance. The bidirectional object layout is used for the
  programming language Theta, and is applicable to languages like C++.
  This paper also demonstrates how to efficiently implement method
  dispatch when method signatures are allowed to change in subclasses.
  Most current statically compiled languages require identical
  signatures for efficiency.

A.
Oct 12 2002