www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Weak Linking, on Windows

reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Why do "all" the Windows libraries
(as in: Derelict, libiconv, etc) use
"runtime loading" to link to DLL files ?

As in: declaring a bunch of function
pointers to the functions, and then
fill them in at runtime with std.loader.


Why not link to the libraries directly ?
Is this because the packaging used does
not handle lib dependencies, or something ?

Because on Linux / Darwin, it's simple...
There you link to dynamic libraries, just
like you could link to the static libraries:

-lfoo -lbar

If it's somehow required, couldn't the compiler
do this - instead of having to do it manually... ?
(I know why eg. the OpenGL extensions does it, but
that doesn't really explain why *all* function do)

On Mac OS 9 (may it rest in pieces), it was called
"weak linking" when you linked to a shared library
like that. If the "DLL" couldn't be found, all the
references to those functions/symbols became NULL.


I've ported the std.loader already, so that's not it.
It's just that it gets reeeaaally boring to cut and
paste "linux" to "darwin" and change ".so" to ".dylib"
or ".framework", and make sure all those function pointers
synch with the traditional C header function declaration...

 public void DerelictGL_Load()
 {
     version(Windows)
         DerelictGL_Load("opengl32.dll");
     version(linux)
         DerelictGL_Load("libGL.so");
     version(darwin)
         DerelictGL_Load("OpenGL.framework");
 }

Just because one of the systems doesn't "cut the mustard" ? I think it makes more sense to just port the C header files, and use the regular "-lGL" or "-framework OpenGL" linking - and let the system handle the library version dependencies... Not that it makes any sense in the OpenGL example used above, but it would also allow you to switch to a static library ? (by using a libfoo.a, instead of libfoo.so / libfoo.dylib) --anders
Mar 28 2005
parent reply Mike Parker <aldacron71 yahoo.com> writes:
Anders F Björklund wrote:
 Why do "all" the Windows libraries
 (as in: Derelict, libiconv, etc) use
 "runtime loading" to link to DLL files ?

I can't speak for anyone else, but for Derelict it was a design decision. I think I explain it pretty well in the Derelict forums and also in the old Readme (now index.html in the trunk). But I'll go for it here, again. The problem, for me, with linking to a static import library is that you have zero flexibility. Lets imagine that you want to create an audio player that can make use of OpenAL, FMOD, or other audio libraries depending upon the user's system specs, what's presinstalled, or some other criteria. If you statically link to the import libraries, all of these shared libs (or DLLs) will then be loaded into memory simultaneously. Manual loading of the library gives you the ability to, instead, load one DLL at a time and swap out DLLs as needed. This is just one benefit. Another benefit is that you, as the application developer, now have the option to handle the case of missing/corrupt DLLs in an application specific manner. Have you ever seen the dialog that Windows pops up when a dependent DLL is missing? It does nothing to help the user solve the problem. By loading manually, you can catch the Exception thrown by Derelict and load an alternative library, display a message box with detailed instructions on how to solve the problem, or whatever else you want to do. What happens if you statically link to an import library for SomeLibrary version 2.0, but the user has SomeLibrary 1.8? Normally, you would distribute the proper library with your application, but in some cases you may decide it's a better option to use the latest features if available, and fallback on older features if not. If you have linked statically with version 2.0's import library, and version 2.0 exports symbols that version 1.9 does not, then you cannot use this earlier version. Derelict, by default, throws an exception when a symbol cannot be loaded. But as of a month ago now allows you override this behavior for particular functions. So, flexibility and control. That's why I've designed it the way I have. And this applies to all platforms, it's not just a Windows thing. Other bindings exist already that require static linkage. You can find many of them in the Bindings project (and a few other projects as well) at DSource. I started Derelict as an alternative for those who prefer, or need, the additional flexibility.
Mar 28 2005
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Mike Parker wrote:

 If you statically link to the import libraries, all of 
 these shared libs (or DLLs) will then be loaded into memory 
 simultaneously. Manual loading of the library gives you the ability to, 
 instead, load one DLL at a time and swap out DLLs as needed. This is 
 just one benefit.

OK. (the system handles all the framework loading on Mac OS X, including the showing of dialogs and bundling of custom ones)
 So, flexibility and control. That's why I've designed it the way I have. 
 And this applies to all platforms, it's not just a Windows thing.  Other 
 bindings exist already that require static linkage. You can find many of 
 them in the Bindings project (and a few other projects as well) at 
 DSource. I started Derelict as an alternative for those who prefer, or 
 need, the additional flexibility.

It's been discussed before, and thanks for repeating it again. BTW: What tools are you using to generate all these "bindings" ? Guess I was just a little surprised to see "std.loader" re-invented again in Derelict, just as I was surprised to see it in Mango too ? Granted, the std.loader code is about the worst I've seen (sorry, Matthew) but it would be better for all if it could be replaced. But I've done one version for Mango, can do one for Derelict. (and the darwin OS patch for std.loader doesn't seem to have made it to DMD 0.119, but it is being included with GDC 0.10) Then it's just a matter of adding frameworks to the _Load methods... --anders
Mar 28 2005
next sibling parent reply Mike Parker <aldacron71 yahoo.com> writes:
Anders F Björklund wrote:
 OK. (the system handles all the framework loading on Mac OS X,
 including the showing of dialogs and bundling of custom ones)

I've never used Mac, but I'm not quite sure what you're saying here. Depending on how I interpret this, the same could be said for windows DLLs and Linux SOs when linking with a static import. If you mean something different... Are you saying it's possible to replace system error messages with custom ones? Is it possible to 'hot swap' shared libraries even though you are statically linked to an import lib? I've never heard that before if that's the case.
 BTW: What tools are you using to generate all these "bindings" ?

My two hands :)
 Guess I was just a little surprised to see "std.loader" re-invented
 again in Derelict, just as I was surprised to see it in Mango too ?

I used std.loader up until about a month ago in Derelict as well. But on linux, std.loader is not compiled into Phobos by default. This means that anyone using Derelict on Linux would also need to include std.loader in the app build or link with the object file. It's just an annoying dependency that is solved by DerelictUtil. Also, I updated DerelictUitl today, so if you are porting Derelict to Mac you might want to update your working copy. I consolidated some of the derelict.util.loader code into platform-agnostic functions, and added a new exception type to derelict.util.exception.
Mar 29 2005
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Mike Parker wrote:

 Are you saying it's possible to replace system 
 error messages with custom ones? Is it possible to 'hot swap' shared 
 libraries even though you are statically linked to an import lib? I've 
 never heard that before if that's the case.

Well, no not really. It's system warning messages and runtime bindings. (i.e. you link with the shared library/framework, no import lib needed) So the method you are using still works just fine on Mac OS X too, and has similar advantages. Just needs "porting" to Mach-O format. But since *each* Mac application ships their own shared library copy (well, not OpenGL...) they know which minimum version they link to.
 BTW: What tools are you using to generate all these "bindings" ?

My two hands :)

Really? Guess your ctrl-C and ctrl-V must be tired by now then ? ;-) I used Perl.
 Guess I was just a little surprised to see "std.loader" re-invented
 again in Derelict, just as I was surprised to see it in Mango too ?

I used std.loader up until about a month ago in Derelict as well. But on linux, std.loader is not compiled into Phobos by default.

AAAARGH. (should have guessed, posted the Darwin patch around DMD 0.102)
 This means that anyone using Derelict on Linux would also need to include 
 std.loader in the app build or link with the object file. It's just an 
 annoying dependency that is solved by DerelictUtil.

I wouldn't bother with std.loader, as both it and it's license suck. :-)
 Also, I updated DerelictUitl today, so if you are porting Derelict to 
 Mac you might want to update your working copy. I consolidated some of 
 the derelict.util.loader code into platform-agnostic functions, and 
 added a new exception type to derelict.util.exception.

OK, thanks...
 U  DerelictUtil/derelict/util/exception.d
 U  DerelictUtil/derelict/util/loader.d
 Updated to revision 102.

--anders
Mar 29 2005
prev sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Anders F Björklund" <afb algonet.se> wrote in message
news:d2aveu$pmp$1 digitaldaemon.com...
 Mike Parker wrote:

 If you statically link to the import libraries, all of these shared libs (or
DLLs) will then be loaded into memory 
 simultaneously. Manual loading of the library gives you the ability to,
instead, load one DLL at a time and swap out 
 DLLs as needed. This is just one benefit.

OK. (the system handles all the framework loading on Mac OS X, including the showing of dialogs and bundling of custom ones)
 So, flexibility and control. That's why I've designed it the way I have. And
this applies to all platforms, it's not 
 just a Windows thing.  Other bindings exist already that require static
linkage. You can find many of them in the 
 Bindings project (and a few other projects as well) at DSource. I started
Derelict as an alternative for those who 
 prefer, or need, the additional flexibility.

It's been discussed before, and thanks for repeating it again. BTW: What tools are you using to generate all these "bindings" ? Guess I was just a little surprised to see "std.loader" re-invented again in Derelict, just as I was surprised to see it in Mango too ? Granted, the std.loader code is about the worst I've seen (sorry, Matthew) but it would be better for all if it could be replaced.

Doesn't bother me. It was given the Walter-treatment before it went in, so barely reflects what I wanted it to be. ;/
Mar 29 2005
next sibling parent "Matthew" <admin.hat stlsoft.dot.org> writes:
"Matthew" <admin.hat stlsoft.dot.org> wrote in message
news:d2b52j$v60$1 digitaldaemon.com...
 "Anders F Björklund" <afb algonet.se> wrote in message
news:d2aveu$pmp$1 digitaldaemon.com...
 Mike Parker wrote:

 If you statically link to the import libraries, all of these shared libs (or
DLLs) will then be loaded into memory 
 simultaneously. Manual loading of the library gives you the ability to,
instead, load one DLL at a time and swap out 
 DLLs as needed. This is just one benefit.

OK. (the system handles all the framework loading on Mac OS X, including the showing of dialogs and bundling of custom ones)
 So, flexibility and control. That's why I've designed it the way I have. And
this applies to all platforms, it's not 
 just a Windows thing.  Other bindings exist already that require static
linkage. You can find many of them in the 
 Bindings project (and a few other projects as well) at DSource. I started
Derelict as an alternative for those who 
 prefer, or need, the additional flexibility.

It's been discussed before, and thanks for repeating it again. BTW: What tools are you using to generate all these "bindings" ? Guess I was just a little surprised to see "std.loader" re-invented again in Derelict, just as I was surprised to see it in Mango too ? Granted, the std.loader code is about the worst I've seen (sorry, Matthew) but it would be better for all if it could be replaced.

Doesn't bother me. It was given the Walter-treatment before it went in, so barely reflects what I wanted it to be. ;/

Yuck! I just checked it out. It's still got the old license!! I think someone really needs to take responsibility for Phobos. It's like a kids toy box where everything's just been stuffed in before mummy comes home.
Mar 29 2005
prev sibling parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
 Granted, the std.loader code is about the worst I've seen (sorry, Matthew) but
it would be better for all if it could 
 be replaced.


One thing: you musn't have seen much production code if that's the worst you've seen. I mean, sure, it's pretty yucksome, but I've seen **far** worse in my travels!! ;)
Mar 29 2005
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Matthew wrote:

 One thing: you musn't have seen much production code if that's the worst
you've seen. I mean, sure, it's pretty 
 yucksome, but I've seen **far** worse in my travels!! ;)

It isn't... (unfortunately). I must even confess to writing uglier code myself. It's amazing what stress and lack of sleep can do :-P However, some things about it are a little discerting:
  * ExeModule functions

Why are all the routines prefixed with ExeModule_ ? Wouldn't the module take care of that mangling... ?
 public alias int                    boolean;

I know you have your reasons for hating the "bit" type (don't we all), but wouldn't it be better to use bool ?
 /* These are "forward declared" here because I don't like the way D forces me
  * to provide my declaration and implementation together, and mixed in with all
  * the other implementation gunk.
  */

I thought that D listed that as a "feature" ? (avoid headers) Anyway, it also seems a bit like a rebel base set up there... --anders
Mar 29 2005
parent reply "Matthew" <admin.hat stlsoft.dot.org> writes:
"Anders F Björklund" <afb algonet.se> wrote in message
news:d2b7m5$11kq$1 digitaldaemon.com...
 Matthew wrote:

 One thing: you musn't have seen much production code if that's the worst
you've seen. I mean, sure, it's pretty 
 yucksome, but I've seen **far** worse in my travels!! ;)

It isn't... (unfortunately). I must even confess to writing uglier code myself. It's amazing what stress and lack of sleep can do :-P

:-)
 However, some things about it are a little discerting:

  * ExeModule functions

Why are all the routines prefixed with ExeModule_ ? Wouldn't the module take care of that mangling... ?

That was somewhat of a learning experience. I'd do it differently now. (See below)
 public alias int                    boolean;

I know you have your reasons for hating the "bit" type (don't we all), but wouldn't it be better to use bool ?

This went out the window, in my version, long ago. The problem is that at the same time I'd started to build up a framework of exception classes, and submitted that to Walter contemporaneously. Alas, that wasn't taken up, so the updates to std.loader weren't either. The lack of progress on this is partly my fault, however, since I undertook to do an exceptions review some time ago, and that went by the by. But notwithstanding that, the difficuly in getting updates to one's code that's already in Phobos is extremely inhibitive, as witnessed by the problems with recls, and the cackiness of std.loader/mmfile.
 /* These are "forward declared" here because I don't like the way D forces me
  * to provide my declaration and implementation together, and mixed in with all
  * the other implementation gunk.
  */

I thought that D listed that as a "feature" ? (avoid headers) Anyway, it also seems a bit like a rebel base set up there...

Can't even remember anything about this one. I'd like to totally rewrite std.loader (and some others), but haven't the time now, and don't have the inclination with the current setup. I reckon it's time to think about get a Phobos steering group and maybe Phobos.sourceforge.net.
Mar 29 2005
parent J C Calvarese <jcc7 cox.net> writes:
Matthew wrote:
 "Anders F Björklund" <afb algonet.se> wrote in message
news:d2b7m5$11kq$1 digitaldaemon.com...

 I'd like to totally rewrite std.loader (and some others), but haven't the time
now, and don't have the inclination with 
 the current setup. I reckon it's time to think about get a Phobos steering
group and maybe Phobos.sourceforge.net.

A r e s i s t h e a n s w e r http://www.dsource.org/forums/viewforum.php?f=31 -- jcc7 http://jcc_7.tripod.com/d/
Mar 29 2005