www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Status of Dynamically Loadable D Libraries

reply "Nick Ulle" <nim.junk gmail.com> writes:
What's the current status of dynamic loading (and/or linking) for 
D libraries?

I've been playing with this a bit in hopes of calling a D shared 
library from R through the C ABI, and noticed that on Windows it 
seems fine, but on Linux dmd won't build my library unless it 
includes main(), and it segfaults if I call writeln().

I've seen this thread:
http://forum.dlang.org/thread/hmhaldyfziejrplgzazt forum.dlang.org

And also this one:
http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com

What I gather is that right now, DLLs work on Windows, but on 
*nix, shared libraries cannot yet use anything from Phobos.

Am I correct that it's safe to dynamically load a D library on 
Windows? How out-of-date is the information in the tutorial for 
making DLLs on dlang?

Is supporting dynamically loadable libraries a priority for the 
developers, or something that might not happen for a while?
Feb 10 2013
next sibling parent reply "jerro" <a a.com> writes:
 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while?


Judging by this thread:

http://forum.dlang.org/thread/kf1223$oo8$1 digitalmars.com

It has just become a priority.
Feb 10 2013
parent "Nick Ulle" <nim.junk gmail.com> writes:
On Sunday, 10 February 2013 at 22:46:29 UTC, jerro wrote:

 Judging by this thread:

 http://forum.dlang.org/thread/kf1223$oo8$1 digitalmars.com

 It has just become a priority.


Thanks! This is great to hear. I didn't come across that thread 
while searching the forums before.
Feb 10 2013
prev sibling next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 10 February 2013 at 22:11:52 UTC, Nick Ulle wrote:

 What's the current status of dynamic loading (and/or linking) 
 for D libraries?

 I've been playing with this a bit in hopes of calling a D 
 shared library from R through the C ABI, and noticed that on 
 Windows it seems fine, but on Linux dmd won't build my library 
 unless it includes main(), and it segfaults if I call writeln().

 I've seen this thread:
 http://forum.dlang.org/thread/hmhaldyfziejrplgzazt forum.dlang.org

 And also this one:
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com

 What I gather is that right now, DLLs work on Windows, but on 
 *nix, shared libraries cannot yet use anything from Phobos.
 
 Am I correct that it's safe to dynamically load a D library on 
 Windows? How out-of-date is the information in the tutorial for 
 making DLLs on dlang?

 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while


For shared libraries (linux) I'd use ldc . It's just works.
Feb 10 2013
next sibling parent "Nick Ulle" <nim.junk gmail.com> writes:
On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:

 For shared libraries (linux) I'd use ldc . It's just works.


Not quite. Using
   ldc2 -shared -relocation-model=pic mylib.d
Compiles and links, but on loading there are undefined symbols 
(_Dmain). Including a main() function gets me a loadable library, 
but much like the case with dmd, it segfaults if I call writeln.
Feb 10 2013
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:

 For shared libraries (linux) I'd use ldc . It's just works.


You are living a dangerous life. ;)

It appears to work at first, but the test suite is known to 
segfault in random places with druntime/Phobos being built as a 
shared library, most probably because the GC/threading code 
doesn't really handle dynamic libraries at all.

David
Feb 10 2013
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 10 February 2013 at 23:52:30 UTC, David Nadlinger 
wrote:

 On Sunday, 10 February 2013 at 23:41:54 UTC, John Colvin wrote:

 For shared libraries (linux) I'd use ldc . It's just works.


 You are living a dangerous life. ;)

 It appears to work at first, but the test suite is known to 
 segfault in random places with druntime/Phobos being built as a 
 shared library, most probably because the GC/threading code 
 doesn't really handle dynamic libraries at all.

 David


Does this pretty much completely invalidate projects like this: 
https://bitbucket.org/ariovistus/pyd ?

I've had success using it, as well as loading d shared libraries 
from C/IDL, with the appropriate rt_init calls etc.
Feb 13 2013
prev sibling next sibling parent "Dave Wilson" <boff1984 hotmail.com> writes:
You can arrange to have d's runtime initialized when dlopen loads
the shared lib by adding a small shim that uses gcc attributes,
as follows:

// shim.c

__attribute__((constructor))
static void dinit() {
      rt_init();
}
__attribute__((destructor))
static void dfini() {
      rt_term();
}

// end of shim.c

Compile this to a .o with gcc, then include the .o in the build.
Note that I've only tried this trick with a simple "hello world"
shared lib, so I apologize in advance if it doesn't work for
real-world uses.
Feb 10 2013
prev sibling next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Sunday, 10 February 2013 at 22:11:52 UTC, Nick Ulle wrote:

 What's the current status of dynamic loading (and/or linking) 
 for D libraries?

 I've been playing with this a bit in hopes of calling a D 
 shared library from R through the C ABI, and noticed that on 
 Windows it seems fine, but on Linux dmd won't build my library 
 unless it includes main(), and it segfaults if I call writeln().


Dynamic linking works in linux (at least observed features work, 
see example 
http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2). 
Dynamic loading does not work.


 Is supporting dynamically loadable libraries a priority for the 
 developers, or something that might not happen for a while?


Yes, couple of days ago Walter announced that shared libraries is 
priority and he will get into this after releasing new dmd 
version.
Feb 10 2013
parent reply Lee Braiden <leebraid gmail.com> writes:
On Mon, 11 Feb 2013 07:25:55 +0100, Maxim Fomin wrote:

 Dynamic linking works in linux (at least observed features work, see
 example
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2).
 Dynamic loading does not work.
 

 [snip]


 Yes, couple of days ago Walter announced that shared libraries is
 priority and he will get into this after releasing new dmd version.


I'm not sure what to hope for here.

Does this mean we'll be able to, say, scan a directory, and dynamically 
load D plugins from it, with proper exception handling, threading, etc., 
across the app/library/plugin divide?

And will it work the same way on different platforms, so you can build 
the (right binaries, obviously) for the platform, and load it the same 
way, with the underlying code taking care of the details?


-- 
Lee
Feb 23 2013
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/24/13 3:53 AM, Lee Braiden wrote:

 On Mon, 11 Feb 2013 07:25:55 +0100, Maxim Fomin wrote:

 Dynamic linking works in linux (at least observed features work, see
 example
 http://forum.dlang.org/thread/k3vfm9$1tq$1 digitalmars.com?page=2).
 Dynamic loading does not work.


 [snip]


 Yes, couple of days ago Walter announced that shared libraries is
 priority and he will get into this after releasing new dmd version.


 I'm not sure what to hope for here.

 Does this mean we'll be able to, say, scan a directory, and dynamically
 load D plugins from it, with proper exception handling, threading, etc.,
 across the app/library/plugin divide?


That's about right.


 And will it work the same way on different platforms, so you can build
 the (right binaries, obviously) for the platform, and load it the same
 way, with the underlying code taking care of the details?


Linux first.


Andrei
Feb 23 2013
prev sibling parent Martin Nowak <code dawg.eu> writes:
On 02/10/2013 11:11 PM, Nick Ulle wrote:

 What's the current status of dynamic loading (and/or linking) for D
 libraries?


No dynamic loading in sight yet. We need to extend the GC, find 
semantics/techniques for TLS initialization, provide tools to load 
symbols ...
What is on the way but will take some more weeks is link-time support 
for shared libraries.
https://github.com/D-Programming-Language/druntime/pull/395


 I've been playing with this a bit in hopes of calling a D shared library
 from R through the C ABI, and noticed that on Windows it seems fine, but
 on Linux dmd won't build my library unless it includes main(), and it
 segfaults if I call writeln().


The reasons for your crashes are missing initialization.
You could experiment with calling rt_init/rt_term which are exported 
from your shared library, but only if you're not threaded.


 What I gather is that right now, DLLs work on Windows, but on *nix,
 shared libraries cannot yet use anything from Phobos.


Yes, there is rudimentary support on Windows, but you'll inevitably run 
into ODR issues.
http://d.puremagic.com/issues/show_bug.cgi?id=7020#c4


 Am I correct that it's safe to dynamically load a D library on Windows?
 How out-of-date is the information in the tutorial for making DLLs on
 dlang?


AFAIK it should work.


 Is supporting dynamically loadable libraries a priority for the
 developers, or something that might not happen for a while?


It's a huge priority but also a huge undertaking.
Feb 13 2013