www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Shared library support: Where we are at

reply Rikki Cattermole <alphaglosined gmail.com> writes:
Hello!

Currently, I'm doing a push towards getting shared library 
support in D into a much more usable state.

My approach has been this:

Build -betterC shared library, build full D executable that 
depends upon said shared library. Regardless of how you feel 
about -betterC, the reason it is in use here is to try and 
prevent issues surrounding druntime during my initial push.

I have not tested on Posix or used GDC. Right now I am only 
interested in Windows support as it has the most 
platform-specific issues and I am entirely ignoring Optlink (as 
far as I'm concerned that is WONTFIX).

Both ldc and dmd can work in this arrangement. However in both 
cases you need to manually generate ModuleInfo stubs. Like this:

```d
export extern(C) void _D6dynlib3app12__ModuleInfoZ() {
     asm {
         naked;
         di 0;
         di 0x1000;
         db 0;
     }
}
```

Both LDC and DMD do not have a way to configure ModuleInfo 
generation. GDC does have a way to turn it on and off. LDC bug 
report [0].

If you remove ``-betterC``, that stub will no longer work in dmd 
due to duplicate symbols (that wasn't exported). LDC will compile 
okay.

In my codebase I have found that ``--fvisibility=public`` is a 
very useful crutch given export is a linker directive that is 
pretending to be a visibility modifier (however let's ignore that 
since that is a DIP).

As part of my codebase I'm working towards getting dub working. 
I'm currently waiting on a PR that is blocked by a vibe.d 
regression with openssl[1]. This should mean build artifacts from 
dependencies will be copied to the target directory of a project 
and DLLs won't be linked against but instead will use their 
import libraries.

I've cherry-picked a few issues surrounding this [2], [3], [4].

However here are the things that if fixed, should lead to this 
working:

- In both LDC and DMD, implement switches to turn on and off 
ModuleInfo generation (ignoring -betterC switch).
- In DMD export ModuleInfo.
- In DMD implement a switch to turn on export for all symbols.

So to cap this off: right now DMD is basically not even an option 
for shared libraries unless you use -betterC or treat it as if it 
was a non-D library. LDC is possible for non ``-betterC``.

[0] https://github.com/ldc-developers/ldc/issues/3996
[1] https://github.com/dlang/dub/pull/2259
[2] https://issues.dlang.org/show_bug.cgi?id=22367
[3] https://issues.dlang.org/show_bug.cgi?id=9816
[4] https://issues.dlang.org/show_bug.cgi?id=4071
Jun 07 2022
next sibling parent Adam Ruppe <destructionator gmail.com> writes:
On Tuesday, 7 June 2022 at 19:07:53 UTC, Rikki Cattermole wrote:

 Currently, I'm doing a push towards getting shared library 
 support in D into a much more usable state.


shared libs in D have been fully operational for over a year now 
with gdc and almost a year with ldc. You can make the 
druntime/phobos dll and all share it. Even on dmd, there's 
separate druntimes but they link some things together, enough to 
kinda work, but the ldc+gdc experience is more reliable (dmd 
botches typeinfo across the dll boundary, so you're best off 
using it COM style).

For the plugin model, probably best to just build with the ldc.
Jun 07 2022
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 7 June 2022 at 19:07:53 UTC, Rikki Cattermole wrote:

 Hello!

 Currently, I'm doing a push towards getting shared library 
 support in D into a much more usable state.


Thanks for working on this.
Jun 07 2022
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
While fixing one of the bugs I have discovered that when using MSVC 
linker, shared libraries that have symbols that contain Unicode is 
currently broken.

It is not on our end. This is a hard blocker for our non-english 
speaking users.

It will mean that we have to do something to sanitize those symbol names 
to make them ASCII in nature.

Unfortunately we had an incomplete test case in the testsuite that only 
covered executables.

https://issues.dlang.org/show_bug.cgi?id=23179
Jun 12 2022
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Sunday, 12 June 2022 at 13:52:17 UTC, rikki cattermole wrote:

 While fixing one of the bugs I have discovered that when using 
 MSVC linker, shared libraries that have symbols that contain 
 Unicode is currently broken.


im p sure that's part of the mangling scheme, prolly missed a 
spot there.
Jun 12 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 13/06/2022 1:56 AM, Adam D Ruppe wrote:

 On Sunday, 12 June 2022 at 13:52:17 UTC, rikki cattermole wrote:

 While fixing one of the bugs I have discovered that when using MSVC 
 linker, shared libraries that have symbols that contain Unicode is 
 currently broken.


 
 im p sure that's part of the mangling scheme, prolly missed a spot there.


I didn't.

Its a bug that LDC has also encountered in MSVC linker (which they 
solved by just disabling the test like I did).
Jun 12 2022
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 13/06/2022 1:52 AM, rikki cattermole wrote:

 While fixing one of the bugs I have discovered that when using MSVC 
 linker, shared libraries that have symbols that contain Unicode is 
 currently broken.
 
 It is not on our end. This is a hard blocker for our non-english 
 speaking users.
 
 It will mean that we have to do something to sanitize those symbol names 
 to make them ASCII in nature.
 
 Unfortunately we had an incomplete test case in the testsuite that only 
 covered executables.
 
 https://issues.dlang.org/show_bug.cgi?id=23179


I did manage to create a fix for this bug, but because of how it was 
discovered we are favoring a wait and see approach to see if it appears 
in the wild.

So if you experience this, please update the bug report!
Jun 13 2022
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
Wahoooooo joy to the world we have another bug report that could be 
closed by getting exportation of ModuleInfo fixed.

https://issues.dlang.org/show_bug.cgi?id=6019

Current running total is four if this one issue were to be solved 
satisfactorily.

https://issues.dlang.org/show_bug.cgi?id=22367
https://issues.dlang.org/show_bug.cgi?id=9816
https://issues.dlang.org/show_bug.cgi?id=4071

This is a set of very high value set of bug reports that could be fixed 
very easily if someone with knowledge of dmd compiler internals wanted 
to fix it.

See: https://forum.dlang.org/post/ohflhdaggtapeqpynkjd forum.dlang.org

Latest attempt of mine to fix this: https://github.com/dlang/dmd/pull/14647
Nov 25 2022