digitalmars.D.learn - Using .lib and .dll in D applications
- moe (82/82) Jun 19 2016 Hello,
- Mike Parker (55/91) Jun 19 2016 This is a compile time error, not a link time error. Imports
- moe (60/156) Jun 19 2016 Thanks for the reply.
- moe (6/14) Jun 19 2016 Darn, I can't seam to edit the post. The line:
- Mike Parker (24/56) Jun 19 2016 You can't do it with only the lib file. You *need* the source
- moe (14/76) Jun 19 2016 I see where I went wrong. I thought that it's possible to only
- docandrew (24/121) Jun 19 2016 Sure, you can just have a file with "extern" functions that tells
- captaindet (4/10) Jun 19 2016 for this purpose there are .di interface files that can be generated
- Mike Parker (30/42) Jun 19 2016 DLLs also have nothing to do with compilation. That's a runtime
- moe (28/28) Jun 20 2016 Thanks everyone for the info. It is very much appreciated!
- Mike Parker (88/113) Jun 20 2016 The problem is that shared library (DLL) support in D is not at a
- Mike Parker (11/18) Jun 20 2016 Sorry, I forgot a couple of commments. I did explain it in the
- moe (6/24) Jun 20 2016 Wow, absolutely awesome! That's exactly what I have been looking
- moe (199/199) Jun 21 2016 I had some time to try it out and I finally got it to work. I
- Mike Parker (12/24) Jun 21 2016 Right, I forgot about that. It's always optional in C and C++,
- moe (12/36) Jun 21 2016 Yes, I did it intentionally. I wanted to ensure that the packages
- Mike Parker (7/18) Jun 21 2016 Understood. FYI, you can specify a specific version and dub
- moe (13/35) Jun 21 2016 I meant like this:
- Mike Parker (7/19) Jun 21 2016 Just to be clear, are you compiling iplugin.d as well? I assumed
- moe (37/59) Jun 22 2016 I have added all of these to the dub.json:
Hello,
I am new to d and doing some small test apps at the moment to
learn d. Currently I must be missing something basic here. I have
installed dub as a project manager and I am trying to use a .lib
file in an app. However, I can not use a module from the .lib
file. I get the following error:
"module barlib is in file 'barlib.d' which cannot be read"
here is what I currently have:
the file to build the library
-----------------------------
module barlib;
import std.stdio;
class BarLib
{
this(){}
void Talk(string msg)
{
writeln("BL: %s", msg);
}
}
the dub.json to build the lib
-----------------------------
{
"name": "dbar",
"authors": ["moe"],
"description": "A minimal D application.",
"copyright": "Copyright © 2016, root",
"license": "proprietary",
"platforms": ["windows"],
"versions": ["DesktopApp"],
"targetType": "staticLibrary",
"configurations": [
{
"name": "debug",
"targetPath": "bin/debug",
"buildOptions": ["debugMode", "debugInfo"]
},
{
"name": "release",
"targetPath": "bin/release",
"buildOptions": ["releaseMode", "optimize", "inline"]
}
]
}
the file for the app
--------------------
import barlib;
import std.stdio;
void main()
{
auto b = new BarLib();
b.Talk("Hello from barlib");
}
the dub.json for the app
------------------------
{
"name": "dfoo",
"authors": ["moe"],
"description": "A minimal D application.",
"copyright": "Copyright © 2016, root",
"license": "proprietary",
"platforms": ["windows"],
"versions": ["DesktopApp"],
"targetType": "executable",
"configurations": [
{
"name": "debug",
"targetPath": "bin/debug",
"buildOptions": ["debugMode", "debugInfo"]
},
{
"name": "release",
"targetPath": "bin/release",
"buildOptions": ["releaseMode", "optimize", "inline"]
}
]
}
I can successfully build the barlib.lib file but I can not use it
in the app. I thought that it should be possible to simply import
the module from the lib-file, but the app cannot find the module.
I would like to be able to build .lib and .dll files and use them
in other applications. Can someone tell me what I am missing?
Jun 19 2016
On Sunday, 19 June 2016 at 15:35:04 UTC, moe wrote:I am new to d and doing some small test apps at the moment to learn d. Currently I must be missing something basic here. I have installed dub as a project manager and I am trying to use a .lib file in an app. However, I can not use a module from the .lib file. I get the following error: "module barlib is in file 'barlib.d' which cannot be read"This is a compile time error, not a link time error. Imports modules and linking to libraries are two different things.the dub.json for the app ------------------------ { "name": "dfoo", "authors": ["moe"], "description": "A minimal D application.", "copyright": "Copyright © 2016, root", "license": "proprietary", "platforms": ["windows"], "versions": ["DesktopApp"], "targetType": "executable", "configurations": [ { "name": "debug", "targetPath": "bin/debug", "buildOptions": ["debugMode", "debugInfo"] }, { "name": "release", "targetPath": "bin/release", "buildOptions": ["releaseMode", "optimize", "inline"] } ] }Nowhere have you set up barlib as a dependency for your app, so DUB has no idea it's supposed to tell dmd that you need to link to the library or where to find the modules. So the easy thing to do is to add a dependency directive. Since your library isn't registered with the dub registry, you'll need to provide a path to the barlib dub project rather than a release tag for the library. Assuming a directory tree like so: -projects --barlib ---dub.json --dfoo ---dub.json Then the path for dfoo's depencency is ../barlib. With a dub.sdl file, it would look like this: dependency "barlib" path="../barlib" You can find the equivalent JSON syntax somewhere at [1]. With this, dub will manage your import path and library dependency for you.I can successfully build the barlib.lib file but I can not use it in the app. I thought that it should be possible to simply import the module from the lib-file, but the app cannot find the module. I would like to be able to build .lib and .dll files and use them in other applications. Can someone tell me what I am missing?Compiling and linking are two distinct steps. The compiler passes any libraries you give it to the linker, but it does not do anything with them itself during the compile step. At that stage, it's only concerned with source modules. So you need to tell it: where to find the source modules (on the command line), which ones to import (with the import statement). It also needs to know which libraries to pass to the linker and where they can be found. Give the directory tree above, when compiling dfoo manually, you might do this on Windows: cd dfoo dmd -I../barlib/source source/app.d ../barlib/bin/barlib.lib On other systems: dmd -I../barlib/source -L-L../barlib/bin -L-lbarlib source/app.d The -I switch tells the compiler where it can find the modules you need to import. DRuntime and Phobos are configured in DMD's config files, so you don't need to every specify those. On the windows command line, you can pass the full path to the library directly and the compiler will do the right thing. This is easier than dealing with the different syntax the two supported Windows linkers use for command line switches. On Posix systems, it's better to use the -L switch, which tell the compiler the immediately following command should be passed to the system linker. Ex: -L-L/some/path -> passes -L/some/path to the linker, which is the switch telling it where to look for libraries. -L-lFoo (-L-l -- the second letter is a lowercase 'L') -> passes -lname to the linker, telling it which library to link with. Given this, the linker will look for /some/path/libFoo.so or /some/path/libFoo.a, in addition to searching the normal search paths. As you can see, it's easier to properly configure your dub.json dependecies so that DUB can handle all of this for you. [1] http://code.dlang.org/package-format?lang=json
Jun 19 2016
On Sunday, 19 June 2016 at 16:31:40 UTC, Mike Parker wrote:On Sunday, 19 June 2016 at 15:35:04 UTC, moe wrote:Thanks for the reply. Unfortunatelly I still don't get it. I would like to have an independant project "dbar". The created lib is then used in another project "dfoo". Assuming that "dfoo" has no access to "dbar" other than the .lib file. My folder structure is like this: -dtest --dbar ----source\barlib.d ----dub.json This project creates a dbar.lib file which seams to work. -dtest --dfoo ----lib\dbar.d // copied from the dbar project ----source\app.d ----dub.json This project would use the dbar.lib but should otherwise not have access to the dbar project. Basically simulating, that someone else made a dbar project to which I would not have access other than using the dbar.lib. How do I have to configure the dub.json file for this to work? I have tried a variety of configurations for the dub.json. At this point it feels like a bad guessing game. That is no way to deveop anything. I need to figure out how to properly setup the dub.json but I don't seam to find the answer online. "http://code.dlang.org/package-format?lang=json" isn't very helpful. I have meanwhile adjusted my dtest/dfoo/dub.json to this: { "name": "dfoo", "authors": ["moe"], "description": "A minimal D application.", "copyright": "Copyright © 2016, root", "license": "proprietary", "platforms": ["windows"], "versions": ["DesktopApp"], "targetType": "executable", "sourcePaths":["lib"], "libs": ["dbar"], "copyFiles": ["lib"], "dependencies": { "dbar": "~master" }, "configurations": [ { "name": "debug", "targetPath": "bin/debug", "buildOptions": ["debugMode", "debugInfo"], }, { "name": "release", "targetPath": "bin/release", "buildOptions": ["releaseMode", "optimize", "inline"], } ] } This gives me the error: "Root package dfoo references unknown package dbar" Removing the dependencies doesn't work either.I am new to d and doing some small test apps at the moment to learn d. Currently I must be missing something basic here. I have installed dub as a project manager and I am trying to use a .lib file in an app. However, I can not use a module from the .lib file. I get the following error: "module barlib is in file 'barlib.d' which cannot be read"This is a compile time error, not a link time error. Imports modules and linking to libraries are two different things.the dub.json for the app ------------------------ { "name": "dfoo", "authors": ["moe"], "description": "A minimal D application.", "copyright": "Copyright © 2016, root", "license": "proprietary", "platforms": ["windows"], "versions": ["DesktopApp"], "targetType": "executable", "configurations": [ { "name": "debug", "targetPath": "bin/debug", "buildOptions": ["debugMode", "debugInfo"] }, { "name": "release", "targetPath": "bin/release", "buildOptions": ["releaseMode", "optimize", "inline"] } ] }Nowhere have you set up barlib as a dependency for your app, so DUB has no idea it's supposed to tell dmd that you need to link to the library or where to find the modules. So the easy thing to do is to add a dependency directive. Since your library isn't registered with the dub registry, you'll need to provide a path to the barlib dub project rather than a release tag for the library. Assuming a directory tree like so: -projects --barlib ---dub.json --dfoo ---dub.json Then the path for dfoo's depencency is ../barlib. With a dub.sdl file, it would look like this: dependency "barlib" path="../barlib" You can find the equivalent JSON syntax somewhere at [1]. With this, dub will manage your import path and library dependency for you.I can successfully build the barlib.lib file but I can not use it in the app. I thought that it should be possible to simply import the module from the lib-file, but the app cannot find the module. I would like to be able to build .lib and .dll files and use them in other applications. Can someone tell me what I am missing?Compiling and linking are two distinct steps. The compiler passes any libraries you give it to the linker, but it does not do anything with them itself during the compile step. At that stage, it's only concerned with source modules. So you need to tell it: where to find the source modules (on the command line), which ones to import (with the import statement). It also needs to know which libraries to pass to the linker and where they can be found. Give the directory tree above, when compiling dfoo manually, you might do this on Windows: cd dfoo dmd -I../barlib/source source/app.d ../barlib/bin/barlib.lib On other systems: dmd -I../barlib/source -L-L../barlib/bin -L-lbarlib source/app.d The -I switch tells the compiler where it can find the modules you need to import. DRuntime and Phobos are configured in DMD's config files, so you don't need to every specify those. On the windows command line, you can pass the full path to the library directly and the compiler will do the right thing. This is easier than dealing with the different syntax the two supported Windows linkers use for command line switches. On Posix systems, it's better to use the -L switch, which tell the compiler the immediately following command should be passed to the system linker. Ex: -L-L/some/path -> passes -L/some/path to the linker, which is the switch telling it where to look for libraries. -L-lFoo (-L-l -- the second letter is a lowercase 'L') -> passes -lname to the linker, telling it which library to link with. Given this, the linker will look for /some/path/libFoo.so or /some/path/libFoo.a, in addition to searching the normal search paths. As you can see, it's easier to properly configure your dub.json dependecies so that DUB can handle all of this for you. [1] http://code.dlang.org/package-format?lang=json
Jun 19 2016
moe <none available.com> On Sunday, 19 June 2016 at 17:33:43 UTC, moe wrote:On Sunday, 19 June 2016 at 16:31:40 UTC, Mike Parker wrote:Darn, I can't seam to edit the post. The line: "----lib\dbar.d // copied from the dbar project" should of course read: ----lib\dbar.lib // copied from the dbar project I am copying the lib file. Not the .d file.[...]Thanks for the reply. Unfortunatelly I still don't get it. I would like to have an independant project "dbar". The created lib is then used in another project "dfoo". Assuming that "dfoo" has no access to "dbar" other than the .lib file. [...]
Jun 19 2016
On Sunday, 19 June 2016 at 17:33:43 UTC, moe wrote:Unfortunatelly I still don't get it. I would like to have an independant project "dbar". The created lib is then used in another project "dfoo". Assuming that "dfoo" has no access to "dbar" other than the .lib file.You can't do it with only the lib file. You *need* the source file too for the import statement. As I explained, the lib file is used by the linker, not the compiler. The compiler needs the source file.My folder structure is like this: -dtest --dbar ----source\barlib.d ----dub.json This project creates a dbar.lib file which seams to work. -dtest --dfoo ----lib\dbar.d // copied from the dbar project ----source\app.d ----dub.jsonYou don't need to copy dbar to the lib directory in this case.This project would use the dbar.lib but should otherwise not have access to the dbar project. Basically simulating, that someone else made a dbar project to which I would not have access other than using the dbar.lib. How do I have to configure the dub.json file for this to work?One of two things would happen: 1) They would register the project with he dub registry, then you add a dependency to a specific version the library. Dub would then download the necessary files for you and ensure that everything you need is passed to the compiler when building your project. 2) They would provide some other means for you to get the source and the library. Then you would need to manually configure your dub.json to pass the import path to the compiler and link with the library.I have tried a variety of configurations for the dub.json. At this point it feels like a bad guessing game. That is no way to deveop anything. I need to figure out how to properly setup the dub.json but I don't seam to find the answer online. "http://code.dlang.org/package-format?lang=json" isn't very helpful.All the information you need is there on that page.I have meanwhile adjusted my dtest/dfoo/dub.json to this:"dependencies": { "dbar": "~master" }This gives me the error: "Root package dfoo references unknown package dear"As I explained above, you need a path attribute for the dependency in this case since it is on your local file system and not in the registry. The documentation link I gave you explains how to to this. Try this: "dependencies": { "dbar": {"path": "../dbar"} }
Jun 19 2016
On Sunday, 19 June 2016 at 18:00:07 UTC, Mike Parker wrote:On Sunday, 19 June 2016 at 17:33:43 UTC, moe wrote:I see where I went wrong. I thought that it's possible to only use the .lib file without the source code of dbar. Having access to the source makes what I am trying somewhat pointless. Is it otherwise possible to provide some functionality without having to give away your source? I would like to put together a library that I can reuse, without having to rely on the source each time. Maybe a dll instead? Note: I don't have a problem with giving away my code. I just want to know if it can be done. Or maybe later build a plugin system where the creator of the plugin does not need the source of the entire application. And in turn the app does not need to be recompiled in order to use the plugin. Thanks for your help!Unfortunatelly I still don't get it. I would like to have an independant project "dbar". The created lib is then used in another project "dfoo". Assuming that "dfoo" has no access to "dbar" other than the .lib file.You can't do it with only the lib file. You *need* the source file too for the import statement. As I explained, the lib file is used by the linker, not the compiler. The compiler needs the source file.My folder structure is like this: -dtest --dbar ----source\barlib.d ----dub.json This project creates a dbar.lib file which seams to work. -dtest --dfoo ----lib\dbar.d // copied from the dbar project ----source\app.d ----dub.jsonYou don't need to copy dbar to the lib directory in this case.This project would use the dbar.lib but should otherwise not have access to the dbar project. Basically simulating, that someone else made a dbar project to which I would not have access other than using the dbar.lib. How do I have to configure the dub.json file for this to work?One of two things would happen: 1) They would register the project with he dub registry, then you add a dependency to a specific version the library. Dub would then download the necessary files for you and ensure that everything you need is passed to the compiler when building your project. 2) They would provide some other means for you to get the source and the library. Then you would need to manually configure your dub.json to pass the import path to the compiler and link with the library.I have tried a variety of configurations for the dub.json. At this point it feels like a bad guessing game. That is no way to deveop anything. I need to figure out how to properly setup the dub.json but I don't seam to find the answer online. "http://code.dlang.org/package-format?lang=json" isn't very helpful.All the information you need is there on that page.I have meanwhile adjusted my dtest/dfoo/dub.json to this:"dependencies": { "dbar": "~master" }This gives me the error: "Root package dfoo references unknown package dear"As I explained above, you need a path attribute for the dependency in this case since it is on your local file system and not in the registry. The documentation link I gave you explains how to to this. Try this: "dependencies": { "dbar": {"path": "../dbar"} }
Jun 19 2016
On Sunday, 19 June 2016 at 18:33:36 UTC, moe wrote:On Sunday, 19 June 2016 at 18:00:07 UTC, Mike Parker wrote:Sure, you can just have a file with "extern" functions that tells the compiler that the source for those functions isn't available at compile time, but that later on when you link the library, the linker will find those function definitions in the .lib file (I'm not a Windows expert, but from my understanding .DLLs are a different ball game, and have to be loaded and called with special functions in your app.) In your example, in your "dfoo.d", you can declare the functions from the dbar.lib inside as "extern myfunction();" and the compiler will say, "OK, myfunction() isn't defined here, so we'll let the linker sort it out." If you try and compile without telling it about dbar.lib, you'll get errors like "undefined reference to myfunction()...". When you see that somebody wrote a "binding" in D for a library like OpenGL, etc., this is what they are providing, a .d file filled with "extern" definitions and maybe some glue code to convert D types into C types if the .lib file is a C library. APIs generally work the same way. Library writers will provide a .h header file that doesn't expose any of their closed source, but gives users of the library the definitions of the functions they need to use it's functions. I hope this helps! -JonOn Sunday, 19 June 2016 at 17:33:43 UTC, moe wrote:I see where I went wrong. I thought that it's possible to only use the .lib file without the source code of dbar. Having access to the source makes what I am trying somewhat pointless. Is it otherwise possible to provide some functionality without having to give away your source? I would like to put together a library that I can reuse, without having to rely on the source each time. Maybe a dll instead? Note: I don't have a problem with giving away my code. I just want to know if it can be done. Or maybe later build a plugin system where the creator of the plugin does not need the source of the entire application. And in turn the app does not need to be recompiled in order to use the plugin. Thanks for your help!Unfortunatelly I still don't get it. I would like to have an independant project "dbar". The created lib is then used in another project "dfoo". Assuming that "dfoo" has no access to "dbar" other than the .lib file.You can't do it with only the lib file. You *need* the source file too for the import statement. As I explained, the lib file is used by the linker, not the compiler. The compiler needs the source file.My folder structure is like this: -dtest --dbar ----source\barlib.d ----dub.json This project creates a dbar.lib file which seams to work. -dtest --dfoo ----lib\dbar.d // copied from the dbar project ----source\app.d ----dub.jsonYou don't need to copy dbar to the lib directory in this case.This project would use the dbar.lib but should otherwise not have access to the dbar project. Basically simulating, that someone else made a dbar project to which I would not have access other than using the dbar.lib. How do I have to configure the dub.json file for this to work?One of two things would happen: 1) They would register the project with he dub registry, then you add a dependency to a specific version the library. Dub would then download the necessary files for you and ensure that everything you need is passed to the compiler when building your project. 2) They would provide some other means for you to get the source and the library. Then you would need to manually configure your dub.json to pass the import path to the compiler and link with the library.I have tried a variety of configurations for the dub.json. At this point it feels like a bad guessing game. That is no way to deveop anything. I need to figure out how to properly setup the dub.json but I don't seam to find the answer online. "http://code.dlang.org/package-format?lang=json" isn't very helpful.All the information you need is there on that page.I have meanwhile adjusted my dtest/dfoo/dub.json to this:"dependencies": { "dbar": "~master" }This gives me the error: "Root package dfoo references unknown package dear"As I explained above, you need a path attribute for the dependency in this case since it is on your local file system and not in the registry. The documentation link I gave you explains how to to this. Try this: "dependencies": { "dbar": {"path": "../dbar"} }
Jun 19 2016
On 2016-06-20 06:33, moe wrote:I see where I went wrong. I thought that it's possible to only use the .lib file without the source code of dbar. Having access to the source makes what I am trying somewhat pointless. Is it otherwise possible to provide some functionality without having to give away your source? I would like to put together a library that I can reuse, without having to rely on the source each time. Maybe a dll instead?for this purpose there are .di interface files that can be generated automatically: https://dlang.org/dmd-windows.html#interface-files
Jun 19 2016
On Sunday, 19 June 2016 at 18:33:36 UTC, moe wrote:I see where I went wrong. I thought that it's possible to only use the .lib file without the source code of dbar. Having access to the source makes what I am trying somewhat pointless. Is it otherwise possible to provide some functionality without having to give away your source? I would like to put together a library that I can reuse, without having to rely on the source each time. Maybe a dll instead? Note: I don't have a problem with giving away my code. I just want to know if it can be done. Or maybe later build a plugin system where the creator of the plugin does not need the source of the entire application. And in turn the app does not need to be recompiled in order to use the plugin.DLLs also have nothing to do with compilation. That's a runtime thing. D is not like Java, where everything you need is bundled in an archive and can be used at both compile time and runtime. It's more like C and C++, where there are very distinct formats in use and compile time and runtime. The compiler needs to know which symbols are available for you to use in your module. It can only get them from the source for the modules you import, not from any static libraries or DLLs. D does support interface files (called 'D Interface' files, with a .di extension). These are stripped down versions of your source code that can be shipped with your binary. Instead of having full function implementations, you have the declarations only. This means that none of those functions can be inlined. And you still need the full implementation of any templates in the library, otherwise the templates can't be instantiated. .di files are more like C and C++ header files. You can generate them from any D source file by passing -H on the command line during compilation. The primary benefit of .di files is to hide the source. Since you say you don't care about that, you probably shouldn't worry about them. Really, if you are using DUB to manage your projects and don't care about hiding the source, then there's nothing to worry about. You can put your libraries on github or BitBucket, register them with the DUB registry at code.dlang.org, and then everyone who uses them as dependencies will have everything they need automatically. You could also bundle them up in zip files and distribute them that way. People can run dub to compile the libraries and then configure their project locally to find what they need. It isn't a problem at all.
Jun 19 2016
Thanks everyone for the info. It is very much appreciated! What works for me right now is dealing with various packages (dub packages) and use them in projects. I previously made too many assumptions about how lib's and dll's work. It's much clearer now. Where I still have a problem is with a plugin system. I would like to write an app that is plugin based. So that I can write a plugin to extend the functionality of the app. I imagine that I could copy the plugin into a folder from the (compiled) app and the app would be able to use the plugin without me having to recompile the app. I have been looking at LoadLibrary (https://dlang.org/library/core/runtime/runtime.load_library.html) and Object.factory (https://dlang.org/library/object/object.factory.html) but I don't quite get how to use them. I did find some info that was several years old. They mentioned that these functions are not yet ready for all platforms. I don't know if that has changed in the meantime. It is not clear from the documentation. I would like to find some snippets or examples but that sort of resource for D is very hard to find online. Most of the time there are only some partial snippets where it turns out they only work under specific conditions or are restricted to one platform. Does D not supply some infrastructure for this kind of a task? It seams to me that is a very common approach to build an app. I expected there would be some out of the box solution for this in D. I would write the app and the plugins purely in D (no C bindings if possible). I have tried to build a mini app which defines an interface for the plugins. The plugins then implement that interface. But when I want to load the plugin both LoadLibrary() and Object.factory() fail (I am aware that Object.factory() requires the full qualified class names). Is there a recommended way of writing a simple plugin system that works on windows and linux? Maybe someone has a working example or knows a book that covers how to write a plugin system?
Jun 20 2016
On Monday, 20 June 2016 at 11:25:04 UTC, moe wrote:Where I still have a problem is with a plugin system. I would like to write an app that is plugin based. So that I can write a plugin to extend the functionality of the app. I imagine that I could copy the plugin into a folder from the (compiled) app and the app would be able to use the plugin without me having to recompile the app. I have been looking at LoadLibrary (https://dlang.org/library/core/runtime/runtime.load_library.html) and Object.factory (https://dlang.org/library/object/object.factory.html) but I don't quite get how to use them. I did find some info that was several years old. They mentioned that these functions are not yet ready for all platforms. I don't know if that has changed in the meantime. It is not clear from the documentation. I would like to find some snippets or examples but that sort of resource for D is very hard to find online. Most of the time there are only some partial snippets where it turns out they only work under specific conditions or are restricted to one platform. Does D not supply some infrastructure for this kind of a task? It seams to me that is a very common approach to build an app. I expected there would be some out of the box solution for this in D. I would write the app and the plugins purely in D (no C bindings if possible).The problem is that shared library (DLL) support in D is not at a level where you can load one at runtime and treat it as the rest of your D code (at least on Windows -- I'm not sure how complete support is elsewhere). Until it is, there are a number of steps you can take to work around potential problems.I have tried to build a mini app which defines an interface for the plugins. The plugins then implement that interface. But when I want to load the plugin both LoadLibrary() and Object.factory() fail (I am aware that Object.factory() requires the full qualified class names). Is there a recommended way of writing a simple plugin system that works on windows and linux? Maybe someone has a working example or knows a book that covers how to write a plugin system?You've got the right idea in defining an interface, but don't bother with Object.factory. Here's a brief rundown of how I wouldn handle it. Let's assume we'e got an interface like so: enum PluginError { none, initFileSys, initAudio, initSomethingElse } class PluginException : Exception { ... } interface Plugin { bool initialize(); void terminate(); Throwable getLastException(); SomeObject getSomeObject(); void returnSomeObject(SomeObject); } Now, I would take the following steps: * Any free functions in the D code for the DLL would be marked as extern(C). This does not make them "C functions" in that you can't use D with them. It only turns off D symbol decoration so that they look like C functions to the linker and ensures that they have the cdecl calling convention. This makes them much easier to load, as the symbol names will match the function names. * Any objects that need to be used in the application from the plugin should be allocated in the DLL. Be careful with this. You have to initialize the runtime yourself in your DLL, so you have two different GC instances running: one in the app and one in the DLL. I believe on Linux its possible to use Phobos + DRuntime as a shared library, so you have only one GC, but on Windows it is not. This means you should keep a reference to any objects you allocate in the DLL to make sure they aren't collected while the app is using them. When the app is finished with an instance, it should let the DLL know. * Don't allocate Exceptions on the DLL side for use on the app side. You can't throw them across boundaries anyway, but you also have the issue of two separate GC instances. If you return an Exception somehow (like, plugin.getLastError or something) it will probably never be collected or, if the DLL terminates before the exception propagates fully, the memory will be invalid. Hence my use of PluginError above. One possible use of the interface: ================= // In the DLL code export extern(C) nothrow Plugin getPluginImpl() { import core.memory : GC; auto plugin = new PluginImpl(); // Make sure the GC doesn't collect the instance GC.addRoot(cast(void*)plugin); return plugin; } // In the app code // You'll use this to load the plugin alias GetPluginImpl = extern(c) nothrow function(); GetPluginImpl getPluginImpl; // Using DerelictUtil [1] makes it easy. // Use derelict-util version ~>2.0.6 as a dub dependency import derelict.util.sharedlib; // Keep this around if you need to unload the library manually // at some point. It does not unload in the destructor, so you // can let it go out of scope with no problems if you intend to // keep the library open for the life of the program. SharedLib lib; lib.load(["plugin.dll"]); getPluginImpl = cast(GetPluginImpl)lib.loadSymbol("getPluginImpl"); auto so = plugin.getSomeObject(); // Now get the plugin interface auto plugin = getPluginImpl(); // And do some work auto res = plugin.initialize(); if(res != PluginError.none) throw new PluginException(res); =================================== Anyway, given my current level of knowledge about shared library support in D, this is the approach I would take. I may very well be behind the times, though, so if you discover this isn't all necessary, then great. But I'm pretty sure Linux has the most complete support of any of the platforms right now, while Windows and Mac are rather behind. [1] https://github.com/DerelictOrg/DerelictUtil/blob/master/source/derelict/util/sharedlib.d
Jun 20 2016
interface Plugin {
bool initialize();
void terminate();
Throwable getLastException();
SomeObject getSomeObject();
void returnSomeObject(SomeObject);
}
Sorry, I forgot a couple of commments. I did explain it in the
text, though. It was supposed to read:
interface Plugin {
bool initialize();
void terminate();
// Dont' do this! Allocate exceptions on app side.
Throwable getLastException();
// Do do this. Allocate plugin objects on DLL side.
SomeObject getSomeObject();
void returnSomeObject(SomeObject);
}
Jun 20 2016
On Monday, 20 June 2016 at 13:51:15 UTC, Mike Parker wrote:Wow, absolutely awesome! That's exactly what I have been looking for. I should have some time to try it out later this week. Many thanks for all the info. I very much appreciate that you took the time to explain everything. It helped me a lot! Also thanks to everybody else for the input!interface Plugin { bool initialize(); void terminate(); Throwable getLastException(); SomeObject getSomeObject(); void returnSomeObject(SomeObject); }Sorry, I forgot a couple of commments. I did explain it in the text, though. It was supposed to read: interface Plugin { bool initialize(); void terminate(); // Dont' do this! Allocate exceptions on app side. Throwable getLastException(); // Do do this. Allocate plugin objects on DLL side. SomeObject getSomeObject(); void returnSomeObject(SomeObject); }
Jun 20 2016
I had some time to try it out and I finally got it to work. I
have only tried in windows so far but there was a pitfall in
windows. Your dll need a DllMain entry to compile. This was the
only thing that was missing from your information. The rest
worked perfectly. This may be obvious to most around here, but I
did not know before. So, I thought it might make sense to show my
working solution in case someone else stumbles upon the same
problem.
I wanted the app and the plugin to be independent projects form
one another. So they both need a shared project containing the
interface for the plugin. That makes the 3 projects shown below.
I can than after compiling the app simply copy the plugin dll
into a plugins folder and the app will find it on demand. So that
I could later add more plugins if desired. Obviously in that case
I would have to get a list of available plugins by reading the
filenames in the plugins directory instead of a hard coded path
in the app. I tried to reduce it to the bare minimum with a
working solution. Even with the DllMain entry (which is necessary
for windows) it turns out to be surprisingly compact.
There are 3 distinct projects. I have the following directories
(omitting some generated by dub):
- PluginTest
-- PluginContract // all files for the shared plugin interface
(separate project)
---- source
------ iplugin.d // the interface for the plugin
---- dub.json // the project file for the shared plugin interface
-- TestApp // all files for the app (separate project)
---- packages
------ DerelictUtil-master // contains the project for derelict
---- source
------ app.d // the app
---- dub.json // the project file for the app
-- TestPlugin // all files for the plugin (separate project)
---- source
------ someplugin.d // the plugin
---- dub.json // the project file for the plugin
Here are the files:
Shared plugin interface (Project 1)
===================================
Note, these are necessary for the linker to find the files:
"targetType": "library"
"targetPath": "lib"
PluginTest/PluginContract/dub.json
----------------------------------
{
"name": "plugincontract",
"authors": ["root"],
"description": "A minimal D application.",
"copyright": "Copyright © 2016, root",
"license": "proprietary",
"platforms": ["windows"],
"versions": ["DesktopApp"],
"targetType": "library",
"configurations": [
{
"name": "debug",
"targetPath": "lib",
"buildOptions": ["debugMode", "debugInfo"],
},
{
"name": "release",
"targetPath": "lib",
"buildOptions": ["releaseMode", "optimize", "inline"],
}
]
}
PluginTest/PluginContract/source/iplugin.d
------------------------------------------
module iplugin;
interface IPlugin
{
void Talk(string msg);
}
TestApp (Project 2)
===================
PluginTest/TestApp/dub.json
---------------------------
{
"name": "testapp",
"authors": ["root"],
"description": "A minimal D application.",
"copyright": "Copyright © 2016, root",
"license": "proprietary",
"platforms": ["windows"],
"versions": ["DesktopApp"],
"targetType": "executable",
"dependencies": {
"derelict-util": {"path": "packages/DerelictUtil-master"},
"plugincontract": {"path": "../PluginContract"}
},
"configurations": [
{
"name": "debug",
"targetPath": "bin/debug",
"buildOptions": ["debugMode", "debugInfo"],
},
{
"name": "release",
"targetPath": "bin/release",
"buildOptions": ["releaseMode", "optimize", "inline"],
}
]
}
PluginTest/TestApp/source/app.d
-------------------------------
import std.stdio;
import derelict.util.sharedlib;
import iplugin;
alias GetPluginImpl = extern(C) nothrow IPlugin function();
GetPluginImpl getPlugin;
void main()
{
SharedLib lib;
lib.load(["plugins/testplugin.dll"]);
getPlugin = cast(GetPluginImpl)lib.loadSymbol("getPlugin");
auto plugin = getPlugin();
plugin.Talk("Hello World.");
writeln("End of app.");
}
TestPlugin (Project 3)
======================
PluginTest/TestPlugin/dub.json
------------------------------
{
"name": "testplugin",
"authors": ["root"],
"description": "A minimal D application.",
"copyright": "Copyright © 2016, root",
"license": "proprietary",
"platforms": ["windows"],
"versions": ["DesktopApp"],
"targetType": "dynamicLibrary",
"importPaths": ["../PluginContract"],
"dependencies": {
"plugincontract": {"path": "../PluginContract"}
},
"configurations": [
{
"name": "debug",
"targetPath": "bin/debug",
"buildOptions": ["debugMode", "debugInfo"],
},
{
"name": "release",
"targetPath": "bin/release",
"buildOptions": ["releaseMode", "optimize", "inline"],
}
]
}
PluginTest/TestPlugin/source/SomePlugin.d
-----------------------------------------
module someplugin;
import std.stdio;
import iplugin;
export extern(C) nothrow IPlugin getPlugin()
{
import core.memory : GC;
auto plugin = new SomePlugin();
// Make sure the GC doesn't collect the instance
GC.addRoot(cast(void*)plugin);
return plugin;
}
class SomePlugin : IPlugin
{
// this() {}
void Talk(string msg)
{
writefln("SomePlugin: %s", msg);
}
}
//shared static this() { printf("plugin shared static this\n"); }
//shared static ~this() { printf("plugin shared static ~this\n");
}
version(Windows)
extern(Windows) bool DllMain(void* hInstance, uint ulReason,
void*)
{
import std.c.windows.windows;
import core.sys.windows.dll;
switch (ulReason)
{
default: assert(0);
case DLL_PROCESS_ATTACH:
dll_process_attach( hInstance, true );
break;
case DLL_PROCESS_DETACH:
dll_process_detach( hInstance, true );
break;
case DLL_THREAD_ATTACH:
dll_thread_attach( true, true );
break;
case DLL_THREAD_DETACH:
dll_thread_detach( true, true );
break;
}
return true;
}
Hope it helps someone. And again thanks for the help!
Jun 21 2016
On Tuesday, 21 June 2016 at 23:59:54 UTC, moe wrote:I had some time to try it out and I finally got it to work. I have only tried in windows so far but there was a pitfall in windows. Your dll need a DllMain entry to compile. This was the only thing that was missing from your information.Right, I forgot about that. It's always optional in C and C++, but in D we need to use it to initialize the runtime.-- TestApp // all files for the app (separate project) ---- packages ------ DerelictUtil-master // contains the project for derelict ---- source ------ app.d // the app ---- dub.json // the project file for the appThis is not the way DerelictUtil, or any DUB package, is intended to be used when you use DUB to manage your project. You don't need to download it this way. You chould change your dependency in TestApp:"dependencies": { "derelict-util": "version=~>2.0.6",2.0.6 is the latest. The '~>' means >=2.0.6 && <2.1.0, so that if there is a 2.0.7 released, you can run 'dub upgrade' on your project and start using it. The same for 2.0.8, 2.0.9, 2.0.n... Of course, if you have a reason for doing it that way, more power to you :)
Jun 21 2016
On Wednesday, 22 June 2016 at 01:40:47 UTC, Mike Parker wrote:On Tuesday, 21 June 2016 at 23:59:54 UTC, moe wrote:Yes, I did it intentionally. I wanted to ensure that the packages are self contained and check whether it would work fine like this. Basically I like to have a project that contains everything it needs with the versions originally used to build. It sort of creates a reference for learning. For testing I would have also liked not to build a dub project for the PluginContract but rather have a simple iplugins.d file in a folder. I could not get it to be imported if it's outside of a dub project. "importPaths": ["../PluginContract"] Did not work for some reason. But that's a minor issue.I had some time to try it out and I finally got it to work. I have only tried in windows so far but there was a pitfall in windows. Your dll need a DllMain entry to compile. This was the only thing that was missing from your information.Right, I forgot about that. It's always optional in C and C++, but in D we need to use it to initialize the runtime.-- TestApp // all files for the app (separate project) ---- packages ------ DerelictUtil-master // contains the project for derelict ---- source ------ app.d // the app ---- dub.json // the project file for the appThis is not the way DerelictUtil, or any DUB package, is intended to be used when you use DUB to manage your project. You don't need to download it this way. You chould change your dependency in TestApp:"dependencies": { "derelict-util": "version=~>2.0.6",2.0.6 is the latest. The '~>' means >=2.0.6 && <2.1.0, so that if there is a 2.0.7 released, you can run 'dub upgrade' on your project and start using it. The same for 2.0.8, 2.0.9, 2.0.n... Of course, if you have a reason for doing it that way, more power to you :)
Jun 21 2016
On Wednesday, 22 June 2016 at 02:38:23 UTC, moe wrote:Yes, I did it intentionally. I wanted to ensure that the packages are self contained and check whether it would work fine like this. Basically I like to have a project that contains everything it needs with the versions originally used to build. It sort of creates a reference for learning.Understood. FYI, you can specify a specific version and dub upgrade won't touch it.For testing I would have also liked not to build a dub project for the PluginContract but rather have a simple iplugins.d file in a folder. I could not get it to be imported if it's outside of a dub project. "importPaths": ["../PluginContract"] Did not work for some reason. But that's a minor issue.Becuase you specified an incomplete path: ../PluginContract/source The compiler needs the root directory of the root package for any imporyed module.
Jun 21 2016
On Wednesday, 22 June 2016 at 02:54:06 UTC, Mike Parker wrote:On Wednesday, 22 June 2016 at 02:38:23 UTC, moe wrote:I meant like this: - PluginContract // not a dub project, just some folder -- iplugin.d - TestApp // all files for the app (separate project) -- packages ---- DerelictUtil-master // contains the project for derelict -- source ---- app.d // the app -- dub.json // the project file for the app The only dub project would be TestApp. PluginContract would just be some folder completely outside the TestApp dub project. I could not get a relative path to work like this.Yes, I did it intentionally. I wanted to ensure that the packages are self contained and check whether it would work fine like this. Basically I like to have a project that contains everything it needs with the versions originally used to build. It sort of creates a reference for learning.Understood. FYI, you can specify a specific version and dub upgrade won't touch it.For testing I would have also liked not to build a dub project for the PluginContract but rather have a simple iplugins.d file in a folder. I could not get it to be imported if it's outside of a dub project. "importPaths": ["../PluginContract"] Did not work for some reason. But that's a minor issue.Becuase you specified an incomplete path: ../PluginContract/source The compiler needs the root directory of the root package for any imporyed module.
Jun 21 2016
On Wednesday, 22 June 2016 at 03:06:29 UTC, moe wrote:I meant like this: - PluginContract // not a dub project, just some folder -- iplugin.d - TestApp // all files for the app (separate project) -- packages ---- DerelictUtil-master // contains the project for derelict -- source ---- app.d // the app -- dub.json // the project file for the app The only dub project would be TestApp. PluginContract would just be some folder completely outside the TestApp dub project. I could not get a relative path to work like this.Just to be clear, are you compiling iplugin.d as well? I assumed you were referring to a compiler error (i.e. missing import), but based on this post I would guess you're getting a linker error. You should probably add this to your dub.json in addition to the importPaths: "sourceFiles": ["../PluginContract/iplugin.d"]
Jun 21 2016
On Wednesday, 22 June 2016 at 05:34:33 UTC, Mike Parker wrote:On Wednesday, 22 June 2016 at 03:06:29 UTC, moe wrote:I have added all of these to the dub.json: "sourcePaths": ["../PluginContract"], "importPaths": ["../PluginContract"], "sourceFiles": ["../PluginContract/iplugin.d"], In my app I use: import iplugin; I would expect that both the compiler and the linker finds the needed files. I would also prefer a path to link a folder rather than adding files individually. It seams more error prone when I have to remember to add every file in a bigger project. However, every combination of the above seams to fail. With a linker error. Linking... OPTLINK (R) for Win32 Release 8.00.17 Copyright (C) Digital Mars 1989-2013 All rights reserved. http://www.digitalmars.com/ctg/optlink.html OPTLINK : Warning 23: No Stack .dub\build\debug-debug-windows-x86-dmd_2071-0BEC1C92408DC77EE5C50BCF4B1225A9\tes tapp.obj(testapp) Error 42: Symbol Undefined _D18TypeInfo_Interface6__vtblZ .dub\build\debug-debug-windows-x86-dmd_2071-0BEC1C92408DC77EE5C50BCF4B1225A9\tes tapp.obj(testapp) Error 42: Symbol Undefined _D14TypeInfo_Class6__vtblZ OPTLINK : Warning 134: No Start Address --- errorlevel 2 dmd failed with exit code 2. Without the adjustments in the dub.json I get the following error (But that is expected if dub only searches in the source folder by default): testplugin ~master: building configuration "debug"... source\SomePlugin.d(3,8): Error: module iplugin is in file 'iplugin.d' which can not be read import path[0] = source import path[1] = C:\D\dmd2\windows\bin\..\..\src\phobos import path[2] = C:\D\dmd2\windows\bin\..\..\src\druntime\import dmd failed with exit code 1.I meant like this: - PluginContract // not a dub project, just some folder -- iplugin.d - TestApp // all files for the app (separate project) -- packages ---- DerelictUtil-master // contains the project for derelict -- source ---- app.d // the app -- dub.json // the project file for the app The only dub project would be TestApp. PluginContract would just be some folder completely outside the TestApp dub project. I could not get a relative path to work like this.Just to be clear, are you compiling iplugin.d as well? I assumed you were referring to a compiler error (i.e. missing import), but based on this post I would guess you're getting a linker error. You should probably add this to your dub.json in addition to the importPaths: "sourceFiles": ["../PluginContract/iplugin.d"]
Jun 22 2016