www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating and loading D plugins in D app

reply aberba <karabutaworld gmail.com> writes:
Want to create and load plugins written in D into a D app at 
run-time, the kind that can make api calls or extended main app 
with other functionality.

I'm currently interested in it for a vibe.d app. How does these 
stuff work?
Jun 01
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 1 June 2017 at 23:24:13 UTC, aberba wrote:
 Want to create and load plugins written in D into a D app at 
 run-time, the kind that can make api calls or extended main app 
 with other functionality.

 I'm currently interested in it for a vibe.d app. How does these 
 stuff work?
It works using shared libraries.
Jun 01
next sibling parent aberba <karabutaworld gmail.com> writes:
On Friday, 2 June 2017 at 02:05:23 UTC, Stefan Koch wrote:
 On Thursday, 1 June 2017 at 23:24:13 UTC, aberba wrote:
 Want to create and load plugins written in D into a D app at 
 run-time, the kind that can make api calls or extended main 
 app with other functionality.

 I'm currently interested in it for a vibe.d app. How does 
 these stuff work?
It works using shared libraries.
Oops
Jun 02
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Friday, 2 June 2017 at 02:05:23 UTC, Stefan Koch wrote:
 On Thursday, 1 June 2017 at 23:24:13 UTC, aberba wrote:
 Want to create and load plugins written in D into a D app at 
 run-time, the kind that can make api calls or extended main 
 app with other functionality.

 I'm currently interested in it for a vibe.d app. How does 
 these stuff work?
It works using shared libraries.
I just read it on Wikipedia. An alternative is to use a scripting/interpreted language. Working with such approach feels unnatural in D (considering pyd, lua-d). A more sustanable approach would be: 1. Get shared libs to work in D (the best approach for all D code) 1. some kind of embeddable interpreter for a scripting language like (a mini js engine) which exposes callable native D APIs at runtime None of which is within my current ability.
Jun 02
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all D 
 code)
I have done very little with this myself but other people have so it is doable.
 1. some kind of embeddable interpreter for a scripting language 
 like (a mini js engine) which exposes callable native D APIs at 
 runtime
My script.d does this kind of thing http://dpldocs.info/experimental-docs/arsd.script.html#examples it is slow though.
Jun 02
next sibling parent reply aberba <karabutaworld gmail.com> writes:
On Friday, 2 June 2017 at 12:19:48 UTC, Adam D. Ruppe wrote:
 On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all D 
 code)
I have done very little with this myself but other people have so it is doable.
 1. some kind of embeddable interpreter for a scripting 
 language like (a mini js engine) which exposes callable native 
 D APIs at runtime
My script.d does this kind of thing http://dpldocs.info/experimental-docs/arsd.script.html#examples it is slow though.
Performance doesn't matter now. Can source of script be reloaded at runtime? Do I have to wrap external APIs in the "global" object passed as argument to "interpreter()" I'm trying to implement a plugin system for a vibe.d server where plugins can be installed to provide additional http routes: extending server functionality.
Jun 02
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 2 June 2017 at 13:05:41 UTC, aberba wrote:
 Can source of script be reloaded at runtime?
It is just an ordinary string.
 Do I have to wrap external APIs in the "global" object passed 
 as argument to "interpreter()"
Yes, anything the script calls must be exposed through that. It does a decent job automatically wrapping functions though, so just assigning them in a list might work.
 I'm trying to implement a plugin system for a vibe.d server 
 where plugins can be installed to provide additional http 
 routes: extending server functionality.
might work.
Jun 02
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Fri, Jun 02, 2017 at 12:19:48PM +0000, Adam D. Ruppe via Digitalmars-d-learn
wrote:
 On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all D code)
I have done very little with this myself but other people have so it is doable.
[...] This is not directly related to the OP's question, but recently I wrote a program that, given a user-specified string, transforms it into D code using a code template, invokes dmd to compile it into a shared object, loads the shared object using dlopen(), and looks up the generated function with dlsym() to obtain a function pointer that can be used for calling the function. The shared object is unloaded after it's done. So it's definitely doable, in the sense that I've successfully generated D code, compiled it into a shared library, loaded it into the running executable, and can call the code. Of course, in my case, the code template is relatively simple so I don't have to worry about things like module ctors, shared data, non-TLS globals, or GC use. If you're planning to support those features, you might need to do a bit more work after calling dlopen(). At the very least you'd have to use dlsym() to look up module ctor symbols and run them before calling any other functions in the shared object, and you may have to initialize druntime in the shared object too, if it's statically linked. I'm not sure what happens if it's dynamically linked, or if it uses its own copy of the GC. In any case, this is all possible, it just takes someone to dig into the details and write the code to make it all work. Then publish it on github or dub, and the rest of us can reap the benefits too. ;-) T -- PNP = Plug 'N' Pray
Jun 02
next sibling parent aberba <karabutaworld gmail.com> writes:
On Friday, 2 June 2017 at 16:36:34 UTC, H. S. Teoh wrote:
 On Fri, Jun 02, 2017 at 12:19:48PM +0000, Adam D. Ruppe via 
 Digitalmars-d-learn wrote:
 [...]
[...] This is not directly related to the OP's question, but recently I wrote a program that, given a user-specified string, transforms it into D code using a code template, invokes dmd to compile it into a shared object, loads the shared object using dlopen(), and looks up the generated function with dlsym() to obtain a function pointer that can be used for calling the function. The shared object is unloaded after it's done. [...]
Blog on it?
Jun 03
prev sibling parent reply aberba <karabutaworld gmail.com> writes:
On Friday, 2 June 2017 at 16:36:34 UTC, H. S. Teoh wrote:
 On Fri, Jun 02, 2017 at 12:19:48PM +0000, Adam D. Ruppe via 
 Digitalmars-d-learn wrote:
 On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all D 
 code)
I have done very little with this myself but other people have so it is doable.
[...] This is not directly related to the OP's question, but recently I wrote a program that, given a user-specified string, transforms it into D code using a code template, invokes dmd to compile it into a shared object, loads the shared object using dlopen(), and looks up the generated function with dlsym() to obtain a function pointer that can be used for calling the function. The shared object is unloaded after it's done.
Will be of much use to me to see the brief instructions for this. I saw the C style on Wikipedia. Seems the functions loaded needs to be casted from void* to a type... before calling. Didn't quite understand that part.
 So it's definitely doable, in the sense that I've successfully 
 generated D code, compiled it into a shared library, loaded it 
 into the running executable, and can call the code.  Of course, 
 in my case, the code template is relatively simple so I don't 
 have to worry about things like module ctors, shared data, 
 non-TLS globals, or GC use.  If you're planning to support 
 those features, you might need to do a bit more work after 
 calling dlopen(). At the very least you'd have to use dlsym() 
 to look up module ctor symbols and run them before calling any 
 other functions in the shared object, and you may have to 
 initialize druntime in the shared object too, if it's 
 statically linked.
That seem like D with its features will be lot of work if I'm using loaded code that way. How about exposing standard interface/apis for loaded object's code to query info about the main app at runtime, similar to COM (if that's how if even works)
 I'm not sure what happens if it's dynamically linked, or if it 
 uses its own copy of the GC.
Does D work different from C in that regard?
 In any case, this is all possible, it just takes someone to dig 
 into the details and write the code to make it all work.  Then 
 publish it on github or dub, and the rest of us can reap the 
 benefits too. ;-)


 T
Jun 03
parent Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Saturday, 3 June 2017 at 09:41:58 UTC, aberba wrote:
 On Friday, 2 June 2017 at 16:36:34 UTC, H. S. Teoh wrote:
 On Fri, Jun 02, 2017 at 12:19:48PM +0000, Adam D. Ruppe via 
 Digitalmars-d-learn wrote:
 On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all 
 D code)
I have done very little with this myself but other people have so it is doable.
[...] This is not directly related to the OP's question, but recently I wrote a program that, given a user-specified string, transforms it into D code using a code template, invokes dmd to compile it into a shared object, loads the shared object using dlopen(), and looks up the generated function with dlsym() to obtain a function pointer that can be used for calling the function. The shared object is unloaded after it's done.
Will be of much use to me to see the brief instructions for this. I saw the C style on Wikipedia. Seems the functions loaded needs to be casted from void* to a type... before calling. Didn't quite understand that part.
Yes, dlsym() returns the address of the object of the shared library you requested. The first parameter is the handle to the shared object that had been loaded by dlopen(). The second parameter is the name of the object one wants the address of. The address must then be casted to the type of the object. If the name was one of a function, one has to cast to a function pointer. That's something illegal in strict C (i.e. undefined behaviour.) but is something that is required by Posix. It's really not very difficult.
Jun 03
prev sibling parent Mike B Johnson <Mikey Ikes.com> writes:
On Friday, 2 June 2017 at 12:19:48 UTC, Adam D. Ruppe wrote:
 On Friday, 2 June 2017 at 11:09:05 UTC, aberba wrote:
 1. Get shared libs to work in D (the best approach for all D 
 code)
I have done very little with this myself but other people have so it is doable.
 1. some kind of embeddable interpreter for a scripting 
 language like (a mini js engine) which exposes callable native 
 D APIs at runtime
My script.d does this kind of thing http://dpldocs.info/experimental-docs/arsd.script.html#examples it is slow though.
You should put a link on the help somewhere so one can get to the github page easily to view the source(or have the ability to go from the module to the github source easily(small icon next to module name in modules list). This would help peruse the source to see how you came up with some of your stuff ;)
Jun 02
prev sibling parent aberba <karabutaworld gmail.com> writes:
On Thursday, 1 June 2017 at 23:24:13 UTC, aberba wrote:
 Want to create and load plugins written in D into a D app at 
 run-time, the kind that can make api calls or extended main app 
 with other functionality.

 I'm currently interested in it for a vibe.d app. How does these 
 stuff work?
Plugin system c++ http://blog.nuclex-games.com/tutorials/cxx/plugin-architecture/
Jun 02