www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - D scripting

reply EntangledQuanta <EQ universe.com> writes:
I would like to use D as a "scripting" language for my D app. 
There seems to be no such thing.

Since we can include the D compiler in our distribution, it is 
easy to enable "plugin" capabilities, but directly interfacing 
with the source code seems like it would require a bit of 
work(duplicating the code that one wants to include so it can be 
linked in and "hot swapping").

e.g., Suppose I have a program like

struct X { ... }

void main()
{
     runScript("X x;"}
}

for the script to have access to X, it must be included in the 
script being compiled. That requires it having access to the 
"program code". I'd have to remove all the stuff I do not want it 
to have access to and that could be a real pain.

But surely we can use attributes, something like

 scriptable struct X { ... }

that is makes X exportable to an obj file that can then be 
included in to the script. Everything, then, in my program, that 
is marked as such can be accessed and the script becomes a simple 
DLL. Dealing with hot swapping is then the only trouble. For 
simple scripts, this shouldn't be a problem though.

Anyone see a way that this could be achieved rather easily? Say, 
at compilation, a template gathers all the scriptable elements, 
gets their source code(which would usually be classes, structs, 
enums, functions, and some global variables, and emits the code 
in a way that ends up in it's own object file(since we can't 
write to files at compile time ;/)

Then in the scripting section of the app, It's just a matter of 
compiling with the obj file to give the script access to some of 
the program internals.

D needs an export("filename")... I'm ok with the security hole. 
No need to bust my balls for it. A switch could be required to 
enable it or a mail in rebate. No need to force me in to a box 
that doesn't exist, is there? (could only export to the -J path 
and maybe require a few other hoops to jump through if one is so 
worried about security... maybe an optometric scanner?)
Sep 05
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 5 September 2017 at 07:32:24 UTC, EntangledQuanta 
wrote:
 I would like to use D as a "scripting" language for my D app. 
 There seems to be no such thing.

 Since we can include the D compiler in our distribution, it is 
 easy to enable "plugin" capabilities, but directly interfacing 
 with the source code seems like it would require a bit of 
 work(duplicating the code that one wants to include so it can 
 be linked in and "hot swapping").
Which OS do you use? I had a similiar idea but failed on windows due to some strange effects. I think they were caused by the known windows dll unload bug, discussed here: http://forum.dlang.org/thread/rreyasqnvyagrkvqrjos forum.dlang.org At the end I decided to use the script engine from Adam Ruppe (arsd) until this bug is fixed. Kind regards André
Sep 05
parent reply EntangledQuanta <EQ universe.com> writes:
On Tuesday, 5 September 2017 at 08:13:02 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 07:32:24 UTC, EntangledQuanta 
 wrote:
 I would like to use D as a "scripting" language for my D app. 
 There seems to be no such thing.

 Since we can include the D compiler in our distribution, it is 
 easy to enable "plugin" capabilities, but directly interfacing 
 with the source code seems like it would require a bit of 
 work(duplicating the code that one wants to include so it can 
 be linked in and "hot swapping").
Which OS do you use? I had a similiar idea but failed on windows due to some strange effects. I think they were caused by the known windows dll unload bug, discussed here: http://forum.dlang.org/thread/rreyasqnvyagrkvqrjos forum.dlang.org At the end I decided to use the script engine from Adam Ruppe (arsd) until this bug is fixed. Kind regards André
Yes, windows ;/ Seems that thread has some answers! Maybe bug him enough to fix the bug? How far did you get with it? "The problem seems to only manifest when a proper DllMain() method is exported from the library. If none is provided, or if the given implementation can be optimized away, the error does not ocurr." Was that the case for you too? That could be overcome with just using a normal function that is called right after loading? I'm curious how the exporting of code as that seems to be the biggest challenge(so that we don't have to hand write the exports ourselves). Thanks.
Sep 05
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 5 September 2017 at 18:37:17 UTC, EntangledQuanta 
wrote:
 On Tuesday, 5 September 2017 at 08:13:02 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 07:32:24 UTC, EntangledQuanta 
 wrote:
 I would like to use D as a "scripting" language for my D app. 
 There seems to be no such thing.

 Since we can include the D compiler in our distribution, it 
 is easy to enable "plugin" capabilities, but directly 
 interfacing with the source code seems like it would require 
 a bit of work(duplicating the code that one wants to include 
 so it can be linked in and "hot swapping").
Which OS do you use? I had a similiar idea but failed on windows due to some strange effects. I think they were caused by the known windows dll unload bug, discussed here: http://forum.dlang.org/thread/rreyasqnvyagrkvqrjos forum.dlang.org At the end I decided to use the script engine from Adam Ruppe (arsd) until this bug is fixed. Kind regards André
Yes, windows ;/ Seems that thread has some answers! Maybe bug him enough to fix the bug? How far did you get with it? "The problem seems to only manifest when a proper DllMain() method is exported from the library. If none is provided, or if the given implementation can be optimized away, the error does not ocurr." Was that the case for you too? That could be overcome with just using a normal function that is called right after loading? I'm curious how the exporting of code as that seems to be the biggest challenge(so that we don't have to hand write the exports ourselves). Thanks.
My issue was that after unload the shared library, the dll file was still locked for deletion on the file system. Therefore I was not able to change something in my "script" and restart it. Somehow even after terminating in task manager, the dll file was still locked. I assume this reproducable effect is caused by the known issue. I already give up at this point ): Just an idea for you: in delphi you can set the properties of a component (a class with runtime reflection enabled) on runtime. You can even call the methods and events of a component. I build a Delphi Bridge for D (see recent post on announce). It is almost the same scenario as here are also dll calls involved. What I want to say, you could build something like the Delphi rtti for your D classes and make generic methods available via the dll interface. Kind regards André
Sep 05
parent reply EntangledQuanta <EQ universe.com> writes:
On Tuesday, 5 September 2017 at 19:19:19 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 18:37:17 UTC, EntangledQuanta 
 wrote:
 On Tuesday, 5 September 2017 at 08:13:02 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 07:32:24 UTC, EntangledQuanta 
 wrote:
 I would like to use D as a "scripting" language for my D 
 app. There seems to be no such thing.

 Since we can include the D compiler in our distribution, it 
 is easy to enable "plugin" capabilities, but directly 
 interfacing with the source code seems like it would require 
 a bit of work(duplicating the code that one wants to include 
 so it can be linked in and "hot swapping").
Which OS do you use? I had a similiar idea but failed on windows due to some strange effects. I think they were caused by the known windows dll unload bug, discussed here: http://forum.dlang.org/thread/rreyasqnvyagrkvqrjos forum.dlang.org At the end I decided to use the script engine from Adam Ruppe (arsd) until this bug is fixed. Kind regards André
Yes, windows ;/ Seems that thread has some answers! Maybe bug him enough to fix the bug? How far did you get with it? "The problem seems to only manifest when a proper DllMain() method is exported from the library. If none is provided, or if the given implementation can be optimized away, the error does not ocurr." Was that the case for you too? That could be overcome with just using a normal function that is called right after loading? I'm curious how the exporting of code as that seems to be the biggest challenge(so that we don't have to hand write the exports ourselves). Thanks.
My issue was that after unload the shared library, the dll file was still locked for deletion on the file system. Therefore I was not able to change something in my "script" and restart it. Somehow even after terminating in task manager, the dll file was still locked. I assume this reproducable effect is caused by the known issue. I already give up at this point ):
Hmm, I used to have that problem with windows and visual studio. It was a Visual studio issue. Not sure if that is what you were using. Sometimes it's just programs that lock on to it for no good reason. There are ways around that: Use "unlocker" to unlock the file before deletion. Possibly rename the file to a random name opening up the space. Remove the generated files later when they build up. Load the DLL from memory(There are some online memory DLL loaders).
 Just an idea for you: in delphi you can set the properties of a 
 component (a class with runtime reflection enabled) on runtime. 
 You can even call the methods and events of a component. I 
 build a Delphi Bridge for D (see recent post on announce). It 
 is almost the same scenario as here are also dll calls involved.
 What I want to say, you could build something like the Delphi 
 rtti for your D classes and make generic methods available via 
 the dll interface.
But that would be quite a bit of work? Modifying the compiler? I'm just looking for something relatively straightforward and simple ;)
Sep 05
parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 5 September 2017 at 19:44:40 UTC, EntangledQuanta 
wrote:
 Just an idea for you: in delphi you can set the properties of 
 a component (a class with runtime reflection enabled) on 
 runtime. You can even call the methods and events of a 
 component. I build a Delphi Bridge for D (see recent post on 
 announce). It is almost the same scenario as here are also dll 
 calls involved.
 What I want to say, you could build something like the Delphi 
 rtti for your D classes and make generic methods available via 
 the dll interface.
But that would be quite a bit of work? Modifying the compiler? I'm just looking for something relatively straightforward and simple ;)
It is possible without modifying the compiler. In every class you want enable for runtime reflection you need to add a generic method which generates for all public properties/methods coding to fill/call them. It is a mix of templates and mixins. In the end compile time reflection capabilities of D are so powerful that you can write runtime reflection with it. Thanks for the tip! Kind regards André
Sep 05
parent reply EntangledQuanta <EQ universe.com> writes:
On Tuesday, 5 September 2017 at 19:59:27 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 19:44:40 UTC, EntangledQuanta 
 wrote:
 Just an idea for you: in delphi you can set the properties of 
 a component (a class with runtime reflection enabled) on 
 runtime. You can even call the methods and events of a 
 component. I build a Delphi Bridge for D (see recent post on 
 announce). It is almost the same scenario as here are also 
 dll calls involved.
 What I want to say, you could build something like the Delphi 
 rtti for your D classes and make generic methods available 
 via the dll interface.
But that would be quite a bit of work? Modifying the compiler? I'm just looking for something relatively straightforward and simple ;)
It is possible without modifying the compiler. In every class you want enable for runtime reflection you need to add a generic method which generates for all public properties/methods coding to fill/call them. It is a mix of templates and mixins. In the end compile time reflection capabilities of D are so powerful that you can write runtime reflection with it. Thanks for the tip! Kind regards André
Thanks, Yeah, that is essentially what I was going to do with attributes, but rather than having a member do it, have a free function that tries to do the same thing... But then the question remains how to output that information so it can then be used to link in to the "script" that will be compiled?
Sep 05
parent Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 5 September 2017 at 21:41:35 UTC, EntangledQuanta 
wrote:
 On Tuesday, 5 September 2017 at 19:59:27 UTC, Andre Pany wrote:
 On Tuesday, 5 September 2017 at 19:44:40 UTC, EntangledQuanta 
 wrote:
 Just an idea for you: in delphi you can set the properties 
 of a component (a class with runtime reflection enabled) on 
 runtime. You can even call the methods and events of a 
 component. I build a Delphi Bridge for D (see recent post on 
 announce). It is almost the same scenario as here are also 
 dll calls involved.
 What I want to say, you could build something like the 
 Delphi rtti for your D classes and make generic methods 
 available via the dll interface.
But that would be quite a bit of work? Modifying the compiler? I'm just looking for something relatively straightforward and simple ;)
It is possible without modifying the compiler. In every class you want enable for runtime reflection you need to add a generic method which generates for all public properties/methods coding to fill/call them. It is a mix of templates and mixins. In the end compile time reflection capabilities of D are so powerful that you can write runtime reflection with it. Thanks for the tip! Kind regards André
Thanks, Yeah, that is essentially what I was going to do with attributes, but rather than having a member do it, have a free function that tries to do the same thing... But then the question remains how to output that information so it can then be used to link in to the "script" that will be compiled?
While thinking about it I only remember the points which makes the story quite hard at the moment on windows: The garbage collector is not shared between your exe and the shared libraries. There is a D Wiki which explains how to overcome this issue, but it seems it only works for statically linked libraries. For dynamically linked libraries I didn't get it working. https://wiki.dlang.org/Win32_DLLs_in_D The export attribute might also get into your way on Windows . There is a D Conf video about it, I think from 2016. I do not know wheter it helps you. There is a module jsvar which is used in the script engine from Adam Ruppe. It is used for getting access to the D functionality out of the script. He also written a good book with information how to build runtime reflection with D. Kind regards André
Sep 06