www.digitalmars.com         C & C++   DMDScript  

D - D scripting (c++ integration)

reply bubu <santyhammer latinmail.com> writes:
Hiya,

im a game programmer. I want to use some D-scripted objects in my game which
is programmed in C++. Can I embed a D scripting host like I can do with Lua,
Phyton or Ruby? thx
Oct 01 2006
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
bubu wrote:
 Hiya,
 
 im a game programmer. I want to use some D-scripted objects in my game which
 is programmed in C++. Can I embed a D scripting host like I can do with Lua,
 Phyton or Ruby? thx

This newsgroup is not used sou much anymore try digitalmars.D instead. To answer your question. I don't know of any way to embed D as a script language but if you want you could require script writers to compile their scripts using some D compiler. I think DDL (dsource.org) can be of use here, alternativly just build the scripts into a .dll/.so file. Hope that helped :)
Oct 01 2006
parent reply Pragma <ericanderton yahoo.removeme.com> writes:
Johan Granberg wrote:
 bubu wrote:
 Hiya,

 im a game programmer. I want to use some D-scripted objects in my game 
 which
 is programmed in C++. Can I embed a D scripting host like I can do 
 with Lua,
 Phyton or Ruby? thx

This newsgroup is not used sou much anymore try digitalmars.D instead. To answer your question. I don't know of any way to embed D as a script language but if you want you could require script writers to compile their scripts using some D compiler. I think DDL (dsource.org) can be of use here, alternativly just build the scripts into a .dll/.so file. Hope that helped :)

Johan, thanks for the plug. :) bubu, DDL is going to work, but only if you have a D program to start. You might be able to roll your own scripting host as a .dll or something similar, provided it's in D. However, there's no embeded compiler product available, so users will still need the D compiler installed. With regards to DDL, using D for scripting is the reason why I hatched the project in the first place. If you go back and read my journal entries in the DSP forum (also on DSource), you'll get an idea of what is ahead of you for dynamic binding on windows, without such a technology. Depending on how you proceed, you may or may not need to worry about the things I did. FWIW, the following pattern can be used to make D flow a bit more like live scripting environments: 1) Launch the D compiler from within your app directly, to compile the "script" you want to run - std.process works well. 2) Use DDL to bind the .obj file from the compiler output. The linker will return a DynamicLibrary object to use. 3) Use the DynamicLibrary to bind any functions, classes, or static fields that you know to exist in the compiled script. That last point is kind of tricky. While you can simply force script coders to adhere to a particular interface, I found it easier to wrap the whole script in a function (call that step #0) before compilation. This ensures that the starting point of the script was always known, at the cost of disallowing some D constructs (bang-line, imports, module static init, etc). It also allows the script author to start from line 1, much like one can in PHP or JavaScript. -- - EricAnderton at yahoo
Oct 02 2006
next sibling parent reply J Duncan <jtd514 nospam.ameritech.net> writes:
Pragma wrote:
 Johan Granberg wrote:
 bubu wrote:
 Hiya,

 im a game programmer. I want to use some D-scripted objects in my 
 game which
 is programmed in C++. Can I embed a D scripting host like I can do 
 with Lua,
 Phyton or Ruby? thx

This newsgroup is not used sou much anymore try digitalmars.D instead. To answer your question. I don't know of any way to embed D as a script language but if you want you could require script writers to compile their scripts using some D compiler. I think DDL (dsource.org) can be of use here, alternativly just build the scripts into a .dll/.so file. Hope that helped :)

Johan, thanks for the plug. :) bubu, DDL is going to work, but only if you have a D program to start. You might be able to roll your own scripting host as a .dll or something similar, provided it's in D. However, there's no embeded compiler product available, so users will still need the D compiler installed. With regards to DDL, using D for scripting is the reason why I hatched the project in the first place. If you go back and read my journal entries in the DSP forum (also on DSource), you'll get an idea of what is ahead of you for dynamic binding on windows, without such a technology. Depending on how you proceed, you may or may not need to worry about the things I did. FWIW, the following pattern can be used to make D flow a bit more like live scripting environments: 1) Launch the D compiler from within your app directly, to compile the "script" you want to run - std.process works well. 2) Use DDL to bind the .obj file from the compiler output. The linker will return a DynamicLibrary object to use. 3) Use the DynamicLibrary to bind any functions, classes, or static fields that you know to exist in the compiled script. That last point is kind of tricky. While you can simply force script coders to adhere to a particular interface, I found it easier to wrap the whole script in a function (call that step #0) before compilation. This ensures that the starting point of the script was always known, at the cost of disallowing some D constructs (bang-line, imports, module static init, etc). It also allows the script author to start from line 1, much like one can in PHP or JavaScript.

Actually, I spent most of my weekend experimenting with such techniques. (btw pragma I am updating the COFF code but i keep getting sidetracked with DDL adventures :) Ive got a little IDE that combines the dparser and DDL - mostly for experimentation at the present time. I have a class I call a DMachine that loads, links, and manages a set of (object/library) modules. Im currently doing basic introspection of the mangled identifiers to obtain a catalog of symbols, but Ive started to integrate dparser to get a better symbol catalogs. im playing around with having a 'extension' source module that is compiled and linked into your running exe - at runtime (I even tried did the trick of embedding a 'script' into a fake module function). Ive even added the expression parsing capabilities of dparser to parse out an expression from a string ( such as 'doSomething("hello");' to a CallExp to call a function). basically, as we have discussed, DDL is still very 'raw' and we still probably need some sort of 'sdk' layer to make this technology much easier. But, we definitely have the capabilities to do what is being discussed in this thread. Im not sure i would call this 'scripting' but that makes sense. at the least these technologies can probably open our binaries or source up to automating the integration of external scripting solutions - automatic stub generation, etc... Anyway, DDL gives us great potential in this area.....
Oct 02 2006
parent reply Pragma <ericanderton yahoo.removeme.com> writes:
J Duncan wrote:
 Pragma wrote:
 Johan Granberg wrote:
 bubu wrote:
 Hiya,

 im a game programmer. I want to use some D-scripted objects in my 
 game which
 is programmed in C++. Can I embed a D scripting host like I can do 
 with Lua,
 Phyton or Ruby? thx

This newsgroup is not used sou much anymore try digitalmars.D instead. To answer your question. I don't know of any way to embed D as a script language but if you want you could require script writers to compile their scripts using some D compiler. I think DDL (dsource.org) can be of use here, alternativly just build the scripts into a .dll/.so file. Hope that helped :)

Johan, thanks for the plug. :) bubu, DDL is going to work, but only if you have a D program to start. You might be able to roll your own scripting host as a .dll or something similar, provided it's in D. However, there's no embeded compiler product available, so users will still need the D compiler installed. With regards to DDL, using D for scripting is the reason why I hatched the project in the first place. If you go back and read my journal entries in the DSP forum (also on DSource), you'll get an idea of what is ahead of you for dynamic binding on windows, without such a technology. Depending on how you proceed, you may or may not need to worry about the things I did. FWIW, the following pattern can be used to make D flow a bit more like live scripting environments: 1) Launch the D compiler from within your app directly, to compile the "script" you want to run - std.process works well. 2) Use DDL to bind the .obj file from the compiler output. The linker will return a DynamicLibrary object to use. 3) Use the DynamicLibrary to bind any functions, classes, or static fields that you know to exist in the compiled script. That last point is kind of tricky. While you can simply force script coders to adhere to a particular interface, I found it easier to wrap the whole script in a function (call that step #0) before compilation. This ensures that the starting point of the script was always known, at the cost of disallowing some D constructs (bang-line, imports, module static init, etc). It also allows the script author to start from line 1, much like one can in PHP or JavaScript.

Actually, I spent most of my weekend experimenting with such techniques. (btw pragma I am updating the COFF code but i keep getting sidetracked with DDL adventures :) Ive got a little IDE that combines the dparser and DDL - mostly for experimentation at the present time. I have a class I call a DMachine that loads, links, and manages a set of (object/library) modules. Im currently doing basic introspection of the mangled identifiers to obtain a catalog of symbols, but Ive started to integrate dparser to get a better symbol catalogs. im playing around with having a 'extension' source module that is compiled and linked into your running exe - at runtime (I even tried did the trick of embedding a 'script' into a fake module function). Ive even added the expression parsing capabilities of dparser to parse out an expression from a string ( such as 'doSomething("hello");' to a CallExp to call a function). basically, as we have discussed, DDL is still very 'raw' and we still probably need some sort of 'sdk' layer to make this technology much easier. But, we definitely have the capabilities to do what is being discussed in this thread. Im not sure i would call this 'scripting' but that makes sense. at the least these technologies can probably open our binaries or source up to automating the integration of external scripting solutions - automatic stub generation, etc... Anyway, DDL gives us great potential in this area.....

Awesome. I can't wait to see what you've cooked up. The idea of using DDL for IDE extensions is extremely cool - reminds me of VB-macros for Visual Studio. I've been trying to develop a DDL-based reflection interface myself. I can't blame you for using a parser to obtain a symbol catalog. After compilation, I've noticed that there are some things that get thrown out: - non-static field information (it's just an offset) - templates (template declarations - instances, however, *are* visible) - enums ... which is going to get a big "duh" from the gallery. :) Anyway, some of this stuff is likely recoverable via debug info, but that's never 100% reliable, nor implemented (in DDL) as of yet. Which leads me to a question I've had for a while now. You mentioned stub generators: do you (Mr. Duncan) think that it's feasible to have a front end that generates runtime reflection data, suitable for build to fold in automatically? Have you seen/done anything that would make this a bad move? -- - EricAnderton at yahoo
Oct 02 2006
parent J Duncan <jtd514 nospam.ameritech.net> writes:
Pragma wrote:
 J Duncan wrote:
 Pragma wrote:
 Johan Granberg wrote:
 bubu wrote:
 Hiya,

 im a game programmer. I want to use some D-scripted objects in my 
 game which
 is programmed in C++. Can I embed a D scripting host like I can do 
 with Lua,
 Phyton or Ruby? thx

This newsgroup is not used sou much anymore try digitalmars.D instead. To answer your question. I don't know of any way to embed D as a script language but if you want you could require script writers to compile their scripts using some D compiler. I think DDL (dsource.org) can be of use here, alternativly just build the scripts into a .dll/.so file. Hope that helped :)

Johan, thanks for the plug. :) bubu, DDL is going to work, but only if you have a D program to start. You might be able to roll your own scripting host as a .dll or something similar, provided it's in D. However, there's no embeded compiler product available, so users will still need the D compiler installed. With regards to DDL, using D for scripting is the reason why I hatched the project in the first place. If you go back and read my journal entries in the DSP forum (also on DSource), you'll get an idea of what is ahead of you for dynamic binding on windows, without such a technology. Depending on how you proceed, you may or may not need to worry about the things I did. FWIW, the following pattern can be used to make D flow a bit more like live scripting environments: 1) Launch the D compiler from within your app directly, to compile the "script" you want to run - std.process works well. 2) Use DDL to bind the .obj file from the compiler output. The linker will return a DynamicLibrary object to use. 3) Use the DynamicLibrary to bind any functions, classes, or static fields that you know to exist in the compiled script. That last point is kind of tricky. While you can simply force script coders to adhere to a particular interface, I found it easier to wrap the whole script in a function (call that step #0) before compilation. This ensures that the starting point of the script was always known, at the cost of disallowing some D constructs (bang-line, imports, module static init, etc). It also allows the script author to start from line 1, much like one can in PHP or JavaScript.

Actually, I spent most of my weekend experimenting with such techniques. (btw pragma I am updating the COFF code but i keep getting sidetracked with DDL adventures :) Ive got a little IDE that combines the dparser and DDL - mostly for experimentation at the present time. I have a class I call a DMachine that loads, links, and manages a set of (object/library) modules. Im currently doing basic introspection of the mangled identifiers to obtain a catalog of symbols, but Ive started to integrate dparser to get a better symbol catalogs. im playing around with having a 'extension' source module that is compiled and linked into your running exe - at runtime (I even tried did the trick of embedding a 'script' into a fake module function). Ive even added the expression parsing capabilities of dparser to parse out an expression from a string ( such as 'doSomething("hello");' to a CallExp to call a function). basically, as we have discussed, DDL is still very 'raw' and we still probably need some sort of 'sdk' layer to make this technology much easier. But, we definitely have the capabilities to do what is being discussed in this thread. Im not sure i would call this 'scripting' but that makes sense. at the least these technologies can probably open our binaries or source up to automating the integration of external scripting solutions - automatic stub generation, etc... Anyway, DDL gives us great potential in this area.....

Awesome. I can't wait to see what you've cooked up. The idea of using DDL for IDE extensions is extremely cool - reminds me of VB-macros for Visual Studio. I've been trying to develop a DDL-based reflection interface myself. I can't blame you for using a parser to obtain a symbol catalog. After compilation, I've noticed that there are some things that get thrown out: - non-static field information (it's just an offset) - templates (template declarations - instances, however, *are* visible) - enums ... which is going to get a big "duh" from the gallery. :) Anyway, some of this stuff is likely recoverable via debug info, but that's never 100% reliable, nor implemented (in DDL) as of yet. Which leads me to a question I've had for a while now. You mentioned stub generators: do you (Mr. Duncan) think that it's feasible to have a front end that generates runtime reflection data, suitable for build to fold in automatically? Have you seen/done anything that would make this a bad move?

Yeah I think its completely feasible. In fact I think the dflect project does something similar with the C++ front end. it generates reflection data into a source module that gets compiled and linked into the final binary. something like this is more than feasible, we just need to come up with a good system. We need a good data representation of reflection info plus an api to access it. we could just generate source code to compile into a project - or even go all the way to produce an OMF binary file.... In my mind just about everything we want to do is right in front of us, we just have to choose a path. And I am not one that is good at defining systems for a community to use, so I have mostly stuck to my personal projects so far. My only prior experience at reflection is openc++ and their compile time system, which in theory is way cooler than in practice. Over the last year or so I have tried to instigate interest in the D community for both compile-time and runtime introspection systems. I dont have any experience with java reflection so I was hoping more informed people than myself would come forward with ideas and comments. as for compile-time reflection it is best handled by the compiler; however we can do it outside the compiler by generating intermediate source code; something I have been experimenting with but not sure its really practical on a large project, and it causes debugging issues. Anyway i tend to ramble - all the pieces are there for basic runtime reflection information; we just need a good system.
Oct 02 2006
prev sibling parent reply mike <vertex gmx.at> writes:
Maybe for DMD 3.0 ...

' import std.compiler;
'
' char[] source =3D "writefln(\"Hello World!\");";
' void delegate() foo =3D compile(source);
' foo();

That would be just awesome :-)

- mike

 1) Launch the D compiler from within your app directly, to compile the=

 "script" you want to run - std.process works well.
 2) Use DDL to bind the .obj file from the compiler output.  The linker=

 will return a DynamicLibrary object to use.
 3) Use the DynamicLibrary to bind any functions, classes, or static  =

 fields that you know to exist in the compiled script.

 That last point is kind of tricky.  While you can simply force script =

 coders to adhere to a particular interface, I found it easier to wrap =

 the whole script in a function (call that step #0) before compilation.=

 This ensures that the starting point of the script was always known, a=

 the cost of disallowing some D constructs (bang-line, imports, module =

 static init, etc).  It also allows the script author to start from lin=

 1, much like one can in PHP or JavaScript.

-- = Erstellt mit Operas revolution=E4rem E-Mail-Modul: http://www.opera.com/= mail/
Oct 03 2006
next sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
mike wrote:
 Maybe for DMD 3.0 ...
 
 ' import std.compiler;
 '
 ' char[] source = "writefln(\"Hello World!\");";
 ' void delegate() foo = compile(source);
 ' foo();
 
 That would be just awesome :-)
 
 - mike

In addition to all the obvious, I'd say this is yet another thing that's not doable in C++. Compiling anything takes simply too long. But D, it's totally different. And especially where the same program keeps compiling different stuff, then speed would become quite fast since the compiler wouldn't be discarded from memory. "Blurring" this way the distinction and chasm between runtime and compile time should give us quite interesting new possibilities and avenues.
 1) Launch the D compiler from within your app directly, to compile 
 the  "script" you want to run - std.process works well.
 2) Use DDL to bind the .obj file from the compiler output.  The 
 linker  will return a DynamicLibrary object to use.
 3) Use the DynamicLibrary to bind any functions, classes, or static  
 fields that you know to exist in the compiled script.

 That last point is kind of tricky.  While you can simply force script  
 coders to adhere to a particular interface, I found it easier to wrap  
 the whole script in a function (call that step #0) before 
 compilation.  This ensures that the starting point of the script was 
 always known, at  the cost of disallowing some D constructs 
 (bang-line, imports, module  static init, etc).  It also allows the 
 script author to start from line  1, much like one can in PHP or 
 JavaScript.


Oct 03 2006
parent Lars Ivar Igesund <larsivar igesund.net> writes:
Georg Wrede wrote:

 mike wrote:
 Maybe for DMD 3.0 ...
 
 ' import std.compiler;
 '
 ' char[] source = "writefln(\"Hello World!\");";
 ' void delegate() foo = compile(source);
 ' foo();
 
 That would be just awesome :-)
 
 - mike

In addition to all the obvious, I'd say this is yet another thing that's not doable in C++. Compiling anything takes simply too long. But D, it's totally different. And especially where the same program keeps compiling different stuff, then speed would become quite fast since the compiler wouldn't be discarded from memory. "Blurring" this way the distinction and chasm between runtime and compile time should give us quite interesting new possibilities and avenues.

I believe that the CS term for these possibilities are genetic programming (which might or might not be used for genetic algorithms). Genetic algorithms are quite common (although that term might not be used), and is often seen as an AI technique. Genetic programming is probably not that common, partially because it is not very obvious how it should be used; What decides how sucessful a program is without it turning into an algorithm itself? I know one company that say they use genetic programming to find search queries (in a form of query language) for biological (DNA searches and such) research, and then in an interactive environment where a researcher can tune the generational environment while looking at the search results unfold. Definately very interesting :) -- Lars Ivar Igesund blog at http://larsivi.net DSource & #D: larsivi
Oct 03 2006
prev sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
mike wrote:
 Maybe for DMD 3.0 ...
 
 ' import std.compiler;
 '
 ' char[] source = "writefln(\"Hello World!\");";
 ' void delegate() foo = compile(source);
 ' foo();
 
 That would be just awesome :-)
 

That's pretty much what SciPy's weave.blitz and weave.inline do for Python code. Some example here: http://scipy.org/PerformancePython Example: # put expression in a string expr = "u[1:-1, 1:-1] = ((u[0:-2, 1:-1] + u[2:, 1:-1])*dy2 + "\ "(u[1:-1,0:-2] + u[1:-1, 2:])*dx2)*dnr_inv" # compile the expression and run it weave.blitz(expr, check_size=0) The first time you call this you take a hit, but the result is cached (and a DLL saved on disk too), so subsequent calls are fast, as well as subsequent runs of the program. This is a big win for optimizing the inner loops of computations in Python code. 300x speed increase over the original Python loop is not uncommon. The difference in speed would clearly be less impressive in D, since D is already very fast, but still it could be useful for building native-speed optimized expressions on the fly. For instance, in a Photoshop type app you could imagine some sort of image filter that gets some parameters from the user then compiles the resulting parameters into to a binary format for fast execution. I don't think there'd be much point in using it to call "writefln" though. :-) The thing is you have to have the entire compiler and linker sitting around to make this work. But I don't see that there's anything about D that makes any of this more feasible than it would be with C++. In fact scipy.weave is compiling C++ code on the fly. --bb
Oct 03 2006