www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is D suitable for my latest project?

reply "chris stevens" <a a.com> writes:
Hi All,

I am considering using D for my latest project and there are a 
few features I would like and am not entirely sure at this point 
whether D has them. They are:

- dynamic creation of classes/structs at runtime (think I can 
emulate this with variants/dynamic)
- dynamic compilation of code files at runtime
- some basic code creation tools

Are the above possible?

Thanks for any help,
Chris Stevens
Sep 06 2015
next sibling parent "chris stevens" <a a.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:
 - dynamic compilation of code files at runtime
I guess I could just invoke the compiler from my code for this? I would also like to be able to load this compiled code into the current process. This probably can be achieved using a simple loadLibrary command right?
Sep 06 2015
prev sibling next sibling parent reply "BBasile" <bb.temp gmx.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:
 - dynamic creation of classes/structs at runtime.
You have Object.factory for this. You can also use a custom factory based on string comparison. (with some: static if(condition) return new This; else static if(otherCondition) return new That; etc).
 - dynamic compilation of code files at runtime
use std.process to call a compiler.
 - some basic code creation tools
if you mean to generate code as string, writing them to a file, of course it will work in D.
Sep 06 2015
next sibling parent reply "chris stevens" <a a.com> writes:
Thanks so much for your reply.

On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:
 if you mean to generate code as string, writing them to a file, 
 of course it will work in D.
I guess you're right it wouldn't be too difficult to do it all using strings. The code generation I'd done before in c# I'd used some 3rd person library where you build up an object model rather than using strings. Ok, all sounds good. Looks like I'm going to be coding in D.:)
Sep 06 2015
parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 6 September 2015 at 15:15:03 UTC, chris stevens wrote:
 I guess you're right it wouldn't be too difficult to do it all 
 using strings. The code generation I'd done before in c# I'd 
 used some 3rd person library where you build up an object model 
 rather than using strings.
Maybe with dparse you can construct an AST and later convert it to string.
Sep 07 2015
parent reply "chris stevens" <a a.com> writes:
On Monday, 7 September 2015 at 07:57:07 UTC, Kagamin wrote:
 On Sunday, 6 September 2015 at 15:15:03 UTC, chris stevens 
 wrote:
 I guess you're right it wouldn't be too difficult to do it all 
 using strings. The code generation I'd done before in c# I'd 
 used some 3rd person library where you build up an object 
 model rather than using strings.
Maybe with dparse you can construct an AST and later convert it to string.
Thanks for the reply Kagamin, just had a quick look at dparse and not sure how I could use it in the way you describe. Any examples of this?
Sep 08 2015
parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 8 September 2015 at 20:01:15 UTC, chris stevens wrote:
 Thanks for the reply Kagamin, just had a quick look at dparse 
 and not sure how I could use it in the way you describe. Any 
 examples of this?
As I understand, you wanted to build an AST tree and format it to string?
Sep 09 2015
parent "chris stevens" <a a.com> writes:
On Wednesday, 9 September 2015 at 08:24:06 UTC, Kagamin wrote:
 As I understand, you wanted to build an AST tree and format it 
 to string?
Thanks again. So I can build an AST and convert it to full D source from that? I think that's what i'm likely going need to do.
Sep 09 2015
prev sibling parent reply "chris stevens" <a a.com> writes:
On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:
 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).
I just had a look at Object.factory and this isn't actually what I wanted. I was looking for something that would allow me to create new (previously undefined) classes in D at runtime that I could then use with Object.factory to create instances of. I think I can do this with Variants and dynamic, is this possible? Or is there another way?
Sep 08 2015
next sibling parent reply "wobbles" <grogan.colin gmail.com> writes:
On Tuesday, 8 September 2015 at 19:30:16 UTC, chris stevens wrote:
 On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:
 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).
I just had a look at Object.factory and this isn't actually what I wanted. I was looking for something that would allow me to create new (previously undefined) classes in D at runtime that I could then use with Object.factory to create instances of. I think I can do this with Variants and dynamic, is this possible? Or is there another way?
"Previously undefined". As far as I know, this is impossible in D. Thr compiler has to know how much memory to allocate/request and it has to know that at compiletime (else it wouldn't be the compiler!) This functionality in Java/C# is possible because it's running on a vm that allows special instructions to make the vm compile and make available a class dynamically (and by extension, slowly)
Sep 08 2015
next sibling parent reply "chris stevens" <a a.com> writes:
On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:
 "Previously undefined". As far as I know, this is impossible in 
 D. Thr compiler has to know how much memory to allocate/request 
 and it has to know that at compiletime (else it wouldn't be the 
 compiler!)
http://wiki.dlang.org/Dynamic_typing This is what I saw that made me think that I could. Have had another closer look and I do believe it's possible.
Sep 09 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 9 September 2015 at 10:23:55 UTC, chris stevens 
wrote:
 http://wiki.dlang.org/Dynamic_typing

 This is what I saw that made me think that I could. Have had 
 another closer look and I do believe it's possible.
These are things I wrote, so let me explain how they work: they do not attempt to actually run through the D compiler, but instead use tagged memory to check it all at runtime. This is kinda similar to creating a class, but it has no help from the compiler and no static type checks. So, let's say the user wanted to create an object with two integers, x and y. Code like the jsvar in the link does it with an associative array: var[string]. var is defined as a struct with a storage variable for the data, and a variable saying what type it is. All operations check that type variable. Then the names are indexed through the hash map. This is like how javascript engines work. You could also do it a bit differently by making a type definition for the whole object and allocating the memory that way. You can also go with the run the compiler and load a library option. There's different pros and cons for each approach. Here's some example code you might go with the second option: a type definition and dynamically allocated memory: --- struct ObjectHandle { immutable(ObjectDefinition) type; ubyte[] data; ubyte[] getRawDataForField(string field) { auto info = type.fields[field]; auto raw = data[info.offset .. info.offset + info.ti.tsize]; return raw; } string toString() { string s = type.name ~ "_dynamic(\n"; foreach(fieldName, fieldDefinition; type.fields) { s ~= "\t" ~ fieldName ~ ": " ~ typeToString(fieldDefinition.ti, getRawDataForField(fieldName)); s ~= "\n"; } return s ~ ")"; } } class ObjectDefinition { // OffsetTypeInfo is defined in object.d // http://dlang.org/phobos/object.html#OffsetTypeInfo // // These TypeInfo things are D's built-in runtime reflection. // We're borrowing that information and the types to create our // own extensions. OffsetTypeInfo[string] fields; string name; this(string name) { this.name = name; } void addField(string type, string name) { switch(type) { case "int": OffsetTypeInfo field; field.ti = typeid(int); field.offset = dataSize(); // you might also want to do some alignment here... fields[name] = field; break; // you can add other supported types here. You could even loop through a list // of them at compile time and generate this code btw. default: assert(0, "unsupported type " ~ type); } } int dataSize() const { int size = 0; foreach(k, v; fields) size += v.ti.tsize; return size; } // this is immutable because you don't want // the layout to change after you've created it. // // So it forces the factory to be called only after // the definition is set in stone. ObjectHandle factory() immutable { auto data = new ubyte[](this.dataSize()); // dynamically allocate space for our object's variables... foreach(k, field; fields) { // and now initialize them properly auto i = cast(const(ubyte)[]) field.ti.init(); if(i.length) data[field.offset .. field.offset + field.ti.tsize] = i[]; } return ObjectHandle(this, data); // and finally return it, along with a reference to this so we can identify the type later } } // Similarly to addField, doing operations on our field need to dispatch based on // type. I'll only show int. Others could be added in the if/else chain, or also // auto-generated by a compile time loop. string typeToString(const(TypeInfo) t, const(ubyte)[] rawData) { import std.conv; if(t == typeid(int)) { int i = *(cast(int*) rawData.ptr); return to!string(i); } return null; } void main() { auto definition = new ObjectDefinition("MyRuntimeType"); definition.addField("int", "x"); definition.addField("int", "y"); auto finalDefinition = cast(immutable) definition; definition = null; // we can't edit it anymore, so nulling out the mutable original reference auto object = finalDefinition.factory(); auto rawData = object.getRawDataForField("x"); rawData[0] = 15; // since I know it is an int, I can do this now // but in actual practice, you'd want to do some kind of conversions and type checking based on the type field // instead of poking raw memory import std.stdio; writeln(object.toString()); } --- There's nothing really D specific in there so far, though the compile time loops would be to add more types to support. Supporting all types would be a lot of code and tricky in points. Tying it into TypeInfo itself and actually making your dynamic class available via static interfaces and other existing facilities like Object.factory gets trickier.... but is doable. You'd need to match the field layouts in your raw data, then fill them in appropriately and poke the appropriate arrays in the runtime. Should be a learning experience if you actually want to run with it!
Sep 09 2015
prev sibling parent "chris stevens" <a a.com> writes:
On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:
 "Previously undefined". As far as I know, this is impossible in 
 D. Thr compiler has to know how much memory to allocate/request 
 and it has to know that at compiletime (else it wouldn't be the 
 compiler!)
There was also this: http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D That made me think I could emulate the functionality I need.
Sep 09 2015
prev sibling next sibling parent "BBasile" <bb.temp gmx.com> writes:
On Tuesday, 8 September 2015 at 19:30:16 UTC, chris stevens wrote:
 On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:
 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).
I just had a look at Object.factory and this isn't actually what I wanted. I was looking for something that would allow me to create new (previously undefined) classes in D at runtime that I could then use with Object.factory to create instances of. I think I can do this with Variants and dynamic, is this possible? Or is there another way?
No, it's not possible with variants. What you want to do is actually complex and won't be solved here. To create a new class instance, the runtime needs the TypeInfo class for the class type. Why ? because for example if in a class declaration you declare an int initially equal to 1, the TypeInfo class provide the memory layout that matches to this initital value. For example this is how class instances are creates in D: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/lifetime.d#L71 --- extern (C) Object _d_newclass(const ClassInfo ci) --- without the 'ClassInfo' parameter, you can't get the initial state of a class. This is what the runtime needs to create a class instance.
Sep 08 2015
prev sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/08/2015 12:30 PM, chris stevens wrote:

 create new
 (previously undefined) classes in D at runtime that I could then use
 with Object.factory to create instances of.
SDC is a D compiler as a library but as far as I know, it's not complete yet: https://github.com/SDC-Developers/SDC Ali
Sep 09 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:
 Hi All,

 I am considering using D for my latest project and there are a 
 few features I would like and am not entirely sure at this 
 point whether D has them. They are:

 - dynamic creation of classes/structs at runtime (think I can 
 emulate this with variants/dynamic)
At runtime, no. However... if it doesn't actually need to be runtime and is in the code itself you can have code be generated at compile time.
 - dynamic compilation of code files at runtime
Only by running a compiler. But as above, if it doesn't strictly have to be at runtime and everything is specified in the source files, you can compute strings at compile-time and mix them in to generate code.
 - some basic code creation tools
mixin, template mixin, CTFE.
 Are the above possible?
It really depends on what you want to do. A lot of the dynamic code generation that goes on is done at runtime because that's what other languages support. In my experience a lot of those use-cases can be done at compile-time in D. A famous example is compile-time regular expressions. Unless you're writing a tool similar to grep, all regexes in your program are known at compile-time. They don't _need_ to be parsed/interpreted/compiled at run-time, that's just what every other language that isn't D does. So, if your use-case depends on information from the file-system, user interaction, networking, etc. then your only choice is to generate text files with D code and compile it. If, on the other hand, it's all defined by the code you write, then D has excellent compile-time code generation tools at your disposal. Atila
Sep 09 2015
parent "chris stevens" <a a.com> writes:
On Wednesday, 9 September 2015 at 09:14:30 UTC, Atila Neves wrote:
 So, if your use-case depends on information from the 
 file-system, user interaction, networking, etc. then your only 
 choice is to generate text files with D code and compile it. 
 If, on the other hand, it's all defined by the code you write, 
 then D has excellent compile-time code generation tools at your 
 disposal.
Thanks for the in depth reply. Unfortunately, yes, my use case depends on user input so definitions will not be known at compiletime. I'm keen to learn D though so I think I'm going to try and make it work with dynamically compiling and loading libraries.
Sep 09 2015