www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is D suitable for my latest project?

reply "chris stevens" <a a.com> writes:
Hi All,

I am considering using D for my latest project and there are a 
few features I would like and am not entirely sure at this point 
whether D has them. They are:

- dynamic creation of classes/structs at runtime (think I can 
emulate this with variants/dynamic)
- dynamic compilation of code files at runtime
- some basic code creation tools

Are the above possible?

Thanks for any help,
Chris Stevens
Sep 06 2015
next sibling parent "chris stevens" <a a.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:

 - dynamic compilation of code files at runtime


I guess I could just invoke the compiler from my code for this? I 
would also like to be able to load this compiled code into the 
current process. This probably can be achieved using a simple 
loadLibrary command right?
Sep 06 2015
prev sibling next sibling parent reply "BBasile" <bb.temp gmx.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:

 - dynamic creation of classes/structs at runtime.


You have Object.factory for this. You can also use a custom 
factory based on string comparison. (with some: static 
if(condition) return new This; else static if(otherCondition) 
return new That; etc).


 - dynamic compilation of code files at runtime


use std.process to call a compiler.


 - some basic code creation tools


if you mean to generate code as string, writing them to a file, 
of course it will work in D.
Sep 06 2015
next sibling parent reply "chris stevens" <a a.com> writes:
Thanks so much for your reply.

On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:

 if you mean to generate code as string, writing them to a file, 
 of course it will work in D.


I guess you're right it wouldn't be too difficult to do it all 

some 3rd person library where you build up an object model rather 
than using strings.

Ok, all sounds good. Looks like I'm going to be coding in D.:)
Sep 06 2015
parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 6 September 2015 at 15:15:03 UTC, chris stevens wrote:

 I guess you're right it wouldn't be too difficult to do it all 

 used some 3rd person library where you build up an object model 
 rather than using strings.


Maybe with dparse you can construct an AST and later convert it 
to string.
Sep 07 2015
parent reply "chris stevens" <a a.com> writes:
On Monday, 7 September 2015 at 07:57:07 UTC, Kagamin wrote:

 On Sunday, 6 September 2015 at 15:15:03 UTC, chris stevens 
 wrote:

 I guess you're right it wouldn't be too difficult to do it all 

 used some 3rd person library where you build up an object 
 model rather than using strings.


 Maybe with dparse you can construct an AST and later convert it 
 to string.


Thanks for the reply Kagamin, just had a quick look at dparse and 
not sure how I could use it in the way you describe. Any examples 
of this?
Sep 08 2015
parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 8 September 2015 at 20:01:15 UTC, chris stevens wrote:

 Thanks for the reply Kagamin, just had a quick look at dparse 
 and not sure how I could use it in the way you describe. Any 
 examples of this?


As I understand, you wanted to build an AST tree and format it to 
string?
Sep 09 2015
parent "chris stevens" <a a.com> writes:
On Wednesday, 9 September 2015 at 08:24:06 UTC, Kagamin wrote:

 As I understand, you wanted to build an AST tree and format it 
 to string?


Thanks again. So I can build an AST and convert it to full D 
source from that? I think that's what i'm likely going need to do.
Sep 09 2015
prev sibling parent reply "chris stevens" <a a.com> writes:
On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:

 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).


I just had a look at Object.factory and this isn't actually what 
I wanted. I was looking for something that would allow me to 
create new (previously undefined) classes in D at runtime that I 
could then use with Object.factory to create instances of.

I think I can do this with Variants and dynamic, is this 
possible? Or is there another way?
Sep 08 2015
next sibling parent reply "wobbles" <grogan.colin gmail.com> writes:
On Tuesday, 8 September 2015 at 19:30:16 UTC, chris stevens wrote:

 On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:

 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).


 I just had a look at Object.factory and this isn't actually 
 what I wanted. I was looking for something that would allow me 
 to create new (previously undefined) classes in D at runtime 
 that I could then use with Object.factory to create instances 
 of.

 I think I can do this with Variants and dynamic, is this 
 possible? Or is there another way?


"Previously undefined". As far as I know, this is impossible in 
D. Thr compiler has to know how much memory to allocate/request 
and it has to know that at compiletime (else it wouldn't be the 
compiler!)


a vm that allows special instructions to make the vm compile and 
make available a class dynamically (and by extension, slowly)
Sep 08 2015
next sibling parent reply "chris stevens" <a a.com> writes:
On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:

 "Previously undefined". As far as I know, this is impossible in 
 D. Thr compiler has to know how much memory to allocate/request 
 and it has to know that at compiletime (else it wouldn't be the 
 compiler!)


http://wiki.dlang.org/Dynamic_typing

This is what I saw that made me think that I could. Have had 
another closer look and I do believe it's possible.
Sep 09 2015
parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 9 September 2015 at 10:23:55 UTC, chris stevens 
wrote:

 http://wiki.dlang.org/Dynamic_typing

 This is what I saw that made me think that I could. Have had 
 another closer look and I do believe it's possible.


These are things I wrote, so let me explain how they work: they 
do not attempt to actually run through the D compiler, but 
instead use tagged memory to check it all at runtime.

This is kinda similar to creating a class, but it has no help 
from the compiler and no static type checks.


So, let's say the user wanted to create an object with two 
integers, x and y. Code like the jsvar in the link does it with 
an associative array: var[string]. var is defined as a struct 
with a storage variable for the data, and a variable saying what 
type it is. All operations check that type variable. Then the 
names are indexed through the hash map. This is like how 
javascript engines work.

You could also do it a bit differently by making a type 
definition for the whole object and allocating the memory that 
way.



You can also go with the run the compiler and load a library 
option. There's different pros and cons for each approach.



Here's some example code you might go with the second option: a 
type definition and dynamically allocated memory:

---

struct ObjectHandle {
	immutable(ObjectDefinition) type;
	ubyte[] data;

	ubyte[] getRawDataForField(string field) {
		auto info = type.fields[field];
		auto raw = data[info.offset .. info.offset + info.ti.tsize];
		return raw;
	}

	string toString() {
		string s = type.name ~ "_dynamic(\n";
		foreach(fieldName, fieldDefinition; type.fields) {
			s ~= "\t" ~ fieldName ~ ": " ~ 
typeToString(fieldDefinition.ti, getRawDataForField(fieldName));
			s ~= "\n";

		}
		return s ~ ")";
	}
}

class ObjectDefinition {
	// OffsetTypeInfo is defined in object.d
	// http://dlang.org/phobos/object.html#OffsetTypeInfo
	//
	// These TypeInfo things are D's built-in runtime reflection.
	// We're borrowing that information and the types to create our
	// own extensions.
	OffsetTypeInfo[string] fields;

	string name;

	this(string name) {
		this.name = name;
	}

	void addField(string type, string name) {
		switch(type) {
			case "int":
				OffsetTypeInfo field;
				field.ti = typeid(int);
				field.offset = dataSize(); // you might also want to do some 
alignment here...
				fields[name] = field;
			break;
			// you can add other supported types here. You could even loop 
through a list
			// of them at compile time and generate this code btw.
			default: assert(0, "unsupported type " ~ type);
		}
	}

	int dataSize() const {
		int size = 0;
		foreach(k, v; fields)
			size += v.ti.tsize;
		return size;
	}

	// this is immutable because you don't want
	// the layout to change after you've created it.
	//
	// So it forces the factory to be called only after
	// the definition is set in stone.
	ObjectHandle factory() immutable {
		auto data = new ubyte[](this.dataSize()); // dynamically 
allocate space for our object's variables...
		foreach(k, field; fields) { // and now initialize them properly
			auto i = cast(const(ubyte)[]) field.ti.init();
			if(i.length)
				data[field.offset .. field.offset + field.ti.tsize] = i[];
		}
		return ObjectHandle(this, data); // and finally return it, 
along with a reference to this so we can identify the type later

	}
}

// Similarly to addField, doing operations on our field need to 
dispatch based on
// type. I'll only show int. Others could be added in the if/else 
chain, or also
// auto-generated by a compile time loop.
string typeToString(const(TypeInfo) t, const(ubyte)[] rawData) {
	import std.conv;
	if(t == typeid(int)) {
		int i = *(cast(int*) rawData.ptr);
		return to!string(i);
	}

	return null;
}



void main() {
	auto definition = new ObjectDefinition("MyRuntimeType");
	definition.addField("int", "x");
	definition.addField("int", "y");

	auto finalDefinition = cast(immutable) definition;
	definition = null; // we can't edit it anymore, so nulling out 
the mutable original reference


	auto object = finalDefinition.factory();

	auto rawData = object.getRawDataForField("x");
	rawData[0] = 15; // since I know it is an int, I can do this now
	// but in actual practice, you'd want to do some kind of 
conversions and type checking based on the type field
	// instead of poking raw memory

	import std.stdio;
	writeln(object.toString());
}

---



There's nothing really D specific in there so far, though the 
compile time loops would be to add more types to support.

Supporting all types would be a lot of code and tricky in points. 
Tying it into TypeInfo itself and actually making your dynamic 
class available via static interfaces and other existing 
facilities like Object.factory gets trickier.... but is doable. 
You'd need to match the field layouts in your raw data, then fill 
them in appropriately and poke the appropriate arrays in the 
runtime.

Should be a learning experience if you actually want to run with 
it!
Sep 09 2015
prev sibling parent "chris stevens" <a a.com> writes:
On Tuesday, 8 September 2015 at 21:51:03 UTC, wobbles wrote:

 "Previously undefined". As far as I know, this is impossible in 
 D. Thr compiler has to know how much memory to allocate/request 
 and it has to know that at compiletime (else it wouldn't be the 
 compiler!)


There was also this:

http://rosettacode.org/wiki/Add_a_variable_to_a_class_instance_at_runtime#D

That made me think I could emulate the functionality I need.
Sep 09 2015
prev sibling next sibling parent "BBasile" <bb.temp gmx.com> writes:
On Tuesday, 8 September 2015 at 19:30:16 UTC, chris stevens wrote:

 On Sunday, 6 September 2015 at 14:45:45 UTC, BBasile wrote:

 You have Object.factory for this. You can also use a custom 
 factory based on string comparison. (with some: static 
 if(condition) return new This; else static if(otherCondition) 
 return new That; etc).


 I just had a look at Object.factory and this isn't actually 
 what I wanted. I was looking for something that would allow me 
 to create new (previously undefined) classes in D at runtime 
 that I could then use with Object.factory to create instances 
 of.

 I think I can do this with Variants and dynamic, is this 
 possible? Or is there another way?


No, it's not possible with variants. What you want to do is 
actually complex and won't be solved here.

To create a new class instance, the runtime needs the TypeInfo 
class for the class type. Why ? because for example if in a class 
declaration you declare an int initially equal to 1, the TypeInfo 
class provide the memory layout that matches to this initital 
value.

For example this is how class instances are creates in D:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/lifetime.d#L71

---
extern (C) Object _d_newclass(const ClassInfo ci)
---

without the 'ClassInfo' parameter, you can't get the initial 
state of a class.
This is what the runtime needs to create a class instance.
Sep 08 2015
prev sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/08/2015 12:30 PM, chris stevens wrote:


 create new
 (previously undefined) classes in D at runtime that I could then use
 with Object.factory to create instances of.


SDC is a D compiler as a library but as far as I know, it's not complete 
yet:

   https://github.com/SDC-Developers/SDC

Ali
Sep 09 2015
prev sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Sunday, 6 September 2015 at 14:36:53 UTC, chris stevens wrote:

 Hi All,

 I am considering using D for my latest project and there are a 
 few features I would like and am not entirely sure at this 
 point whether D has them. They are:

 - dynamic creation of classes/structs at runtime (think I can 
 emulate this with variants/dynamic)


At runtime, no. However... if it doesn't actually need to be 
runtime and is in the code itself you can have code be generated 
at compile time.


 - dynamic compilation of code files at runtime


Only by running a compiler. But as above, if it doesn't strictly 
have to be at runtime and everything is specified in the source 
files, you can compute strings at compile-time and mix them in to 
generate code.


 - some basic code creation tools


mixin, template mixin, CTFE.


 Are the above possible?


It really depends on what you want to do. A lot of the dynamic 
code generation that goes on is done at runtime because that's 
what other languages support. In my experience a lot of those 
use-cases can be done at compile-time in D. A famous example is 
compile-time regular expressions. Unless you're writing a tool 
similar to grep, all regexes in your program are known at 
compile-time. They don't _need_ to be parsed/interpreted/compiled 
at run-time, that's just what every other language that isn't D 
does.

So, if your use-case depends on information from the file-system, 
user interaction, networking, etc. then your only choice is to 
generate text files with D code and compile it. If, on the other 
hand, it's all defined by the code you write, then D has 
excellent compile-time code generation tools at your disposal.

Atila
Sep 09 2015
parent "chris stevens" <a a.com> writes:
On Wednesday, 9 September 2015 at 09:14:30 UTC, Atila Neves wrote:

 So, if your use-case depends on information from the 
 file-system, user interaction, networking, etc. then your only 
 choice is to generate text files with D code and compile it. 
 If, on the other hand, it's all defined by the code you write, 
 then D has excellent compile-time code generation tools at your 
 disposal.


Thanks for the in depth reply. Unfortunately, yes, my use case 
depends on user input so definitions will not be known at 
compiletime. I'm keen to learn D though so I think I'm going to 
try and make it work with dynamically compiling and loading 
libraries.
Sep 09 2015