www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Compile time data structure

reply Marek Janukowicz <marek janukowicz.net> writes:
I need to gather some data in compile time basing on UDA.

struct Attr {
  string name;
}

mixin template Model() {

  static string[string] columns () {
    string[string] cols;
    alias type = typeof(this);
     // Basically - get all members with  Attr UDA and build AA out of those
    foreach( field; __traits(derivedMembers, type)) {
      static if( is(typeof(__traits(getAttributes, __traits(getMember, type, 
field))) blah)) {
        foreach( uda; __traits(getAttributes, __traits(getMember, type, 
field))) {
          static if (is (typeof(uda) == Attr)) {
            static if (uda.name == "") cols[field] = field;
            else cols[field] = uda.name;
          }
        }
      }
    }
    return cols;
  }
}

class ExampleModel {

   Attr() int id;
   Attr( "field_id" ) int fieldId;

  mixin Model;
}

This works and the result of columns() method is as expected. However, if I 
try to use it in another compile time fuction, eg:

    foreach( attr, col; columns ) {
      __traits(getMember, obj, attr) = 1;
    }

it fails saying "attr" cannot be read at compile time. 

I suspect I can't just build AA during compile time, but is there any other 
way for columns() method to return a structure that could be further 
evaluated at compile time?

-- 
Marek Janukowicz
Sep 15 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Could you please provide complete code.

Thank you,
Ali
Sep 15 2013
parent reply Marek Janukowicz <marek janukowicz.net> writes:
Ali Çehreli wrote:

 Could you please provide complete code.


Sure. This is of course stripped down just for demonstration purposes:

struct Attr {
  string name;
}

mixin template Model() {

  static string[string] columns () {
    string[string] cols;
    alias type = typeof(this);
     // Basically - get all members with  Attr UDA and build AA out of those
    foreach( field; __traits(derivedMembers, type)) {
      static if( is(typeof(__traits(getAttributes, __traits(getMember, type,
field))) blah)) {
        foreach( uda; __traits(getAttributes, __traits(getMember, type,
field))) {
          static if (is (typeof(uda) == Attr)) {
            static if (uda.name == "") cols[field] = field;
            else cols[field] = uda.name;
          }
        }
      }
    }
    return cols;
  }

  alias typeof(this) Me;

  static Me initialize () {
    Me me = new Me();
    foreach( attr, col; columns() ) {
      __traits(getMember, me, attr) = typeof(__traits(getMember, me, 
attr)).init;
    }
    return me;
  }

}

class ExampleModel {

   Attr() int id;
   Attr( "field_id" ) int fieldId;

  mixin Model;
}


void main () {
  ExampleModel ex = ExampleModel.initialize();
}

-- 
Marek Janukowicz
Sep 16 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/16/2013 01:24 PM, Marek Janukowicz wrote:


    static string[string] columns () {


// ...

    }


Although the function itself is static, it returns a dynamic value.


      foreach( attr, col; columns() ) {
        __traits(getMember, me, attr) = typeof(__traits(getMember, me,
 attr)).init;
      }


That foreach is a run-time foreach because columns()'s return value is a 
run-time value.

As far as I know, static foreach is only for tuples (or TypeTuples). If 
you can generate the AA as a tuple, then the foreach will be evaluated 
at compile time.

Ali
Sep 17 2013
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 18 September 2013 at 01:24:37 UTC, Ali Çehreli 
wrote:

 As far as I know, static foreach is only for tuples (or 
 TypeTuples)


TypeTuples are tuples. Sortof. We really need to get that whole 
situation sorted....
Sep 17 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Sep 18, 2013 at 04:12:01AM +0200, John Colvin wrote:

 On Wednesday, 18 September 2013 at 01:24:37 UTC, Ali Çehreli wrote:

As far as I know, static foreach is only for tuples (or TypeTuples)


 
 TypeTuples are tuples. Sortof. We really need to get that whole
 situation sorted....


Not to mention there are also "parameter tuples" that behave sorta like
TypeTuples but not quite.


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi
Sep 17 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/17/2013 07:26 PM, H. S. Teoh wrote:


 On Wed, Sep 18, 2013 at 04:12:01AM +0200, John Colvin wrote:

 On Wednesday, 18 September 2013 at 01:24:37 UTC, Ali Çehreli wrote:

 As far as I know, static foreach is only for tuples (or TypeTuples)


 TypeTuples are tuples. Sortof. We really need to get that whole
 situation sorted....


 Not to mention there are also "parameter tuples" that behave sorta like
 TypeTuples but not quite.


I have just published two chapters that present tuples and TypeTuple 
sufficiently-completely and sufficiently-consistently (of course, 
according to me ;) ):

Tuples:

   http://ddili.org/ders/d.en/tuples.html

More Templates:

   http://ddili.org/ders/d.en/templates_more.html

I haven't officially announced those yet. I appreciate any review. (I am 
sure there are lots of English grammar and syntax issues as well. :-/ 
Those mature over time organically. :) )

Thank you,
Ali
Sep 19 2013
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 19 September 2013 at 07:46:41 UTC, Ali Çehreli wrote:

 I have just published two chapters that present tuples and 
 TypeTuple sufficiently-completely and sufficiently-consistently 
 (of course, according to me ;) ):

 Tuples:

   http://ddili.org/ders/d.en/tuples.html

 More Templates:

   http://ddili.org/ders/d.en/templates_more.html

 I haven't officially announced those yet. I appreciate any 
 review. (I am sure there are lots of English grammar and syntax 
 issues as well. :-/ Those mature over time organically. :) )

 Thank you,
 Ali


Some obvious catches:

http://ddili.org/ders/d.en/tuples.html : `S.tupleof[i].stringof` 
value was recently changed.

`object.tupleof` part does not mention important fact that tuple 
does not contain values of object members but actual members as 
if it is a reference.

There are few imprecise wordings but I am afraid being 100% 
precise when speaking about tuples will result in "WTF, what dark 
magic this is?" :(

http://ddili.org/ders/d.en/templates_more.html : "The following 
code instantiates the template for int and dchar" - `double` is 
actually used in snippet instead of dchar.

Variadic template arg chapter should probably mention "variadic 
args of length 1" idiom used to have parameter accepting types, 
values and aliases at once.
Sep 19 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/19/2013 04:32 AM, Dicebot wrote:


 Some obvious catches:


Thank you. I have made those changes except the following one.


 Variadic template arg chapter should probably mention "variadic args of
 length 1" idiom used to have parameter accepting types, values and
 aliases at once.


Could you please expand on that.

Thank you,
Ali
Sep 19 2013
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 20 September 2013 at 06:20:09 UTC, Ali Çehreli wrote:

 On 09/19/2013 04:32 AM, Dicebot wrote:


 Some obvious catches:


 Thank you. I have made those changes except the following one.


 Variadic template arg chapter should probably mention


 "variadic args of

 length 1" idiom used to have parameter accepting types,


 values and

 aliases at once.


 Could you please expand on that.

 Thank you,
 Ali


Quoting Phobos:

```
template fullyQualifiedName(T...)
     if (T.length == 1)
{
     static if (is(T[0]))
         enum fullyQualifiedName = 
fullyQualifiedNameImplForTypes!(T[0], false, false, false, false);
     else
         enum fullyQualifiedName = 
fullyQualifiedNameImplForSymbols!(T[0]);
}
```

It is a relatively common idiom because there is no other way to 
express a single template parameter that accepts anything.
Sep 20 2013
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 19, 2013 at 12:46:39AM -0700, Ali Çehreli wrote:

 On 09/17/2013 07:26 PM, H. S. Teoh wrote:
 

 On Wed, Sep 18, 2013 at 04:12:01AM +0200, John Colvin wrote:

 On Wednesday, 18 September 2013 at 01:24:37 UTC, Ali Çehreli wrote:

 As far as I know, static foreach is only for tuples (or TypeTuples)


 TypeTuples are tuples. Sortof. We really need to get that whole
 situation sorted....


 Not to mention there are also "parameter tuples" that behave sorta
 like TypeTuples but not quite.


 
 I have just published two chapters that present tuples and TypeTuple
 sufficiently-completely and sufficiently-consistently (of course,
 according to me ;) ):
 
 Tuples:
 
   http://ddili.org/ders/d.en/tuples.html
 
 More Templates:
 
   http://ddili.org/ders/d.en/templates_more.html
 
 I haven't officially announced those yet. I appreciate any review.
 (I am sure there are lots of English grammar and syntax issues as
 well. :-/ Those mature over time organically. :) )


[...]

I didn't read everything in detail but skimming over it, I think the
chapters look OK.

But they didn't describe parameter tuples -- and I'm not sure if you
want to, because it's very dark arcane magic. But if you *really* want
to know:

A parameter tuple is the type of Params in the following code:

	import std.stdio;
	
	// Function to analyze
	int func(string x, int y=123, float z=1.618) {
		return 0;
	}
	
	// Prints out types, names, and default arguments of func.
	void main() {
		static if (is(typeof(func) Params == __parameters)) {
			foreach (i, param; Params) {
				writefln("[%d]:", i);
				writeln("\tType: ", param.stringof);
				writeln("\tName: ", __traits(identifier, Params[i..i+1]));
	
				auto getDefArg(int i)(Params[i..i+1] args) {
					return args[0];
				}
				static if (is(typeof(getDefArg!i())))
					writeln("\tDefault value: ", getDefArg!i());
			}
		}
	}

This code looks pretty innocent until you start looking at it closer.

For starters, the first line in main() must be written *exactly* like
that, because the magic keyword __parameters *only* works in this exact
invocation of is(), and doesn't work anywhere else in the language. This
must be one of the darkest corners of is() syntax, because there's
basically no consistent rationalization of what each element in `is(X Y
== Z)` actually means. Each different Z changes the interpretation of X
and Y in unpredictable ways (cf. the language spec on dlang.org, under
Expressions -> isExpression -> 6).

Then there's the main question of what exactly Params is. It's what the
docs call a "parameter tuple", but what is that? Well, first of all,
contrary to what one might expect, calling foreach over Params does NOT
actually iterate over its actual elements. Rather, it only iterates over
the *types* of each func's parameters (i.e., string, int, float).

This is why in the next line, `param.stringof` prints only the type name
of the parameter.

So how does one get the name of the parameter? Well, a first thought
might be, since foreach doesn't give us actual elements in Params, maybe
array-indexing notation (i.e., Params[i]) might? Wrong! Params[i] also
returns only the *type* of the parameter. :)  It turns out that the only
way to actually get an element of Params that isn't reduced to just a
type is to take a 1-element slice of it. That's why we have to write
Params[i..i+1] in the code. Are you confused yet? :) Once we have it,
though, how do we get the parameter name? Well, it turns out that
Params[i..i+1].stringof returns strings of the form "(int y = 123)"
which we'd have to manually parse. To avoid needing manual string
parsing, we have to use __traits(identifier...) to get the parameter
name. (Of course! Isn't that obvious!?)

Finally, what about default arguments? We *could* in theory parse
Params[i..i+1].stringof to extract it as a string, and then use mixin()
to get at its actual value... but I don't feel very confident that my
string parsing code would actually be correct in all possible cases, so
I'd rather have the compiler tell me what the default value is directly.
Except that there is no way to directly get the default value at all!
The only way (and this I learned from Phobos, since it's pretty much
impossible for anybody to guess on their own) is to declare a function
using Params[i..i+1] -- which, being a parameter type tuple, obviously
can be used to declare function parameters -- that returns the value of
the default parameter.

The additional (int i) compile-time argument was added to appease the
compiler, because otherwise each iteration of the foreach loop would
declare a different function body under the same name "getDefArg", which
wouldn't work. So we templatize the function with the loop index in
order to generate unique names for its various incarnations.

Now, how do we know if the parameter has a default value? Well, it has a
default value if getDefArg!i can be called with no arguments, you see,
since writing getDefArg(int i)(Params[i..i+1] args) is equivalent to
writing getDefArg(int i)(int y=123) when Params[i..i+1] represents the
parameter `int y=123`. So if the default parameter is present, getDefArg
can be called with no arguments. So that's what the static if checks
for.

But here, `args` binds not to the parameter itself, but to the 1-element
tuple representing the argument list, so to get the actual default
value, we have to return args[0], not just args. (Of course! Isn't that
obvious??!)


And that concludes today's lesson on parameter tuples. I hope you're
thoroughly confused and utterly perplexed by now, because next class,
we're going to talk about how to extract parameter qualifiers (like ref,
scope, etc.) from a parameter tuple when the language actually has no
API to directly fetch this information. :-P

(On a more serious note, though: all of the above probably should *not*
be any D textbook; readers should instead be directed to std.traits
where all of this arcane black magic is wrapped by a nicer user-facing
API that isn't insanity-inducing.)


T

-- 
Маленькие детки - маленькие бедки.
Sep 19 2013
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/19/2013 12:53 PM, H. S. Teoh wrote:


 I hope you're
 thoroughly confused and utterly perplexed by now


No, because you explained it very well. :)


 readers should instead be directed to std.traits


Indeed, I will add some :) of this information to the Traits chapter 
which comes later in the book.

Ali
Sep 19 2013
prev sibling parent reply Marek Janukowicz <marek janukowicz.net> writes:
Ali Çehreli wrote:


 On 09/16/2013 01:24 PM, Marek Janukowicz wrote:
 
  >    static string[string] columns () {
 // ...
  >    }
 
 Although the function itself is static, it returns a dynamic value.
 
  >      foreach( attr, col; columns() ) {
  >        __traits(getMember, me, attr) = typeof(__traits(getMember, me,
  > attr)).init;
  >      }
 
 That foreach is a run-time foreach because columns()'s return value is a
 run-time value.
 
 As far as I know, static foreach is only for tuples (or TypeTuples). If
 you can generate the AA as a tuple, then the foreach will be evaluated
 at compile time.


I read your articles about tuples and templates and it all now makes more 
sense (why is your book not linked on dlang.org? It is much better stuff for 
beginners than official D docs). However, there is still one thing I 
struggle with: how do I create a tuple at compile time if I'm getting 
information I want to put into it in a foreach? All the examples I found 
create a tuple with all it's attributes available, while I get mine in an 
iterative manner...

-- 
Marek Janukowicz
Sep 19 2013
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Sep 19, 2013 at 09:12:10PM +0200, Marek Janukowicz wrote:

 Ali Çehreli wrote:


[...]

 As far as I know, static foreach is only for tuples (or TypeTuples).
 If you can generate the AA as a tuple, then the foreach will be
 evaluated at compile time.


 
 I read your articles about tuples and templates and it all now makes
 more sense (why is your book not linked on dlang.org? It is much
 better stuff for beginners than official D docs). However, there is
 still one thing I struggle with: how do I create a tuple at compile
 time if I'm getting information I want to put into it in a foreach?
 All the examples I found create a tuple with all it's attributes
 available, while I get mine in an iterative manner...


[...]

Hmm. That's a hard one. How do you generate your attributes? My first
impression -- though I could be wrong -- is that you can't modify a type
tuple dynamically, that is, once created, a tuple can't be changed, you
can only make new tuples out of it. So first, you'd have to factor out
your generated attributes so that they can be generated by, for example,
instantiating a template with an integer index argument.

For example, say you define a template like this:

	template GenerateElement(int i) {
		// This is just an example, the point is that the
		// template returns something different depending on the
		// index i.
		static if (i==0)
			alias GenerateElement = byte;
		else static if (i==1)
			alias GenerateElement = short;
		else static if (i==2)
			alias GenerateElement = int;
		else static if (i==3)
			alias GenerateElement = float;
		else static if (i==4)
			alias GenerateElement = double;
		else static assert(0, "i is out of range");
	}

Then say you want to assemble a tuple out of a given set of indices, so
that, for example, MakeTupleFromIndices!(0,2,3) would return (byte, int,
float), and MakeTupleFromIndices!(1,4) would return (short, double). You
would then write it like this:

	template MakeTupleFromIndices(indices...) {
		alias MakeTupleFromIndices =
			TypeTuple!(GenerateElement!(indices[0]),
				MakeTupleFromIndices!(indices[1..$]));
	}

	static assert(is(MakeTupleFromIndices!(0, 2, 4) ==
			TypeTuple!(byte, int, double)));

This is a recursive template that assembles the final tuple
piece-by-piece. Each piece is looked up by instantiating
GenerateElement, which you can adapt to do whatever it is you need to do
to determine the desired type.

Once you have the TypeTuple, you can then use it to create Tuples of the
desired type:

	// Same as saying: alias Tuple1 = Tuple!(int, int, float);
	alias Tuple1 = Tuple!(MakeTupleFromIndices!(2, 2, 3));
	Tuple1 t1 = tuple(1, 2, 1.0);
	Tuple1 t2 = tuple(0, 1, 5.5);

	// Same as saying: alias Tuple2 = Tuple!(double, byte);
	alias Tuple2 = Tuple!(MakeTupleFromIndices!(4, 0));
	Tuple2 u1 = tuple(3.14159, 0xff);
	Tuple2 u2 = tuple(1.61803, 0xfe);

Of course, if you want to generate the values at compile-time as well,
then you'll have to use value tuples (which are actually the same as
TypeTuples -- the compiler built-in tuples can contain both types and
values, so "TypeTuple" is actually a misnomer). The most obvious way is
to use recursive templates in the same way as above to assemble the
tuple piecemeal, for example:

	template EvenNumbers(int n) {
		static if (n==1)
			alias EvenNumbers = TypeTuple!(n*2);
		else
			// This takes advantage of auto-expansion of
			// tuples.
			alias EvenNumbers =
				TypeTuple!(EvenNumbers!(n-1), n*2);
	}

	// Prints: tuple(2, 4, 6, 8, 10)
	pragma(msg, EvenNumbers!5);

Hope this helps.


T

-- 
Those who've learned LaTeX swear by it. Those who are learning LaTeX swear at
it. -- Pete Bleackley
Sep 19 2013
prev sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/19/2013 12:12 PM, Marek Janukowicz wrote:


 (why is your book not linked on dlang.org?


It will appear there once the translation is complete.


 how do I create a tuple at compile time if I'm getting
 information I want to put into it in a foreach? All the
 examples I found create a tuple with all it's attributes
 available, while I get mine in an iterative manner...


I think the automatic expansion of a TypeTuple as parameter list may be 
used in a some sort of recursive templates:

import std.typetuple;

void main()
{
     alias t0 = TypeTuple!(1, "hello");
     alias t1 = TypeTuple!(t0, 2.3);    // Adds to t0

     pragma(msg, t1);    // prints tuple(1, "hello", 2.3)
}

Ali
Sep 19 2013