www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Templates and stringof...

reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  This is moved/copied from the D.learning group. I need opinions 
on the matter, and thoughts perhaps from Walter or Andrei.

  Here are the highlights

---

While working on bitfields code I've found a unique scenario that 
poses some annoyances when generating the code.

template XYZ(alias x) {
     enum XYZ = x.stringof ~ "=100;";
}

struct I { int i;}

I i_num;
int n;

mixin(XYZ!(i_num.i)); //cannot find variable i
mixin(XYZ!(n));

the mixins become:
i=100;
n=100;

  Now, how do I get the template's stringof to print out 'i_num.i' 
and not 'i'?

On Friday, 3 August 2012 at 21:19:08 UTC, David Nadlinger wrote:

 You don't. Using .stringof in conjunction with string mixins is 
 The Wrong Thing (tm) in virtually all cases.

 What do you want to achieve? Why can't you pass a string 
 instead?

Because a string doesn't hold it's type information for size checking. On Friday, 3 August 2012 at 22:47:52 UTC, David Nadlinger wrote:
 On Friday, 3 August 2012 at 22:23:23 UTC, Era Scarecrow wrote:
 Seems like an ugly hack though (to get this done). Why not 
 have another method of fullpathStringof or something similar? 
 Then again if this is one of the few cases that could benefit 
 from it, then maybe we should make it ugly so no one else will 
 use it.

This can't work in general. What should such a function return? The fully qualified name, I.e. including packages and modules? What is if the referred to entity is a nested function/local variable? What is if it is defined in a module which is not imported in the place where the string is mixed in?

I would think, the exact 'text' of the way the variable was called, in my example it would be "inum.i", and since it would be at the local level (where the mixin was called) it should have access to everything that it did at that time. On Friday, 3 August 2012 at 22:50:54 UTC, David Nadlinger wrote:
 No, it shouldn't be. There are almost no legit use cases for 
 it, and having it in the library encourages abuse of string 
 mixins/stringof in cases like this one.

Then doesn't it seem like we're missing a potentially important piece of the puzzle for mixins and templates? very likely my modified template will include you including the same variable twice, but if someone gets lazy then it may not work. True I can add asserts as part of the output code, but as mentioned hopefully the constraints could do that work, plus adding extra checks as part of a mixin seems a little... excessive, dirty and ugly. Let's assume we use 'localStringof' or 'callingStringof' and that it returns the string of how the variable is called/referenced; Give me some examples how it would be abused or used wrongly? Aliased variables are pretty much perfect forwarded in templates (Unless my understanding is off) so they would carry that information forward.
Aug 03 2012
next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 03:07:21 UTC, Era Scarecrow wrote:
 Then doesn't it seem like we're missing a potentially important 
 piece of the puzzle for mixins and templates? very likely my 
 modified template will include you including the same variable 
 twice, but if someone gets lazy then it may not work.

Well I have a working work-around that I need to test further, however it does the job; Although for other applications it may fail miserably on, and has very limited application. template bitfieldsOn(string storeName, T...) { enum bitfieldsOn = "mixin(bitfieldsOn_b!(\"" ~ storeName ~ "\"," ~ storeName ~ TupleToString!(T) ~ "));"; } template TupleToString(T...) { static if (T.length) enum TupleToString = "," ~ T[0].stringof ~ TupleToString!(T[1 .. $]); else enum TupleToString = ""; } so this makes... mixin(bitfieldsOn!("bits", ulong, "back", 32, ulong, "", 32)); and becomes: mixin(bitfieldsOn_b!("bits",bits,ulong,"back",32,ulong,"",32));
Aug 03 2012
prev sibling next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 03:07:21 UTC, Era Scarecrow wrote:
 This can't work in general. What should such a function 
 return? The fully qualified name, I.e. including packages and 
 modules? What is if the referred to entity is a nested 
 function/local variable? What is if it is defined in a module 
 which is not imported in the place where the string is mixed 
 in?

I would think, the exact 'text' of the way the variable was called, in my example it would be "inum.i", and since it would be at the local level (where the mixin was called) it should have access to everything that it did at that time.

How do you determine what the »local level« is? A string mixin isn't something you can »call«, just a compile-time constant string, which can be »evaluated« in a completely different place than it can be constructed. Maybe some heuristic could be used, but it's not worth the trouble, in my opinion.
  True I can add asserts as part of the output code, but as 
 mentioned hopefully the constraints could do that work, plus 
 adding extra checks as part of a mixin seems a little...  
 excessive, dirty and ugly.

Why would it be dirty? String mixins are just compile-time code generation. Also, using template constraints has almost zero benefits if overload resolution is not involved anyway – remember, this is just about the helper function/template generating the string to be mixed in.
  Then doesn't it seem like we're missing a potentially 
 important piece of the puzzle for mixins and templates? very 
 likely my modified template will include you including the same 
 variable twice, but if someone gets lazy then it may not work.

As I said, there are almost always ways to achieve what you want without resorting to getting the string representation of an alias parameter. For example: --- mixin template BitfieldsOn(alias target, <…>) if (isIntegral!(typeof(target))) { mixin({ string code; // Generate code using "target" as identifier. return code; }()); } mixin BitfieldsOn!(foo, <…>); --- David
Aug 04 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/04/2012 01:35 PM, Philippe Sigaud wrote:
 On Sat, Aug 4, 2012 at 10:53 AM, David Nadlinger<see klickverbot.at>  wrote:

 example:

 ---
 mixin template BitfieldsOn(alias target,<…>) if
 (isIntegral!(typeof(target))) {
      mixin({
          string code;
          // Generate code using "target" as identifier.
          return code;
      }());
 }

 mixin BitfieldsOn!(foo,<…>);
 ---

Yes, I recently rediscovered this trick of having a mixin integrate an anonymous delegate where code-as-string is generated. This is a very nice interaction between different parts of D. This should be explained somewhere, as it's a tool anyone using templates to do metaprogramming should be aware of.

http://d.puremagic.com/issues/show_bug.cgi?id=7653
Aug 04 2012
prev sibling next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 08:53:20 UTC, David Nadlinger wrote:
 How do you determine what the »local level« is? A string 
 mixin isn't something you can »call«, just a compile-time 
 constant string, which can be »evaluated« in a completely 
 different place than it can be constructed. Maybe some 
 heuristic could be used, but it's not worth the trouble, in my 
 opinion.

Mmm I would define it as the first one to call on any templates, at least if you plan on mixing stuff at that spot, further called/aliased stuff carries over the same information regardless how many levels deep it goes. If it began inside a template or from another function then I really don't know, probably holds the local scope from that level (or none at all).
 Why would it be dirty? String mixins are just compile-time code 
 generation. Also, using template constraints has almost zero 
 benefits if overload resolution is not involved anyway – 
 remember, this is just about the helper function/template 
 generating the string to be mixed in.

Depending on how many asserts I need to add in the string output seems a bit much if there may be more for the checks after the generation than before, or the ratio of checks in the returned string outweigh the actual code you want; Or so I would think. If all the checks are static than it won't show up in the executable, but if you need to output it out to glance over the generated code it would be quite messy to sift through it, depending on how complex it got. But I'm probably wrong.
 As I said, there are almost always ways to achieve what you 
 want without resorting to getting the string representation of 
 an alias parameter. For example:

 ---
 mixin template BitfieldsOn(alias target, <…>) if 
 (isIntegral!(typeof(target))) {
     mixin({
         string code;
         // Generate code using "target" as identifier.
         return code;
     }());
 }

 mixin BitfieldsOn!(foo, <…>);

I really don't see how that is an improvement. If the only way to use the mixin properly is the full variable location as a string (and stringof won't do) than the only acceptable input is a string; Since strings won't hold the type information... It ends up with a mixin calling another mixin.
Aug 04 2012
prev sibling next sibling parent "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 10:08:34 UTC, Era Scarecrow wrote:
 ---
 mixin template BitfieldsOn(alias target, <…>) if 
 (isIntegral!(typeof(target))) {
    mixin({
        string code;
        // Generate code using "target" as identifier.
        return code;
    }());
 }

 mixin BitfieldsOn!(foo, <…>);

I really don't see how that is an improvement.

It's a sketch of an implementation for your template which actually works with foo being passed only as an alias parameter. It's not a string mixin, but I tend to avoid exposing string mixins to user code as much as possible anyway. I'm not sure what you mean with »[…] the only acceptable input is a string; Since strings won't hold the type information...« – the input to BitfieldsOn is _not_ a string, it is an alias. The trick is that you can refer to it as »target« inside the mixin template, so the problem of obtaining the »real name« of something doesn't even arise. David
Aug 04 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 10:53 AM, David Nadlinger <see klickverbot.at> wrote=
:

 example:

 ---
 mixin template BitfieldsOn(alias target, <=E2=80=A6>) if
 (isIntegral!(typeof(target))) {
     mixin({
         string code;
         // Generate code using "target" as identifier.
         return code;
     }());
 }

 mixin BitfieldsOn!(foo, <=E2=80=A6>);
 ---

Yes, I recently rediscovered this trick of having a mixin integrate an anonymous delegate where code-as-string is generated. This is a very nice interaction between different parts of D. This should be explained somewhere, as it's a tool anyone using templates to do metaprogramming should be aware of. I'll update my tutorial on templates. Last example in line, I needed to have a gensym, a 'fresh' symbol, guaranteed not to be aleady defined in the local scope. That is, user code uses: mixin(gensym) to get a fresh identifier. D templates are becoming powerful enough that something like Lisp macros is becoming possible and, hence, hygiene problems. (Hygiene, for those reading this, is having a template use an already-defined variable and breaking local code, it's not a quip on the personal habits of Lisp hackers) Anyway, so, this gensym uses the exact same pattern.
Aug 04 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 1:23 PM, David Nadlinger <see klickverbot.at> wrote:
 On Saturday, 4 August 2012 at 10:08:34 UTC, Era Scarecrow wrote:
 ---
 mixin template BitfieldsOn(alias target, <=E2=80=A6>) if
 (isIntegral!(typeof(target))) {
    mixin({
        string code;
        // Generate code using "target" as identifier.
        return code;
    }());
 }

 mixin BitfieldsOn!(foo, <=E2=80=A6>);

I really don't see how that is an improvement.

It's a sketch of an implementation for your template which actually works with foo being passed only as an alias parameter. It's not a string mixin=

 but I tend to avoid exposing string mixins to user code as much as possib=

 anyway.

 I'm not sure what you mean with =C2=BB[=E2=80=A6] the only acceptable inp=

 Since strings won't hold the type information...=C2=AB =E2=80=93 the inpu=

 is _not_ a string, it is an alias. The trick is that you can refer to it =

 =C2=BBtarget=C2=AB inside the mixin template, so the problem of obtaining=

 name=C2=AB of something doesn't even arise.

Yes, it's a design I like: - using a standard template gives you problem when you try to 'send' the name in another module, as David demonstrated. - using a string mixin is indeed a little less 'clean', as the use might get his grubby hands on the string and then, who knows what may happen. - but using a mixin template that incorporates a string-generating block is good, as you get the full access to types and symbols and can still incorporate any D code inside the delegate/block. Who here knows that D blocks are also void delegate()'s?
Aug 04 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 3:04 PM, Timon Gehr <timon.gehr gmx.ch> wrote:

 This should be
 explained somewhere, as it's a tool anyone using templates to do
 metaprogramming should be aware of.

http://d.puremagic.com/issues/show_bug.cgi?id=7653

Yes. There is a workaround, but we lose the elegance of the previous solution. mixin template AddMembers() { private static string __AddMembers() { string r; foreach(x;["a","b","c"]) r~="int "~x~";"; return r; } mixin(__AddMembers()); } struct S { mixin AddMembers; } void main() { S s; writeln(s.a); } The limit is of course this introduce a new symbol in the local scope (here, __AddMembers, made private and static). Not that the collision risk with user code is great, the catch is if a second instantiation of an equivalent template is done, thus trying to create in the scope an already defined symbol. Hence my need for a gensym template I alluded to in a previous message. AddMembers would first create a fresh, non-already-used-here symbol and sue that as its helper function's name. And... drat, my gensym use the same trick, so back to the starting point. I'll vote this bug up, one never knows.
Aug 04 2012
prev sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 11:23:07 UTC, David Nadlinger wrote:
 I'm not sure what you mean with »[…] the only acceptable 
 input is a string; Since strings won't hold the type 
 information...« – the input to BitfieldsOn is _not_ a 
 string, it is an alias. The trick is that you can refer to it 
 as »target« inside the mixin template, so the problem of 
 obtaining the »real name« of something doesn't even arise.

Honestly i don't understand. In my code i was referring to the variable as 'target'; But when came time to return a string it needs the full name/location to refer to (as a string) to inject the code, which then the 'stringof' didn't help. I've already done a workaround that resolves the problem, we can refer back to this another time if another solution presents itself.
Aug 04 2012