www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - UDA + Pegged AST Hack

reply "Daniel N" <ufo orbiting.us> writes:
Thanks to the wonderful UDA implementation in 2.061, I thought of 
a little hack, which may interest at least someone.

I'm using it for prototyping ctfe ast transformations together 
with: https://github.com/PhilippeSigaud/Pegged

Yes, it was possible to do similar stuff before and 
"examples/dgrammar.d" from Pegged is quite impressive, but not 
bug free.

This UDA-line-hack-way is actually significantly cleaner, because 
when using __LINE__ you are certain that there will be at least 
one [__LINE__] which is not inside a string or comment on the 
attributed line... this allows one to make a very limited 
grammar/parser which doesn't have to know/parse/understand the 
entire file.

I know my proof-of-concept doesn't handle multi-line, but it can 
be added, and doesn't matter for the sake of prototyping AST 
manipulations.

If anyone else have fun UDA hacks, considering that it's a new 
feature, please share.

import std.string;
import std.range;

struct magic
{
   [__LINE__] int var1;
   [__LINE__] int var2;
   [__LINE__] int var3;
}

enum dsrc = import(__FILE__).splitLines(KeepTerminator.yes);

string parse()
{
   string result = "";

   foreach(m; __traits(allMembers, magic))
     result ~= dsrc.drop(__traits(getAttributes, mixin("magic." ~ 
m))[0]-1).takeOne().front;

   // insert pegged parsing / transformations here.

   return result;
}

mixin("struct magic2\n{\n" ~ parse() ~"}");
Nov 28 2012
next sibling parent "Robik" <szadows gmail.com> writes:
On Wednesday, 28 November 2012 at 21:40:35 UTC, Daniel N wrote:

 Thanks to the wonderful UDA implementation in 2.061, I thought 
 of a little hack, which may interest at least someone.

 I'm using it for prototyping ctfe ast transformations together 
 with: https://github.com/PhilippeSigaud/Pegged

 Yes, it was possible to do similar stuff before and 
 "examples/dgrammar.d" from Pegged is quite impressive, but not 
 bug free.

 This UDA-line-hack-way is actually significantly cleaner, 
 because when using __LINE__ you are certain that there will be 
 at least one [__LINE__] which is not inside a string or comment 
 on the attributed line... this allows one to make a very 
 limited grammar/parser which doesn't have to 
 know/parse/understand the entire file.

 I know my proof-of-concept doesn't handle multi-line, but it 
 can be added, and doesn't matter for the sake of prototyping 
 AST manipulations.

 If anyone else have fun UDA hacks, considering that it's a new 
 feature, please share.

 import std.string;
 import std.range;

 struct magic
 {
   [__LINE__] int var1;
   [__LINE__] int var2;
   [__LINE__] int var3;
 }

 enum dsrc = import(__FILE__).splitLines(KeepTerminator.yes);

 string parse()
 {
   string result = "";

   foreach(m; __traits(allMembers, magic))
     result ~= dsrc.drop(__traits(getAttributes, mixin("magic." 
 ~ m))[0]-1).takeOne().front;

   // insert pegged parsing / transformations here.

   return result;
 }

 mixin("struct magic2\n{\n" ~ parse() ~"}");



I made two, but not as cool as yours: 
http://dpaste.dzfl.pl/32536704 http://dpaste.dzfl.pl/15e4591b
Nov 28 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
--20cf30334fc5ba98d804cf95579f
Content-Type: text/plain; charset=UTF-8

On Wed, Nov 28, 2012 at 10:40 PM, Daniel N <ufo orbiting.us> wrote:


 Thanks to the wonderful UDA implementation in 2.061, I thought of a little
 hack, which may interest at least someone.

 I'm using it for prototyping ctfe ast transformations together with:
 https://github.com/**PhilippeSigaud/Pegged<https://github.com/PhilippeSigaud/Pegged>




Aha!



 Yes, it was possible to do similar stuff before and "examples/dgrammar.d"
 from Pegged is quite impressive, but not bug free.



No kidding :)
I've a few contributors who're helping me stabilize / make the engine
better. We will tackle the D grammar afterwards. It does make a good
testing ground for when I want to push the CTFE throttle.




 This UDA-line-hack-way is actually significantly cleaner, because when
 using __LINE__ you are certain that there will be at least one [__LINE__]
 which is not inside a string or comment on the attributed line... this
 allows one to make a very limited grammar/parser which doesn't have to
 know/parse/understand the entire file.



I don't understand what you're trying to do. Could you please explain it a
bit more? UDA's a quite new and I'd be very interested in their interaction
with compile-time code manipulation.




 If anyone else have fun UDA hacks, considering that it's a new feature,
 please share.

 import std.string;
 import std.range;

 struct magic
 {
   [__LINE__] int var1;
   [__LINE__] int var2;
   [__LINE__] int var3;
 }

 enum dsrc = import(__FILE__).splitLines(**KeepTerminator.yes);

 string parse()
 {
   string result = "";

   foreach(m; __traits(allMembers, magic))
     result ~= dsrc.drop(__traits(**getAttributes, mixin("magic." ~
 m))[0]-1).takeOne().front;



What I don't get is how the [__LINE__] part in magic contains significant
info.

--20cf30334fc5ba98d804cf95579f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<br><br><div class=3D"gmail_quote">On Wed, Nov 28, 2012 at 10:40 PM, Daniel=
 N <span dir=3D"ltr">&lt;<a href=3D"mailto:ufo orbiting.us" target=3D"_blan=
k">ufo orbiting.us</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quot=
e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks to the wonderful UDA implementation in 2.061, I thought of a little =
hack, which may interest at least someone.<br>
<br>
I&#39;m using it for prototyping ctfe ast transformations together with: <a=
 href=3D"https://github.com/PhilippeSigaud/Pegged" target=3D"_blank">https:=
//github.com/<u></u>PhilippeSigaud/Pegged</a></blockquote><div><br></div><d=
iv>
Aha!</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Yes, it was possible to do similar stuff before and &quot;examples/dgrammar=
.d&quot; from Pegged is quite impressive, but not bug free.<br></blockquote=

<div><br></div><div>No kidding :)</div><div>I&#39;ve a few contributors wh=



D grammar afterwards. It does make a good testing ground for when I want to=
 push the CTFE throttle.</div>
<div><br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"=
margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">This UDA-lin=
e-hack-way is actually significantly cleaner, because when using __LINE__ y=
ou are certain that there will be at least one [__LINE__] which is not insi=
de a string or comment on the attributed line... this allows one to make a =
very limited grammar/parser which doesn&#39;t have to know/parse/understand=
 the entire file.<br>
</blockquote><div><br></div><div>I don&#39;t understand what you&#39;re try=
ing to do. Could you please explain it a bit more? UDA&#39;s a quite new an=
d I&#39;d be very interested in their interaction with compile-time code ma=
nipulation.</div>
<div><br></div><div><br></div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
If anyone else have fun UDA hacks, considering that it&#39;s a new feature,=
 please share.<br>
<br>
import std.string;<br>
import std.range;<br>
<br>
struct magic<br>
{<br>
=C2=A0 [__LINE__] int var1;<br>
=C2=A0 [__LINE__] int var2;<br>
=C2=A0 [__LINE__] int var3;<br>
}<br>
<br>
enum dsrc =3D import(__FILE__).splitLines(<u></u>KeepTerminator.yes);<br>
<br>
string parse()<br>
{<br>
=C2=A0 string result =3D &quot;&quot;;<br>
<br>
=C2=A0 foreach(m; __traits(allMembers, magic))<br>
=C2=A0 =C2=A0 result ~=3D dsrc.drop(__traits(<u></u>getAttributes, mixin(&q=
uot;magic.&quot; ~ m))[0]-1).takeOne().front;<br></blockquote><div><br></di=
v><div>What I don&#39;t get is how the [__LINE__] part in magic contains si=
gnificant info.</div>
<div><br></div><div><br></div></div>

--20cf30334fc5ba98d804cf95579f--
Nov 28 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
--20cf300e4befdf322c04cf958f9c
Content-Type: text/plain; charset=UTF-8


 I made two, but not as cool as yours: http://dpaste.dzfl.pl/32536704




I like this one :)

I think this part:

static T readArgs(T)(string[] args)
{
T ret;
pragma(msg, "getopt(args"~buildGetOptArguments!(T, "ret")~");");
mixin("getopt(args"~buildGetOptArguments!(T, "ret")~");");
 return ret;
}


Could be slightly simplified somewhat. Try passing 'ret' as an alias. That
way, buildGetOptArguments should be able to create the "ret" string by
itself, like this:


private static string buildGetOptArguments(T, alias instance)()
{
    enum string instanceName = instance.stringof; // for example
...

And I'd have buildGetOptArguments return the "getopt(args" ... ")" part
also.
Which gives:

T readArgs(T)(string[] args)
{
T ret;
pragma(msg, buildGetOptArguments!(T, ret));
mixin(buildGetOptArguments!(T, ret)~";");
 return ret;
}

--20cf300e4befdf322c04cf958f9c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I made two, but not as cool as yours: <a href=3D"http://dpaste.dzfl.pl/3253=
6704" target=3D"_blank">http://dpaste.dzfl.pl/32536704</a> </blockquote><di=
v><br></div><div>I like this one :)</div><div><br></div><div>I think this p=
art:</div>
<div><br></div><div><div>static T readArgs(T)(string[] args)</div><div>{</d=
iv><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>T =
ret;</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</=
span>pragma(msg, &quot;getopt(args&quot;~buildGetOptArguments!(T, &quot;ret=
&quot;)~&quot;);&quot;);</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>mixin=
(&quot;getopt(args&quot;~buildGetOptArguments!(T, &quot;ret&quot;)~&quot;);=
&quot;);</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre"=

	</span></div>



n ret;</div><div>}</div></div><div>=C2=A0</div><div><br></div><div>Could be=
 slightly simplified somewhat. Try passing &#39;ret&#39; as an alias.=C2=A0=
That way, buildGetOptArguments should be able to create the &quot;ret&quot;=
 string by itself, like this:</div>
<div><br></div><div><br></div><div>private static string buildGetOptArgumen=
ts(T, alias instance)()</div><div>{</div><div>=C2=A0 =C2=A0 enum string ins=
tanceName =3D instance.stringof; // for example</div><div>...</div><div><br=

</div><div>



s&quot; ... &quot;)&quot; part also.</div><div>Which gives:</div><div><br><=
/div><div><div><div>T readArgs(T)(string[] args)</div><div>{</div><div><spa=
n class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>T ret;</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>pragm=
a(msg, buildGetOptArguments!(T, ret));</div><div><span class=3D"Apple-tab-s=
pan" style=3D"white-space:pre">	</span>mixin(buildGetOptArguments!(T, ret)~=
&quot;;&quot;);</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span></div=

<div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	</span>retu=




</div>




--20cf300e4befdf322c04cf958f9c--
Nov 28 2012
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 28 November 2012 at 21:40:35 UTC, Daniel N wrote:

 Thanks to the wonderful UDA implementation in 2.061, I thought 
 of a little hack, which may interest at least someone.

 I'm using it for prototyping ctfe ast transformations together 
 with: https://github.com/PhilippeSigaud/Pegged

 Yes, it was possible to do similar stuff before and 
 "examples/dgrammar.d" from Pegged is quite impressive, but not 
 bug free.

 This UDA-line-hack-way is actually significantly cleaner, 
 because when using __LINE__ you are certain that there will be 
 at least one [__LINE__] which is not inside a string or comment 
 on the attributed line... this allows one to make a very 
 limited grammar/parser which doesn't have to 
 know/parse/understand the entire file.

 I know my proof-of-concept doesn't handle multi-line, but it 
 can be added, and doesn't matter for the sake of prototyping 
 AST manipulations.

 If anyone else have fun UDA hacks, considering that it's a new 
 feature, please share.

 import std.string;
 import std.range;

 struct magic
 {
   [__LINE__] int var1;
   [__LINE__] int var2;
   [__LINE__] int var3;
 }

 enum dsrc = import(__FILE__).splitLines(KeepTerminator.yes);

 string parse()
 {
   string result = "";

   foreach(m; __traits(allMembers, magic))
     result ~= dsrc.drop(__traits(getAttributes, mixin("magic." 
 ~ m))[0]-1).takeOne().front;

   // insert pegged parsing / transformations here.

   return result;
 }

 mixin("struct magic2\n{\n" ~ parse() ~"}");



Here we go, annotation with strings. The feature is not even out 
that its quirks already shows up.

You take the least resistance path, and this is comprehensible, 
but anotationg with a string is known to be a bad idea. It have 
been mentionned in UDA and what you did here is the perfect 
example : when the least resistance path is something plain 
wrong, people will do it anyway.

Can someone remember me why this ended up in master ? This 
feature is clearly not ready.
Nov 28 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-29 01:16, deadalnix wrote:


 Can someone remember me why this ended up in master ? This feature is
 clearly not ready.



Because that's what Walter do, committing new features to the master 
branch willy nilly.

-- 
/Jacob Carlborg
Nov 28 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-11-29 10:56, Max Samukha wrote:


 I want to remind you that there is still no consensus about whether
 unconstrained attributes are good or bad.



That's why it's not good to have it in master, the whole feature.

-- 
/Jacob Carlborg
Nov 29 2012
prev sibling next sibling parent "Max Samukha" <maxsamukha gmail.com> writes:
On Thursday, 29 November 2012 at 07:37:31 UTC, Jacob Carlborg 
wrote:

 On 2012-11-29 01:16, deadalnix wrote:


 Can someone remember me why this ended up in master ? This 
 feature is
 clearly not ready.



 Because that's what Walter do, committing new features to the 
 master branch willy nilly.



I want to remind you that there is still no consensus about 
whether unconstrained attributes are good or bad.
Nov 29 2012
prev sibling next sibling parent "Daniel N" <ufo orbiting.us> writes:
On Thursday, 29 November 2012 at 09:56:29 UTC, Max Samukha wrote:

 On Thursday, 29 November 2012 at 07:37:31 UTC, Jacob Carlborg 
 wrote:

 On 2012-11-29 01:16, deadalnix wrote:


 Can someone remember me why this ended up in master ? This 
 feature is
 clearly not ready.



 Because that's what Walter do, committing new features to the 
 master branch willy nilly.



 I want to remind you that there is still no consensus about 
 whether unconstrained attributes are good or bad.



Furthermore I intentionally used __LINE__ (int), as an 
optimization, among other things; it results in reduced memory 
consumption during CTFE.

In my case I only intend to use the magic type inside the same 
source file to generate another type(magic2) which I then will 
expose externally, thus it's perfectly safe. Magic will be 
discarded and never escape, nor be used for anything.
Nov 29 2012
prev sibling next sibling parent "Daniel N" <ufo orbiting.us> writes:
On Wednesday, 28 November 2012 at 21:43:40 UTC, Robik wrote:

 I made two, but not as cool as yours: 
 http://dpaste.dzfl.pl/32536704 http://dpaste.dzfl.pl/15e4591b



awesome, this is really nice inspiration too, thanks! :)
Nov 29 2012
prev sibling next sibling parent "Daniel N" <ufo orbiting.us> writes:
On Wednesday, 28 November 2012 at 22:04:14 UTC, Philippe Sigaud 
wrote:

 I don't understand what you're trying to do. Could you please 
 explain it a
 bit more? UDA's a quite new and I'd be very interested in their 
 interaction
 with compile-time code manipulation.



I'm trying to find a safe spot in the middle of a file, where I 
can start parsing, normally you have to parse the entire file to 
understand it, ex with your grammar files, there are many 
keywords inside huge strings, so basically I let the compiler 
handle comments and white-space parsing, which allows to me to 
make a stable localized parser for a subset of the language.


 What I don't get is how the [__LINE__] part in magic contains 
 significant
 info.



It allows me to retrieve the entire original source for a 
declaration... it kinda emulates a ".sourceof" trait / property. 
:)
Nov 29 2012
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
--20cf3074b298a4577104cfa796bf
Content-Type: text/plain; charset=UTF-8


 I'm trying to find a safe spot in the middle of a file, where I can start
 parsing, normally you have to parse the entire file to understand it, ex
 with your grammar files, there are many keywords inside huge strings, so
 basically I let the compiler handle comments and white-space parsing, which
 allows to me to make a stable localized parser for a subset of the language.




OK, this I get.



  What I don't get is how the [__LINE__] part in magic contains significant

 info.



 It allows me to retrieve the entire original source for a declaration...
 it kinda emulates a ".sourceof" trait / property. :)



Yeah but... What does this have to do with __LINE__? I'm sorry if I'm dumb
here, I still haven't used UDAs.

--20cf3074b298a4577104cfa796bf
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<br><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=3D=
"im">
<br></div>
I&#39;m trying to find a safe spot in the middle of a file, where I can sta=
rt parsing, normally you have to parse the entire file to understand it, ex=
 with your grammar files, there are many keywords inside huge strings, so b=
asically I let the compiler handle comments and white-space parsing, which =
allows to me to make a stable localized parser for a subset of the language=
.</blockquote>
<div><br></div><div>OK, this I get.</div><div>=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex"><div class=3D"im"><br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
What I don&#39;t get is how the [__LINE__] part in magic contains significa=
nt<br>
info.<br>
</blockquote>
<br></div>
It allows me to retrieve the entire original source for a declaration... it=
 kinda emulates a &quot;.sourceof&quot; trait / property. :)<br></blockquot=
e><div><br></div><div>Yeah but... What does this have to do with __LINE__? =
I&#39;m sorry if I&#39;m dumb here, I still haven&#39;t used UDAs.</div>
<div>=C2=A0</div></div><br>

--20cf3074b298a4577104cfa796bf--
Nov 29 2012
prev sibling parent "Rob T" <rob ucora.com> writes:
On Thursday, 29 November 2012 at 00:16:28 UTC, deadalnix wrote:

 Can someone remember me why this ended up in master ? This 
 feature is clearly not ready.



Please, read this thread, esp towards the end.

"Breaking D2 language/spec changes with D1 being discontinued in 
a month"

The current lack of a sane development and release process is 
messing up a lot of people like myself. I cannot take D seriously 
for commercial usage if I cannot rely on at least 3 versions of 
the compiler. We need one for stable, another for testing/release 
candidate (these two I would rely on), and a 3rd for adding in 
new stuff for the compiler + lib devs. However there probably 
should be a forth "experimental" branch for people like Walter 
who perform experimental work, such as the spontaneous 
introduction of UDA's.

Debian follows a similar system, and is a clear real-world 
example of a very good development and release process.

Currently, I'm concluding that I simply cannot rely on D for 
anything but experimental tinkering, which means back to C++ for 
the commercial work that I do for a living, and that's a real 
shame! D has a lot of potential, but it's being undermined by a 
willy-nilly process, which honestly speaking boils down to 
decisions being made by the current management at the helm.

--rt
Nov 29 2012