www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - another idea for compile time functions

reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
I think templates is the wrong way to go for compile-time meta-programming.

There's been some talk recently about executing functions at 
compile-time by some people, because these people (including me) feel 
that template recursion and tricks are not the appropriate way to 
manipulate/parse strings for specialized DSL (domain specific languages).

I will give you some ideas I have, and I would like to know your 
thoughts about them.

To me, compile time functions should employ the same concepts behind 
"constant folding": if an expression is composed of compile time 
constants, the compiler will compute the expression at compile time and 
replace it with the result.

# int x = 2 + 3;
will be optimized by the compiler to:
# int x = 5;
Here, the compiler computed the expression "2+3" at compile time, and 
replaced the expression with its result, "5".

The same concept should be applicable to compile time functions. A 
function is a complicated set of expressions.

Some functions are trivial; they take a set of parameters, do something 
with them, without calling or depending on other functions, and return 
the result.
A "trivial" example:
# int add( int x, int y ) { return x + y; }
If they take constant arguments, such functions can surely be computed 
at compile time without much hassle. Such that a call to
# int Q = add( 10, 12 );
can be replaced, at compile time, with this:
# int Q = 22;

The trick may lie in letting the compiler recognize these kind of 
functions. A simple solution might me to come up with a new attribute; 
let's call it "meta":
# meta int add( int x, int y ) { return x + y; }
This attribute will assist the compiler in recognizing that this 
function can be computed at compile time if it's given constant arguments.

Now, this may not be very useful because of the restriction that meta 
functions must not call other functions, which is very limiting; so we 
need to extend it.

If a function takes constant arguments and does call other functions, 
then it can only be executed at compile time if all of these other 
functions are meta functions.

example:
# void subtract( int x, int y ) { return add( x, -y ); }
This function calls "add", but add is a meta function that can be 
executed at compile time, and will be replaced at compile time with:
# void subtract( int x, int y ) { return x - y ); }
So this type of function can definitely be executed at compile time, and 
thus deserves to be a meta function.

# meta void subtract( int x, int y ) { return add( x, -y ); }

The same idea could be extended to string-manipulating functions.

Using these concepts as basis for compile-time meta-programming might 
give us another benefit: we don't have to write compile-time functions 
that repeat the same tasks already done by run-time functions (regex for 
example); we just have to be careful and design our functions so that 
they can be meta functions.

The idea is that if you call a meta function with non-constant 
arguments, the compiler shouldn't complain; it will just treat the 
function call in this case as a regular function call to a runtime function.

What do you think about this?! Please tell me your opinions.
Feb 08 2007
next sibling parent Derek Parnell <derek nomail.afraid.org> writes:
On Thu, 08 Feb 2007 21:48:50 -0700, Hasan Aljudy wrote:

 I think templates is the wrong way to go for compile-time meta-programming.

 What do you think about this?! Please tell me your opinions.

It seems like it is worth considering. I'm not fully convinced the templates should be used for everything compile-time either. The syntax and paradigm seems awkward and doesn't flow naturally for me. -- Derek (skype: derek.j.parnell) Melbourne, Australia "Justice for David Hicks!" 9/02/2007 3:56:56 PM
Feb 08 2007
prev sibling next sibling parent Knud Soerensen <4tuu4k002 sneakemail.com> writes:
On Thu, 08 Feb 2007 21:48:50 -0700, Hasan Aljudy wrote:

 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new attribute; 
 let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant arguments.
 
 Now, this may not be very useful because of the restriction that meta 
 functions must not call other functions, which is very limiting; so we 
 need to extend it.
 
 If a function takes constant arguments and does call other functions, 
 then it can only be executed at compile time if all of these other 
 functions are meta functions.
 

I don't think a meta attribute is need, if the compiler detects that a function is called with only constant arguments it should try to inline the function and do constant folding on it. If the function calls another function then it should check if constant folding reduce the arguments to constant and then try to inline and folding it, if this fails it should just call the function. If a signal to the compiler is really needed then maybe ! can be used add!(2,3) would make the compiler look for the template "add!" first and then try to fold "add" if the template is not found.
Feb 08 2007
prev sibling next sibling parent reply "Andrei Alexandrescu (See Website For Email)" <SeeWebsiteForEmail erdani.org> writes:
Hasan Aljudy wrote:
[snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new attribute; 
 let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei
Feb 08 2007
next sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 Hasan Aljudy wrote:
 [snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new attribute; 
 let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant 
 arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei

Wow, I think that's just the perfect solution ..
Feb 08 2007
prev sibling next sibling parent reply janderson <askme me.com> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 Hasan Aljudy wrote:
 [snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new attribute; 
 let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant 
 arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei

Although I like this idea, I fear it will not work for anything hidden in a library (if you are using something that is moved to a .lib, your code will stop working). Maybe that's ok. Actually now I think about it, that would be safer, because even if the D compiler could put an "ok" signatures in a library, someone could create fake signatures. It could get confusing if you don't know which functions will work and which won't. Perhaps the compiler could help there (spit out a list of functions u can use or something). I think the compiler would compile these commands on demand and cache them so it doesn't need to do them every time. That would help a lot. It could even cache results so it only needs to compute them once. Anyways, there i think there is so much possibility with compile time coding. -joel
Feb 08 2007
parent reply "Andrei Alexandrescu (See Website For Email)" <SeeWebsiteForEmail erdani.org> writes:
janderson wrote:
 Andrei Alexandrescu (See Website For Email) wrote:
 Hasan Aljudy wrote:
 [snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new 
 attribute; let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant 
 arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei

Although I like this idea, I fear it will not work for anything hidden in a library (if you are using something that is moved to a .lib, your code will stop working). Maybe that's ok. Actually now I think about it, that would be safer, because even if the D compiler could put an "ok" signatures in a library, someone could create fake signatures.

Good point. This is reasonable. To execute code during compilation it's reasonable to expect transparency. C++ commercial vendors did not really suffer financial loss due to this requirement, and at any rate, OSS is on the rise :o).
 It could get confusing if you don't know which functions will work and 
 which won't.  Perhaps the compiler could help there (spit out a list of 
 functions u can use or something).

Yes, that's a documentation issue. The nice thing is that a plethora of really useful functions (e.g. string manipulation) can be written in D's subset that can be interpreted. Pretty much the entire string library will be meta-executable. No more need for the metastrings lib!
 I think the compiler would compile these commands on demand and cache 
 them so it doesn't need to do them every time.  That would help a lot. 
 It could even cache results so it only needs to compute them once.
 
 Anyways, there i think there is so much possibility with compile time 
 coding.

Me too. Once compile-time interpretation (and mutation) makes it in, I think there's no fear of efficiency loss anymore. Speed will be comparable to that of any interpreted code, and there are entire communities that don't have a problem with that. The question is, what is the subset of D that can be interpreted? I'm thinking: * Basic data types (they will be stored as a dynamically-typed variant anyway), except pointers to functions and delegates. * Arrays and hashes * Basic expressions (except 'is', 'delete' et al.) * if, for, foreach, while, do, switch, continue, break, return I'm constructing this list thinking what it takes to write basic data manipulation functions. What did I forget? Andrei
Feb 08 2007
next sibling parent reply Reiner Pope <xxxx xxx.xxx> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 The question is, what is the subset of D that can be interpreted? I'm 
 thinking:
 
 * Basic data types (they will be stored as a dynamically-typed variant 
 anyway), except pointers to functions and delegates.

need to support them for user-written foreach's, and (as long as they are written in an alias-free form) they are const-foldable.
 I'm constructing this list thinking what it takes to write basic data 
 manipulation functions. What did I forget?

virtual functions. Is there something tricky with them that I'm missing? Cheers, Reiner
Feb 08 2007
parent "Andrei Alexandrescu (See Website For Email)" <SeeWebsiteForEmail erdani.org> writes:
Reiner Pope wrote:
 Andrei Alexandrescu (See Website For Email) wrote:
 The question is, what is the subset of D that can be interpreted? I'm 
 thinking:

 * Basic data types (they will be stored as a dynamically-typed variant 
 anyway), except pointers to functions and delegates.

need to support them for user-written foreach's, and (as long as they are written in an alias-free form) they are const-foldable.
 I'm constructing this list thinking what it takes to write basic data 
 manipulation functions. What did I forget?

virtual functions. Is there something tricky with them that I'm missing?

I guess I'm just trying to simplify Walter's life :o). Andrei
Feb 08 2007
prev sibling next sibling parent reply Kevin Bealer <kevinbealer gmail.com> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 janderson wrote:
 Andrei Alexandrescu (See Website For Email) wrote:
 Hasan Aljudy wrote:
 [snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new 
 attribute; let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant 
 arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei

Although I like this idea, I fear it will not work for anything hidden in a library (if you are using something that is moved to a .lib, your code will stop working). Maybe that's ok. Actually now I think about it, that would be safer, because even if the D compiler could put an "ok" signatures in a library, someone could create fake signatures.

Good point. This is reasonable. To execute code during compilation it's reasonable to expect transparency. C++ commercial vendors did not really suffer financial loss due to this requirement, and at any rate, OSS is on the rise :o).
 It could get confusing if you don't know which functions will work and 
 which won't.  Perhaps the compiler could help there (spit out a list 
 of functions u can use or something).

Yes, that's a documentation issue. The nice thing is that a plethora of really useful functions (e.g. string manipulation) can be written in D's subset that can be interpreted. Pretty much the entire string library will be meta-executable. No more need for the metastrings lib!
 I think the compiler would compile these commands on demand and cache 
 them so it doesn't need to do them every time.  That would help a lot. 
 It could even cache results so it only needs to compute them once.

 Anyways, there i think there is so much possibility with compile time 
 coding.

Me too. Once compile-time interpretation (and mutation) makes it in, I think there's no fear of efficiency loss anymore. Speed will be comparable to that of any interpreted code, and there are entire communities that don't have a problem with that. The question is, what is the subset of D that can be interpreted? I'm thinking: * Basic data types (they will be stored as a dynamically-typed variant anyway), except pointers to functions and delegates. * Arrays and hashes * Basic expressions (except 'is', 'delete' et al.) * if, for, foreach, while, do, switch, continue, break, return I'm constructing this list thinking what it takes to write basic data manipulation functions. What did I forget? Andrei

Simple answer: the intersection of a simple interpreted language and D. What can D do that a scripted language cannot? Take that out. Approaching it from another angle, imagine I have a parse tree and I'm writing code to interpret it... what do I want to leave out? - inline asm - exceptions - synchronization and volatile and related - calls to C (though this might prompt some rewriting of Phobos to avoid calling things like strlen() if they aren't builtins/intrinsics. What I see as a bigger snag is processing order. The whole D universe expects to be able to forward or backward reference anything regardless of how it is defined. Let's say I have these two functions: char[] foo(char[] a, char[] b) metaok; char[] bar(char[] a, char[] b, char[] c) metaok; The metaok keyword means "this can be run at compile time." Suppose further that bar(a,b,c) calls foo(a,b) as a base case, and that both are generated via the "meta" mechanism. No mutual recursion or anything though. What order do you compile them in? Obviously, foo then bar -- bar needs foo to be runable at compile time. I love the idea of makeing all of std.strings compile-time-able, but the D compiler sees the code as a big array of unrelated functions right now (or it can if it chooses to.) With this stuff, the compiler is thrown back into the shell or C world where every function definition and execution is an event in a history, and order is everything. The alternative I prefer, is to collect all function definitions, sort out the dependency order in the compiler using graphs or what not, and work in that order. To be consistent with D philosophy as I see it, the detection of cycles in this graph should be flagged like an ambiguous overload. That should be okay, but when compiling across modules it might get somewhat harder...? Kevin
Feb 08 2007
parent reply janderson <askme me.com> writes:
Kevin Bealer wrote:
 Let's say I have these two functions:
 
 char[] foo(char[] a, char[] b) metaok;
 char[] bar(char[] a, char[] b, char[] c) metaok;
 
 The metaok keyword means "this can be run at compile time."
 
 Suppose further that bar(a,b,c) calls foo(a,b) as a base case, and that 
 both are generated via the "meta" mechanism.  No mutual recursion or 
 anything though.
 
 What order do you compile them in?  Obviously, foo then bar -- bar needs 
 foo to be runable at compile time.
 

 Kevin

I think that is a good case for labeling which functions are compile-time safe, although I think it solveable. Here's another case for problems: void mixin { "char [] foo1" ~ foo3 ~ "{...}" }; void mixin { "foo2" ~ foo1() ~ "{...}" }; How is the compiler going to figure that one out. Parhaps there should be some more rules to help with these cases? Maybe its possible to compile it? Maybe something to do with namespacing? I'm not sure. I think would be possible. I think you shouldn't be able to access functions a compile time defined in a mixin. But I'm sure there are many other cases like this to figure out. -Joel
Feb 08 2007
parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
janderson wrote:
 Kevin Bealer wrote:
 Let's say I have these two functions:

 char[] foo(char[] a, char[] b) metaok;
 char[] bar(char[] a, char[] b, char[] c) metaok;

 The metaok keyword means "this can be run at compile time."

 Suppose further that bar(a,b,c) calls foo(a,b) as a base case, and 
 that both are generated via the "meta" mechanism.  No mutual recursion 
 or anything though.

 What order do you compile them in?  Obviously, foo then bar -- bar 
 needs foo to be runable at compile time.

 Kevin

I think that is a good case for labeling which functions are compile-time safe, although I think it solveable. Here's another case for problems: void mixin { "char [] foo1" ~ foo3 ~ "{...}" }; void mixin { "foo2" ~ foo1() ~ "{...}" };

That one might work by luck, but that's not the intended usage of compile-time execution so I think it'd be safe to define that as an error; i.e. specify that mixed-in code is not compile-time executable.
 
 
 How is the compiler going to figure that one out.  Parhaps there should 
 be some more rules to help with these cases?  Maybe its possible to 
 compile it?  Maybe something to do with namespacing?  I'm not sure.  I 
 think would be possible.
 
 I think you shouldn't be able to access functions a compile time defined 
 in a mixin.  But I'm sure there are many other cases like this to figure 
 out.
 
 -Joel

Feb 09 2007
prev sibling next sibling parent janderson <askme me.com> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 janderson wrote:
 Andrei Alexandrescu (See Website For Email) wrote:
 Hasan Aljudy wrote:
 [snip]
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new 
 attribute; let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant 
 arguments.

This is much in keep with my idea on how metaprogramming should be done, with the little semantic nit that "meta" should be "dual" as add has dual functionality. The attribute is not even needed if meta-code is flagged as such. My thoughts currently gravitate around the idea of using mixin as an escape into compile-time world. Anything that's printed to standard output in compile-time world becomes code, e.g. (using your function): mixin { writefln("int x = ", add(10, 20), ";"); } is entirely equivalent to: int x = 30; No annotation is needed on add because mixin clarifies that it's called during compilation. The compiler will complain if add cannot be user dually. Using mixin and write as separation devices makes it very clear what is to be done when; otherwise, it quickly becomes confusing what code is meant to actually get evaluated eagerly, and what code is to actually be "output" for compilation. The not-so-nice thing is that we get to manipulate numbers, strings, and arrays, not trees like in LISP. Andrei

Although I like this idea, I fear it will not work for anything hidden in a library (if you are using something that is moved to a .lib, your code will stop working). Maybe that's ok. Actually now I think about it, that would be safer, because even if the D compiler could put an "ok" signatures in a library, someone could create fake signatures.

Good point. This is reasonable. To execute code during compilation it's reasonable to expect transparency. C++ commercial vendors did not really suffer financial loss due to this requirement, and at any rate, OSS is on the rise :o).
 It could get confusing if you don't know which functions will work and 
 which won't.  Perhaps the compiler could help there (spit out a list 
 of functions u can use or something).

Yes, that's a documentation issue. The nice thing is that a plethora of really useful functions (e.g. string manipulation) can be written in D's subset that can be interpreted. Pretty much the entire string library will be meta-executable. No more need for the metastrings lib!
 I think the compiler would compile these commands on demand and cache 
 them so it doesn't need to do them every time.  That would help a lot. 
 It could even cache results so it only needs to compute them once.

 Anyways, there i think there is so much possibility with compile time 
 coding.

Me too. Once compile-time interpretation (and mutation) makes it in, I think there's no fear of efficiency loss anymore. Speed will be comparable to that of any interpreted code, and there are entire communities that don't have a problem with that. The question is, what is the subset of D that can be interpreted? I'm thinking: * Basic data types (they will be stored as a dynamically-typed variant anyway), except pointers to functions and delegates.

Agreed. Pointers are a likely area for abuse. Maybe they could be added at a later time once we understand how this goes in practice.
 
 * Arrays and hashes
 
 * Basic expressions (except 'is', 'delete' et al.)
 
 * if, for, foreach, while, do, switch, continue, break, return
 
 I'm constructing this list thinking what it takes to write basic data 
 manipulation functions. What did I forget?
 
 
 Andrei

This is a good start. * Import and mixin's as well so we can read in stuff (safely) at compile time and do self reflection and stuff. * Classes + structs would be a nice addition, although they wouldn't be need in the first additions. Classes would mean you'd need a GC and would add more complications. I think once a foundation is in they could be added with more discussions. OO is a powerful abstraction concept so would be useful in simplifying the compile-time code. -Joel
Feb 08 2007
prev sibling parent janderson <askme me.com> writes:
Andrei Alexandrescu (See Website For Email) wrote:
 janderson wrote:
 Me too. Once compile-time interpretation (and mutation) makes it in, I 
 think there's no fear of efficiency loss anymore. Speed will be 
 comparable to that of any interpreted code, and there are entire 
 communities that don't have a problem with that.
 
 The question is, what is the subset of D that can be interpreted? I'm 
 thinking:
 
 * Basic data types (they will be stored as a dynamically-typed variant 
 anyway), except pointers to functions and delegates.
 
 * Arrays and hashes
 
 * Basic expressions (except 'is', 'delete' et al.)
 
 * if, for, foreach, while, do, switch, continue, break, return
 
 I'm constructing this list thinking what it takes to write basic data 
 manipulation functions. What did I forget?
 
 
 Andrei

Looking at this from a template perspective. If we don't have classes/structs. How much would the template system need to be changed to work like this? I mean, as I said before "static if" -> "if" and the other operations and compile-time data types (arrays/AA's). Its very close to what we want. Not quite as reusable, but the template syntax would be improved. One issue I have with templates is they cause compilation to slow down and bloat the executable code. Maybe a template with no arguments should be compiled as a normal function without inlining. That would solve most of those problems. -Joel
Feb 09 2007
prev sibling parent reply "Yauheni Akhotnikau" <eao197 intervale.ru> writes:
On Fri, 09 Feb 2007 08:42:37 +0300, Andrei Alexandrescu (See Website For=
  =

Email) <SeeWebsiteForEmail erdani.org> wrote:

 The attribute is not even needed if meta-code is flagged as such. My  =

 thoughts currently gravitate around the idea of using mixin as an esca=

 into compile-time world. Anything that's printed to standard output in=

 compile-time world becomes code, e.g. (using your function):

 mixin
 {
    writefln("int x =3D ", add(10, 20), ";");
 }

 is entirely equivalent to:

 int x =3D 30;

 No annotation is needed on add because mixin clarifies that it's calle=

 during compilation. The compiler will complain if add cannot be user  =

 dually.

 Using mixin and write as separation devices makes it very clear what i=

 to be done when; otherwise, it quickly becomes confusing what code is =

 meant to actually get evaluated eagerly, and what code is to actually =

 "output" for compilation.

This is a good idea! But I think it is no need to iterpret 'pure D' code at compile time. If = = compiler sees such mixin expression it can create temporary D program = which has content of the mixin expression and then execute it in = background with standard output redirection (with help of rdmd). This approach can be used even with current string mixin expression. For= = example, if compiler sees a construct: mixin( SomeDSLProcessor( `some DSL code` ) ); and knowns than SomeDSLProcessor is an ordinary D function then compiler= = can create a simple temporary program: import <module in which SomeDSLProcessor is defined>; void main() { writefln( SomeDSLProcessor( `some DSL code` ) ); } then run it by rdmd and use its output as argument for mixin expression = in = original program. -- = Regards, Yauheni Akhotnikau
Feb 09 2007
parent reply "Andrei Alexandrescu (See Website For Email)" <SeeWebsiteForEmail erdani.org> writes:
Yauheni Akhotnikau wrote:
 On Fri, 09 Feb 2007 08:42:37 +0300, Andrei Alexandrescu (See Website For 
 Email) <SeeWebsiteForEmail erdani.org> wrote:
 
 The attribute is not even needed if meta-code is flagged as such. My 
 thoughts currently gravitate around the idea of using mixin as an 
 escape into compile-time world. Anything that's printed to standard 
 output in compile-time world becomes code, e.g. (using your function):

 mixin
 {
    writefln("int x = ", add(10, 20), ";");
 }

 is entirely equivalent to:

 int x = 30;

 No annotation is needed on add because mixin clarifies that it's 
 called during compilation. The compiler will complain if add cannot be 
 user dually.

 Using mixin and write as separation devices makes it very clear what 
 is to be done when; otherwise, it quickly becomes confusing what code 
 is meant to actually get evaluated eagerly, and what code is to 
 actually be "output" for compilation.

This is a good idea! But I think it is no need to iterpret 'pure D' code at compile time. If compiler sees such mixin expression it can create temporary D program which has content of the mixin expression and then execute it in background with standard output redirection (with help of rdmd).

I've said it before, this is useless. Metacode must have access to the program's symbols. Andrei
Feb 09 2007
parent reply "Yauheni Akhotnikau" <eao197 intervale.ru> writes:
 But I think it is no need to iterpret 'pure D' code at compile time. If  
 compiler sees such mixin expression it can create temporary D program  
 which has content of the mixin expression and then execute it in  
 background with standard output redirection (with help of rdmd).

I've said it before, this is useless. Metacode must have access to the program's symbols. Andrei

Can you provide some examples which show for what that is need? (May be I miss something but I don't see more or less realistic examples yet). -- Regards, Yauheni Akhotnikau
Feb 09 2007
parent reply "Andrei Alexandrescu (See Website For Email)" <SeeWebsiteForEmail erdani.org> writes:
Yauheni Akhotnikau wrote:
 But I think it is no need to iterpret 'pure D' code at compile time. 
 If compiler sees such mixin expression it can create temporary D 
 program which has content of the mixin expression and then execute it 
 in background with standard output redirection (with help of rdmd).

I've said it before, this is useless. Metacode must have access to the program's symbols. Andrei

Can you provide some examples which show for what that is need? (May be I miss something but I don't see more or less realistic examples yet).

In a previous post I described the white hole and black hole classes. Starting from an interface, build two degenerate implementations of it. This is not possible with the naive approach of spawning execution of separate programs. Andrei
Feb 09 2007
parent "Yauheni Akhotnikau" <eao197 intervale.ru> writes:
On Fri, 09 Feb 2007 12:00:01 +0300, Andrei Alexandrescu (See Website For=
  =

Email) <SeeWebsiteForEmail erdani.org> wrote:

 In a previous post I described the white hole and black hole classes. =

 Starting from an interface, build two degenerate implementations of it=

 This is not possible with the naive approach of spawning execution of =

 separate programs.

Thanks. Yes, it is not possible. But for this task it is necessary to run DSL co= de = inside compiler. I don't like idea of interpretating D code with some = limitation. But there can be (at least :) ) two alternatives: 1. Compile temporary D program as DLL and upload it into compiler. And = provide some API to access to currently available compiler information = (symbols, types, modules and so on). So, SomeDSLProcessor can use this A= PI: import std.compiler.internals; char[] SomeDSLProcessor( char[] dsl ) { // parsing of dsl... // accessing compiler information via singleton object... auto symbol_info =3D DCompiler.getIntance().getSymbolInfo( ... ); ... } 2. Compiler plug-in system. Plug-in implement some special interface = (interfaces): // SomeDSLProcessor.d import std.compiler.plugins; class SomeDSLProcessor : ICompilerPlugIn { // This method will be called for processing content of mixin express= ion. char[] invoke( ICompilerInformation compilerInfo, char[] dsl ) { ... = } ... } // Some file which use SomeDSLProcessor plugin. import plugin SomeDSLProcessor; mixin( SomeDSLProcessor( `some DSL code` ) ); When compiler sees 'import plugin' clause it finds and uploads plugin DL= L. = When compiler sees name of plugin in mixin expression it instantiates = object of class SomeDSLProcessor and calls its method invoke. -- = Regards, Yauheni Akhotnikau
Feb 09 2007
prev sibling next sibling parent janderson <askme me.com> writes:
Hasan Aljudy wrote:
 I think templates is the wrong way to go for compile-time meta-programming.
 
 There's been some talk recently about executing functions at 
 compile-time by some people, because these people (including me) feel 
 that template recursion and tricks are not the appropriate way to 
 manipulate/parse strings for specialized DSL (domain specific languages).
 
 I will give you some ideas I have, and I would like to know your 
 thoughts about them.
 
 To me, compile time functions should employ the same concepts behind 
 "constant folding": if an expression is composed of compile time 
 constants, the compiler will compute the expression at compile time and 
 replace it with the result.
 
 # int x = 2 + 3;
 will be optimized by the compiler to:
 # int x = 5;
 Here, the compiler computed the expression "2+3" at compile time, and 
 replaced the expression with its result, "5".
 
 The same concept should be applicable to compile time functions. A 
 function is a complicated set of expressions.
 
 Some functions are trivial; they take a set of parameters, do something 
 with them, without calling or depending on other functions, and return 
 the result.
 A "trivial" example:
 # int add( int x, int y ) { return x + y; }
 If they take constant arguments, such functions can surely be computed 
 at compile time without much hassle. Such that a call to
 # int Q = add( 10, 12 );
 can be replaced, at compile time, with this:
 # int Q = 22;
 
 The trick may lie in letting the compiler recognize these kind of 
 functions. A simple solution might me to come up with a new attribute; 
 let's call it "meta":
 # meta int add( int x, int y ) { return x + y; }
 This attribute will assist the compiler in recognizing that this 
 function can be computed at compile time if it's given constant arguments.
 
 Now, this may not be very useful because of the restriction that meta 
 functions must not call other functions, which is very limiting; so we 
 need to extend it.
 
 If a function takes constant arguments and does call other functions, 
 then it can only be executed at compile time if all of these other 
 functions are meta functions.
 
 example:
 # void subtract( int x, int y ) { return add( x, -y ); }
 This function calls "add", but add is a meta function that can be 
 executed at compile time, and will be replaced at compile time with:
 # void subtract( int x, int y ) { return x - y ); }
 So this type of function can definitely be executed at compile time, and 
 thus deserves to be a meta function.
 
 # meta void subtract( int x, int y ) { return add( x, -y ); }
 
 The same idea could be extended to string-manipulating functions.
 
 Using these concepts as basis for compile-time meta-programming might 
 give us another benefit: we don't have to write compile-time functions 
 that repeat the same tasks already done by run-time functions (regex for 
 example); we just have to be careful and design our functions so that 
 they can be meta functions.
 
 The idea is that if you call a meta function with non-constant 
 arguments, the compiler shouldn't complain; it will just treat the 
 function call in this case as a regular function call to a runtime 
 function.
 
 What do you think about this?! Please tell me your opinions.

This method has been mentioned a few times now ("extention", "static", "plugin" ect...). If so many people keep coming up with the same idea, there must be something going for it. I agree this is probably the best way to go. I should add (and I've already said this) things marked with "what-ever-we call this" would be evaluated a compile time so to ensure their compiletime-ness. -Joel
Feb 08 2007
prev sibling parent Reiner Pope <xxxx xxx.xxx> writes:
Hasan Aljudy wrote:
 I think templates is the wrong way to go for compile-time meta-programming.

template programming environment and the one you describe (as far as I can see, const-foldable and alias-free are equivalent) are in fact equivalent in my opinion (see my post in 'executing pure D at compile-time' where I that these two things are just syntactically different ways of expressing the same thing). For this reason, I think that we could relatively easily get nicer environment such as the one you describe, because it is trivially convertable to the already-working template system. Perhaps Walter could do it, and we would be impressed for a while. But I don't really want that, because later we will come and look at it and think, 'hey, this disallows aliasing, just because we were too lazy back then to support a full compile-time language' and then we will feel stuck with a limited language, just as we now feel stuck with a limited template language. Or else the Ruby, LISP, and Nemerle people will look at D and say, 'well we can do meta-programming with whole language; why does D (seemingly arbitrarily) only allow meta-programming in a limited subset?' ----------------- I also think we should consider that the ability of the compile-time language to do type manipulation might not extend to runtime. A good old type-template just means parameterising the function by type as well as value. Further, a type-tuple is just an array of types (it has .length, it has indexing, and it has basic concatenation). I think we should make this similarity explicit, by actually calling a type-tuple an array of types, giving them D's advanced array features, as well as allowing generic array code to be used on type-arrays. I think the best way to do this is to unify types with TypeInfo, so that a template Type parameter is just a template value parameter of type TypeInfo, and also having the additional rule that you can do the following: const TypeInfo T = foo!(stuff); T x; // declare x as type T This also gives us a nice symmetry between the two varargs approaches: they both pass TypeInfo to the function, but the old one does it at runtime, and the new one does it at compile time. Furthermore, treating type-tuples as arrays allows nesting of type-tuples, which may occasionally be useful, and which type-tuples currently do not support. Cheers, Reiner
Feb 08 2007