www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE thoughts & functional approach

reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
I like CTFE and the meta programming idea for languages like D.

However, I'm wondering why most (everyone?) is trying to do 
meta-programming using the same language as the one getting compiled. 
IMO the use-cases a pretty different and doing CTFE, code-generation 
etc. needs some other approach. If you look at all the strange template 
syntax, strange hacks etc. it's all far from being obvious.

Why not have a CTL (compile-time-language) that has access to some 
compiler internals, that follows a more functional concept? We are 
evaluating sequences of things to generate code, include / exclude code 
etc.

From my experience with the different approaches, functional thinking 
is much better suited and simpler to use for CTFE goals.

IMO that would really be a big step ahead. Because you know a hammer, 
not everything is a nail...

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Jan 31
next sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.
I see things differently. First of all I don't see everyone trying to do meta-programming with the same language. C++ for example has a quite specific syntax between its arcane templates and the preprocessor. I see having the same language as a *huge* advantage. If a function doesn't have side effects then it can be used at runtime or at compile-time and integrated with your logic easily. D's metaprogramming success is IMHO directly linked to it not having a separate language for it, because it lowers the cost of learning and using metaprogramming. It's saying "metaprogramming isn't different from any other kind of programming, you can use the same tools".
 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.
Having access to some compiler internals is already what is done, or I don't understand exactly what you mean by that.
 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.
I think you should put some sort of example of how you'd want it, because right now I don't understand. D has some nice functional tools and they already show their strength at compile-time. For example function purity is central to their CTFE-ability, using template recursion instead of loops has often proved being a winning strategy etc... Yes applying functionnal principles works well at compile-time, but why should we need another language to do that?
Jan 31
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 14:35:13 +0000, cym13 said:

 I see things differently. First of all I don't see everyone trying
 to do meta-programming with the same language. C++ for example has
 a quite specific syntax between its arcane templates and the
 preprocessor.
Well, ok, maybe a "using the same concepts as the underlaying language" might hit it better.
 I see having the same language as a *huge* advantage. If a function
 doesn't have side effects then it can be used at runtime or at
 compile-time and integrated with your logic easily.
The thing I mean is not that you shouldn't be able to reference or use things from the "target language" but how to write down what you want to do. Code generation by building strings using the D operators for example is something I think is not very elegant. If I could use a list and build it up without having to care about the code / data difference, that would simplify things a lot. Imagine we could use Lua during compile time and have access to the AST etc.
 D's metaprogramming success is IMHO directly linked to it not having
 a separate language for it, because it lowers the cost of learning
 and using metaprogramming.
But limits you to a subset that doesn't feel very natural for doing a lot of common things in code-generation.
 It's saying "metaprogramming isn't different from any other kind of 
 programming, you can use the same tools".
Yes, and I don't think this statement holds. It's very different because the goal is totally different. I need a tool that allows me to manipulate my underlying code during compilation. The main aspect is: Manipulate D code.
 Why not have a CTL (compile-time-language) that has access to some 
 compiler internals, that follows a more functional concept? We are 
 evaluating sequences of things to generate code, include / exclude code 
 etc.
Having access to some compiler internals is already what is done, or I don't understand exactly what you mean by that.
Sure, but the question is how do to deal with it. IMO it's not very straight forward at the moment.
 I think you should put some sort of example of how you'd want it,
 because right now I don't understand. D has some nice functional
 tools and they already show their strength at compile-time.
Ok, here is a simple example: I want to ensure that specific switch statements handle all cases for a given enum, list, etc. This is a common pattern and often a source of problems because you change a collection but miss to update all side-effecting places. This is an example how I have done it (maybe there is a much better way to do it, but I didn't come up with one): ==> BEGIN import std.conv; import std.stdio; import std.string; import std.traits; // example code should work with classes as well enum A {afoo, bfoo, cfoo}; enum members1 = __traits(allMembers, A); // returns TypeTuple auto members2 = __traits(allMembers, A); pragma(msg, typeof(members1)); // pragma(msg, typeid(members1)); // run-time only // static assert(is(members1 : enum)); // Error: basic type expected, not enu pragma(msg, typeof(members2)); // pragma(msg, typeid(members2)); // run-time only // function that generates a string which is used as a mixin at compile time // result string must conform to syntax as it was hand-written code string generateEnums(T...)(string type){ string code = "enum " ~ type ~ " {"; // this is a static foreach (compile time) foreach(m; T){ debug pragma(msg, m ~ ","); // check what code we get at compile time code ~= m ~ ","; } return(code ~ "}"); } int main(){ A switch_var_a; final switch(switch_var_a){ case A.afoo: case A.bfoo: case A.cfoo: } string user_input = readln(); mixin(generateEnums!members1("B")); B switch_var_b = chomp(user_input).to!B; // get rid of terminating chars final switch (switch_var_b) { case B.afoo: { writeln("a"); break; } case B.bfoo: // if commeted will cause a compiler error { writeln("b"); break; } case B.cfoo: {writeln("c");} } return(0); } <== END How about being able to write something like "ensure_final_switch B;" and have this call a CTF that generates the necessary code and has access to tool for building D structured code, AST etc.? And has a compile-time state I can later access in a upcoming CTF. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Feb 02
parent anonymous <anonymous example.com> writes:
On 02.02.2016 09:27, Robert M. Münch wrote:

 ==> BEGIN
[...]
 enum A {afoo, bfoo, cfoo};
(Aside: In D no semicolon is needed here.)
 string generateEnums(T...)(string type){
     string code = "enum " ~ type ~ " {";

     // this is a static foreach (compile time)
     foreach(m; T){
       debug pragma(msg, m ~ ","); // check what code we get at compile time
       code ~= m ~ ",";
     }

     return(code ~ "}");
(Aside: Those parentheses are misleading. return is not a function.)
 }

 int main(){
[...]
     mixin(generateEnums!members1("B"));
     B switch_var_b = chomp(user_input).to!B; // get rid of terminating
 chars
[...]
 }

 <== END
I'm not saying that everything is perfect as it is, but that code can be made nicer with what we have right now: ---- template generateEnums(string[] members) { // 'join' variant: import std.array: join; mixin("enum generateEnums {" ~ members.join(",") ~ "}"); // 'format' variant: // import std.format; // mixin(format(q{ enum generateEnums {%-(%s, %)} }, members)); } void main() { alias B = generateEnums!([members1]); B switch_var_b = chomp(readln()).to!B; /* ... */ } ----
 How about being able to write something like "ensure_final_switch B;"
 and have this call a CTF that generates the necessary code and has
 access to tool for building D structured code, AST etc.? And has a
 compile-time state I can later access in a upcoming CTF.
So you're asking for AST macros, I suppose. There are two DIPs for them: http://wiki.dlang.org/DIP50 - AST Macros http://wiki.dlang.org/DIP78 - AST Macros Lite I don't know where they stand, as I'm not really interested in the whole thing, but maybe one of those matches your vision. If that's not what you have in mind, please be more concrete about what you think of. Maybe show some pseudo code showing how you'd like to be able to solve the example of generating enums.
Feb 02
prev sibling next sibling parent reply anonymous <anonymous example.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.
You're conflating CTFE with the other meta programming tools here. CTFE is the same language as run-time D, but it doesn't have strange template syntax. Templates, static if, __traits, etc. have strange syntax, but they're sort of a different language already. Are you maybe wishing for a nicer alternative to templates, etc?
 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.

 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.

 IMO that would really be a big step ahead. Because you know a 
 hammer, not everything is a nail...
I think this is too vague to lead anywhere. At least you should identify specific problems with D's toolset. And if you have concrete ideas for improvements, you should desribe them in more detail, spelling out how they improve upon the status quo. If you want to have an entirely different meta programming system, then you should show how it would look like, and how it would be better than the status quo. I don't think anyone can make much of "make it more functional". Also, when it's fundamentally different from what we have now, then I don't see it getting into D at the moment. The language is not in a phase of designing fundamentals.
Jan 31
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 15:31:59 +0000, anonymous said:

 You're conflating CTFE with the other meta programming tools here. CTFE 
 is the same language as run-time D, but it doesn't have strange 
 template syntax. Templates, static if, __traits, etc. have strange 
 syntax, but they're sort of a different language already.
See my example in the other post. IMO what I want to achieve is a very simple example but the implementation I came up with is far from being simple.
 Are you maybe wishing for a nicer alternative to templates, etc?
No, my point is that CTFE and meta-programming seems to be much simpler and powerful if I can use a more functional programming approach for it. Working with powerful lists and data = code and code = data concept would simplify this. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Feb 02
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 2 February 2016 at 08:31:07 UTC, Robert M. Münch 
wrote:
 No, my point is that CTFE and meta-programming seems to be much 
 simpler and powerful if I can use a more functional programming 
 approach for it. Working with powerful lists and data = code 
 and code = data concept would simplify this.
Ultimately a restricted deductive programming language would be the better option. There is no need for compile time evaluation to be turing complete; after all, you want the compiler to finish within a fixed time limit.
Feb 02
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.
I have trouble coming up with a proper use-case to use Regex during CTFE; I mean, sure you can do a replacement expansion and encode a few bits to expand for a switch statement or something, but at the same time you can already do that.
Jan 31
parent cym13 <cpicard openmailbox.org> writes:
On Monday, 1 February 2016 at 00:02:55 UTC, Era Scarecrow wrote:
 On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch 
 wrote:
 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.
I have trouble coming up with a proper use-case to use Regex during CTFE; I mean, sure you can do a replacement expansion and encode a few bits to expand for a switch statement or something, but at the same time you can already do that.
Wrong thread?
Jan 31
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 31 January 2016 at 13:59:06 UTC, Robert M. Münch wrote:
 I like CTFE and the meta programming idea for languages like D.

 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting 
 compiled. IMO the use-cases a pretty different and doing CTFE, 
 code-generation etc. needs some other approach. If you look at 
 all the strange template syntax, strange hacks etc. it's all 
 far from being obvious.

 Why not have a CTL (compile-time-language) that has access to 
 some compiler internals, that follows a more functional 
 concept? We are evaluating sequences of things to generate 
 code, include / exclude code etc.

 From my experience with the different approaches, functional 
 thinking is much better suited and simpler to use for CTFE 
 goals.

 IMO that would really be a big step ahead. Because you know a 
 hammer, not everything is a nail...
I'm not sure what is preventing you from doing that already. There is compile time reflection (has access to some compiler internals) and D support functional style. Unless you have some specific in mind, I don't think there is anything we can do to help here. t seems you already have the pieces you want.
Feb 01
parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-02-01 08:15:11 +0000, deadalnix said:

 I'm not sure what is preventing you from doing that already. There is 
 compile time reflection (has access to some compiler internals) and D 
 support functional style. Unless you have some specific in mind, I 
 don't think there is anything we can do to help here. t seems you 
 already have the pieces you want.
See my other example. Yes, maybe I can do everything already, but it's not straightforward. I have to fiddle around a lot to get it to work. Of course that can be my incompetence with CTFE. However, my experience and intuition is that what I would like to use CTFE for (code-generation depending on some other parts of the code like enums, presence of member functions, etc.) can be done much simpler than at the moment. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Feb 02
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 2 February 2016 at 08:34:38 UTC, Robert M. Münch 
wrote:
 On 2016-02-01 08:15:11 +0000, deadalnix said:

 I'm not sure what is preventing you from doing that already. 
 There is compile time reflection (has access to some compiler 
 internals) and D support functional style. Unless you have 
 some specific in mind, I don't think there is anything we can 
 do to help here. t seems you already have the pieces you want.
See my other example. Yes, maybe I can do everything already, but it's not straightforward. I have to fiddle around a lot to get it to work. Of course that can be my incompetence with CTFE. However, my experience and intuition is that what I would like to use CTFE for (code-generation depending on some other parts of the code like enums, presence of member functions, etc.) can be done much simpler than at the moment.
That is definitely true that the compile time API is kind of screwy. That's definitively not you. I think the best path forward at this stage is to provide nice API as a library on top of it.
Feb 02
parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-02-02 08:39:34 +0000, deadalnix said:

 That is definitely true that the compile time API is kind of screwy. 
 That's definitively not you. I think the best path forward at this 
 stage is to provide nice API as a library on top of it.
And if we do this, it's only a small step to add a functional layer to deal with this API where I can chain tranformations to generate my code. My point to discuss was, that I think the combination "compile time API" + "compile time suited language" is something different than the road currently followed. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Feb 02
prev sibling parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-01-31 13:59:06 +0000, Robert M. Münch said:

 I like CTFE and the meta programming idea for languages like D.
 
 However, I'm wondering why most (everyone?) is trying to do 
 meta-programming using the same language as the one getting compiled. 
 IMO the use-cases a pretty different and doing CTFE, code-generation 
 etc. needs some other approach. If you look at all the strange template 
 syntax, strange hacks etc. it's all far from being obvious.
 
 Why not have a CTL (compile-time-language) that has access to some 
 compiler internals, that follows a more functional concept? We are 
 evaluating sequences of things to generate code, include / exclude code 
 etc.
 
 From my experience with the different approaches, functional thinking 
 is much better suited and simpler to use for CTFE goals.
 
 IMO that would really be a big step ahead. Because you know a hammer, 
 not everything is a nail...
Here is a link http://terralang.org/ where these guys mix Lua and their compiled language, to achieve what I was thinking about in the same line: "In this use-case, Lua serves as a powerful meta-programming language. You can think of it as a replacement for C++ template metaprogramming3 or C preprocessor X-Macros4 with better syntax and nicer properties such as hygiene5." Maybe this better explains, where I think it makes sense to seperate the two levels: language for the buidling-step, and language for the actual solution. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Feb 08