www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - foreach over split string

reply "JS" <js.mdnq gmail.com> writes:
foreach(n; std.string.split(s, ","))
{
     // n can't be read at compile time
}

using in a ctfe.

How to loop over a string array in a ctfe?
Jul 17 2013
next sibling parent reply "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 14:02:01 UTC, JS wrote:

 foreach(n; std.string.split(s, ","))
 {
     // n can't be read at compile time
 }

 using in a ctfe.

 How to loop over a string array in a ctfe?



Strange...

The code actually works, but when I use my wrapper template it 
doesn't...

template splitstr(string n, string d = " ") { enum splitstr = 
std.string.split(n, d); }


I guess it has something to do with using an enum as the return 
type?
Jul 17 2013
parent reply "JS" <js.mdnq gmail.com> writes:
Ok, spoke too soon again,

my string requires compound splitting:

foreach(ss; split(s, ","))
{
     split(ss, "|"); // ss can't be read at compile time although 
I can use ss directly
     string a = ss; // works fine.
}
Jul 17 2013
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 17 July 2013 at 14:09:28 UTC, JS wrote:

 Ok, spoke too soon again,

 my string requires compound splitting:

 foreach(ss; split(s, ","))
 {
     split(ss, "|"); // ss can't be read at compile time 
 although I can use ss directly
     string a = ss; // works fine.
 }


Is there any possibility you could provide a compilable example 
(or not compilable, if that's the point :p )? It's an awful lot 
easier to quickly give the right answer to a question if you take 
the time to ask it clearly and precisely, in one go, with a 
minimised example.
Also, by virtue of you having made an example and checked it, it 
prevents accidentally talking about problems in code that is 
actually fine.
Jul 17 2013
parent reply "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 14:18:25 UTC, John Colvin wrote:

 On Wednesday, 17 July 2013 at 14:09:28 UTC, JS wrote:

 Ok, spoke too soon again,

 my string requires compound splitting:

 foreach(ss; split(s, ","))
 {
    split(ss, "|"); // ss can't be read at compile time 
 although I can use ss directly
    string a = ss; // works fine.
 }


 Is there any possibility you could provide a compilable example 
 (or not compilable, if that's the point :p )? It's an awful lot 
 easier to quickly give the right answer to a question if you 
 take the time to ask it clearly and precisely, in one go, with 
 a minimised example.
 Also, by virtue of you having made an example and checked it, 
 it prevents accidentally talking about problems in code that is 
 actually fine.


Yes, but if ctfe's wern't so shitty it wouldn't be a problem in 
the first place. The error messages are crap, the type system is 
disjoint(passing strings to templates must be distinguished in 
many cases from types).

All I did was create a wrapper to std.string.split. when I use 
it, the code breaks... I don't get any obvious error about my 
template but just that I can't use a non-compile time variable.

What you don't understand it is is a lot of work to simplify my 
code to something that will make sense so you can "give your 
quick answer"(which is not what I'm looking for. This isn't stack 
overflow and you don't get a golden ticket for answering 
correctly).

There is also a lot of support code that I use which I have to 
remove... it's a lot more work than you want to make it out to 
be. What I'm learning is that error messages in CTFE's are about 
useless... but they still throw me for a loop.


If it will make you feel better to see some code, which does work,


template tSplitStr(string n, string d = " ") { enum tSplitStr = 
std.string.split(n, d); }


/// Implements each (name, type) pair in T
template tPImplS(T...)
{
	string eval()
	{
		string s;
		foreach(nt; tSplitStr!(tPDeclH!(T), "&"))
		{
			string name = (std.string.split(nt, ","))[0];
			string tname = (std.string.split(nt, ","))[1];
			s ~= "private "~tname~" _"~name~";\n";
			s ~= " property "~tname~" "~name~"() { return 
_"~name~"}\n"~" property "~tname~" "~name~"("~tname~" value) { 
return _"~name~" = value; }\n";
		}

		return s;
	}
	enum tPImplS = eval();
}

but note when I try to use the tSplitStr, I get errors. This 
makes no sense to me because tSplitStr simply wraps split, and I 
should be able to use it where I use split. I do realize one is a 
template and the other is ctfe... and realize that tSplitStr does 
work, in some cases, just not all(which is what makes it a bitch 
to figure out with errors like "nt can't be read at compile 
time").

My guess is the compiler is not smart enough to realize that 
tSplitStr does use any compile time features and really is just a 
wrapper to split.


It seems that if the compiler see's ! used it throws the generic 
error "cannot be read at compile time"... which I'm so sick of 
seeing it's driving me insane.
Jul 17 2013
next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/17/13 11:38 AM, JS wrote:

 On Wednesday, 17 July 2013 at 14:18:25 UTC, John Colvin wrote:

 On Wednesday, 17 July 2013 at 14:09:28 UTC, JS wrote:

 Ok, spoke too soon again,

 my string requires compound splitting:

 foreach(ss; split(s, ","))
 {
    split(ss, "|"); // ss can't be read at compile time although I can
 use ss directly
    string a = ss; // works fine.
 }


 Is there any possibility you could provide a compilable example (or
 not compilable, if that's the point :p )? It's an awful lot easier to
 quickly give the right answer to a question if you take the time to
 ask it clearly and precisely, in one go, with a minimised example.
 Also, by virtue of you having made an example and checked it, it
 prevents accidentally talking about problems in code that is actually
 fine.


 Yes, but if ctfe's wern't so shitty it wouldn't be a problem in the
 first place.


If you think the language is shitty why are you using it?

You didn't pay for D, you are receiving it as a free tool. I don't think 
many people will help you if you continue with that attitude -.-
Jul 17 2013
parent reply "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 14:55:12 UTC, Ary Borenszweig wrote:

 On 7/17/13 11:38 AM, JS wrote:

 On Wednesday, 17 July 2013 at 14:18:25 UTC, John Colvin wrote:

 On Wednesday, 17 July 2013 at 14:09:28 UTC, JS wrote:

 Ok, spoke too soon again,

 my string requires compound splitting:

 foreach(ss; split(s, ","))
 {
   split(ss, "|"); // ss can't be read at compile time 
 although I can
 use ss directly
   string a = ss; // works fine.
 }


 Is there any possibility you could provide a compilable 
 example (or
 not compilable, if that's the point :p )? It's an awful lot 
 easier to
 quickly give the right answer to a question if you take the 
 time to
 ask it clearly and precisely, in one go, with a minimised 
 example.
 Also, by virtue of you having made an example and checked it, 
 it
 prevents accidentally talking about problems in code that is 
 actually
 fine.


 Yes, but if ctfe's wern't so shitty it wouldn't be a problem 
 in the
 first place.


 If you think the language is shitty why are you using it?

 You didn't pay for D, you are receiving it as a free tool. I 
 don't think many people will help you if you continue with that 
 attitude -.-


Go fuck yourself you righteous asshole. First, I never said I 
paid for it, second, I never said D was shitty, I said ctfe was.  
Also, I never asked for your damn help, if you don't like my 
attitude, don't respond, simple as that. If your ego gets hurt 
too fucking bad, grow up. Stop making assumptions to suit your 
own needs.
Jul 17 2013
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 17 July 2013 at 15:18:03 UTC, JS wrote:

 On Wednesday, 17 July 2013 at 14:55:12 UTC, Ary Borenszweig 
 wrote:

 On 7/17/13 11:38 AM, JS wrote:

 On Wednesday, 17 July 2013 at 14:18:25 UTC, John Colvin wrote:

 On Wednesday, 17 July 2013 at 14:09:28 UTC, JS wrote:

 Ok, spoke too soon again,

 my string requires compound splitting:

 foreach(ss; split(s, ","))
 {
  split(ss, "|"); // ss can't be read at compile time 
 although I can
 use ss directly
  string a = ss; // works fine.
 }


 Is there any possibility you could provide a compilable 
 example (or
 not compilable, if that's the point :p )? It's an awful lot 
 easier to
 quickly give the right answer to a question if you take the 
 time to
 ask it clearly and precisely, in one go, with a minimised 
 example.
 Also, by virtue of you having made an example and checked 
 it, it
 prevents accidentally talking about problems in code that is 
 actually
 fine.


 Yes, but if ctfe's wern't so shitty it wouldn't be a problem 
 in the
 first place.


 If you think the language is shitty why are you using it?

 You didn't pay for D, you are receiving it as a free tool. I 
 don't think many people will help you if you continue with 
 that attitude -.-


 Go fuck yourself you righteous asshole. First, I never said I 
 paid for it, second, I never said D was shitty, I said ctfe 
 was.  Also, I never asked for your damn help, if you don't like 
 my attitude, don't respond, simple as that. If your ego gets 
 hurt too fucking bad, grow up. Stop making assumptions to suit 
 your own needs.


This is completely out of order.

I think I can speak quite safely for the majority of the 
community when I say that you are only welcome here if you can 
keep your aggressive and disrespectful comments to yourself.
Jul 17 2013
parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Wednesday, 17 July 2013 at 15:26:19 UTC, John Colvin wrote:

 I think I can speak quite safely for the majority of the 
 community when I say that you are only welcome here if you can 
 keep your aggressive and disrespectful comments to yourself.


I think you cannot, as nobody provided you such right and I, in 
particular, do object, as saying "you may ask but please without 
obscene language" is not a way how public technical newsgroup 
should be organized. It is pretty obvious in this case that flow 
of obscene language will not stop. Also, your politeness does 
harm in this case rather than good.
Jul 17 2013
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Wednesday, 17 July 2013 at 16:19:46 UTC, Maxim Fomin wrote:

 On Wednesday, 17 July 2013 at 15:26:19 UTC, John Colvin wrote:

 I think I can speak quite safely for the majority of the 
 community when I say that you are only welcome here if you can 
 keep your aggressive and disrespectful comments to yourself.


 I think you cannot, as nobody provided you such right and I, in 
 particular, do object, as saying "you may ask but please 
 without obscene language" is not a way how public technical 
 newsgroup should be organized. It is pretty obvious in this 
 case that flow of obscene language will not stop. Also, your 
 politeness does harm in this case rather than good.


Sorry, I assumed we were all against having people make 
aggressive attacks on other members?

Perhaps there is a misunderstanding here: When I say "I think I 
can quite safely speak for the majority" what I mean is "I'm 
pretty sure people most people here agree with the following". I 
don't literally mean I'm some sort of elected mouthpiece.

Btw, I have no problem with the obscene language.
Jul 17 2013
prev sibling parent "anonymous" <anonymous example.com> writes:
On Wednesday, 17 July 2013 at 15:18:03 UTC, JS wrote:

 On Wednesday, 17 July 2013 at 14:55:12 UTC, Ary Borenszweig 
 wrote:


[...]

 If you think the language is shitty why are you using it?

 You didn't pay for D, you are receiving it as a free tool. I 
 don't think many people will help you if you continue with 
 that attitude -.-


 Go fuck yourself you righteous asshole. First, I never said I 
 paid for it, second, I never said D was shitty, I said ctfe 
 was.  Also, I never asked for your damn help, if you don't like 
 my attitude, don't respond, simple as that. If your ego gets 
 hurt too fucking bad, grow up. Stop making assumptions to suit 
 your own needs.


G o o s f r a b a
Jul 17 2013
prev sibling parent Artur Skawina <art.08.09 gmail.com> writes:
On 07/17/13 16:38, JS wrote:

 it is is a lot of work to simplify my code to something that will make sense 


Indeed.



 If it will make you feel better to see some code, which does work,


[...snipped incomplete example which doesn't...]



 template tSplitStr(string n, string d = " ") { enum tSplitStr =
std.string.split(n, d); }


Use a function and forget about '!':

   auto tSplitStr(string n, string d = " ") { return std.string.split(n, d); }
   //...
   foreach(nt; tSplitStr(tPDeclH!(T), "&"))




 you don't get a golden ticket for answering correctly


I was going to explain /why/ your code can't work. But without a ticket? Hmm.
Nah.


artur
Jul 17 2013
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, July 17, 2013 16:02:00 JS wrote:

 foreach(n; std.string.split(s, ","))
 {
 // n can't be read at compile time
 }
 
 using in a ctfe.
 
 How to loop over a string array in a ctfe?


I would point out that if you're just splitting a string to iterate over it, 
then you should probably use std.algorithm.splitter, as it avoids allocating a 
new array.

But the code that you have here appears to work just fine in CTFE for me, so 
you probably have something else going on in code that you didn't show which 
is making it so that CTFE isn't working for you.

- Jonathan M Davis
Jul 17 2013
parent reply "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 18:00:15 UTC, Jonathan M Davis 
wrote:

 On Wednesday, July 17, 2013 16:02:00 JS wrote:

 foreach(n; std.string.split(s, ","))
 {
 // n can't be read at compile time
 }
 
 using in a ctfe.
 
 How to loop over a string array in a ctfe?


 I would point out that if you're just splitting a string to 
 iterate over it,
 then you should probably use std.algorithm.splitter, as it 
 avoids allocating a
 new array.

 But the code that you have here appears to work just fine in 
 CTFE for me, so
 you probably have something else going on in code that you 
 didn't show which
 is making it so that CTFE isn't working for you.

 - Jonathan M Davis


I will try that but the error message given hides the real issue. 
I think the problem is, as demonstrated, simply wrapping a 
standard function in a template breaks ctfe.

I believe the issue is more so about the semantic analysis of 
ctfe more than anything...

I can't put together a working example right now(other things to 
do) but the gist of the matter is:

template strsplit(string n) { enum strsplit = std.string.split(n, 
","); }

... inside a ctfe ...

foreach(n; strsplit!(s)) // doesn't work
foreach(n; std.string.split(s, ",")) // works

To me, this is confusing as hell and "n can't be read at compile 
time" took me a good few hours to realize it was simply using the 
template.  I'm guessing that it has something to do with the enum 
"return type" of strsplit but I've tried other various things.
Jul 17 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jul 18, 2013 at 01:02:26AM +0200, JS wrote:
[...]

 I can't put together a working example right now(other things to do)
 but the gist of the matter is:
 
 template strsplit(string n) { enum strsplit = std.string.split(n,
 ","); }
 
 ... inside a ctfe ...
 
 foreach(n; strsplit!(s)) // doesn't work
 foreach(n; std.string.split(s, ",")) // works
 
 To me, this is confusing as hell and "n can't be read at compile
 time" took me a good few hours to realize it was simply using the
 template.  I'm guessing that it has something to do with the enum
 "return type" of strsplit but I've tried other various things.


I think it will help to understand the difference between templates and
CTFE. Even though both are processed at compile-time, they are not the
same thing, and mixing them may not always do what you think it's doing.

Basically, a template is used for generating code, so any template
parameters must be known "at compile-time", which means that it must
ultimately reduce to a typename, or a literal of a built-in type. Note
that this is NOT the same thing as "this variable is being used by CTFE
so the compiler should know what its value is at `compile-time'". At
this conceptual stage of compilation, the compiler hasn't fully
transformed the source code into runnable form yet, so the only thing it
can do at this point is to work with, conceptually-speaking, text-level
entities. It can't handle variables and other such things that only
exist when a program has been transformed into a runnable form and
executed by an interpreter.

CTFE is basically a subsystem of the compiler that has the ability to
run code *after* it has been generated (either via parsing the source
code directly, or via expanding templates). That is to say, the program
is now in a (semi)runnable form, but the compiler hasn't output the
object files / executables yet. Basically kinda like an "early
execution" of your program, if you will. Since this happens in a
conceptually later stage than template expansion, whatever's going on in
CTFE generally can't be fed back into template parameters. Template
expansion must have been completed before CTFE even comes into play.

I think part of the confusion may stem from the liberal use of the term
"compile-time", which may have given a false impression that if entity X
is known "at compile-time", then it should be usable by construct Y
which requires stuff that's known "at compile-time". This oversimplifies
the process of compilation; in reality, "compile-time" consists of
several distinct conceptual phases (in implementation this may not
necessarily be true literally, but it helps to understand the conceptual
stages). What generally happens is that the compiler needs to:

1) Read the text representation of the source code, break that down
into tokens ("lexing"), and based on the sequence of tokens, construct
an abstract syntax tree that represents the structure of the source code
("parsing");

2) Translate the syntax tree into an internal representation (IR) of the
program that more-or-less maps to the machine code that will eventually
be output;

3) Translate the IR into actual instructions the target CPU can
understand, and output that as the object file / executable.

Conceptually-speaking, template expansion comes somewhere between (1)
and (2): it's a mechanism for producing large numbers of very-similar
subtrees of the syntax tree without requiring the programmer to do lots
and lots of typing. CTFE is roughly somewhere between (2) and (3): the
program isn't fully compiled yet, but enough translation has been done
that it is now in a form that can actually be run (by an interpreter).
In order to get to this point, template expansion must have been
completed first, since otherwise your syntax tree isn't complete yet, so
by definition it can't have been translated into a runnable form yet.
But since template expansion must have already taken place, that means
you can't feed CTFE values back into templates -- since otherwise
template expansion couldn't have already taken place!

So, long story short, template parameters must be known at "text-level",
if we can coin a term for that, the compiler must be able to reduce said
parameters into a concrete type name or a literal of a built-in type.
CTFE, on the other hand, works after template expansion has completed
(conceptually speaking), and so is essentially a kind of "early
execution" of your program. CTFE variables and the like are
"interpreter-level", so they can't be passed back into template
parameters that expect "text-level" input.

The actual details of what the compiler does is, of course, quite a bit
more complex than the above (over)simplified description, but generally
speaking, if you stick by the rule that CTFE values can't be fed back
into the templating system, you will experience much less frustration.


T

-- 
There are two ways to write error-free programs; only the third one works.
Jul 17 2013
parent reply "JS" <js.mdnq gmail.com> writes:
On Wednesday, 17 July 2013 at 23:56:11 UTC, H. S. Teoh wrote:

 On Thu, Jul 18, 2013 at 01:02:26AM +0200, JS wrote:
 [...]

 I can't put together a working example right now(other things 
 to do)
 but the gist of the matter is:
 
 template strsplit(string n) { enum strsplit = 
 std.string.split(n,
 ","); }
 
 ... inside a ctfe ...
 
 foreach(n; strsplit!(s)) // doesn't work
 foreach(n; std.string.split(s, ",")) // works
 
 To me, this is confusing as hell and "n can't be read at 
 compile
 time" took me a good few hours to realize it was simply using 
 the
 template.  I'm guessing that it has something to do with the 
 enum
 "return type" of strsplit but I've tried other various things.


 I think it will help to understand the difference between 
 templates and
 CTFE. Even though both are processed at compile-time, they are 
 not the
 same thing, and mixing them may not always do what you think 
 it's doing.

 Basically, a template is used for generating code, so any 
 template
 parameters must be known "at compile-time", which means that it 
 must
 ultimately reduce to a typename, or a literal of a built-in 
 type. Note
 that this is NOT the same thing as "this variable is being used 
 by CTFE
 so the compiler should know what its value is at 
 `compile-time'". At
 this conceptual stage of compilation, the compiler hasn't fully
 transformed the source code into runnable form yet, so the only 
 thing it
 can do at this point is to work with, conceptually-speaking, 
 text-level
 entities. It can't handle variables and other such things that 
 only
 exist when a program has been transformed into a runnable form 
 and
 executed by an interpreter.

 CTFE is basically a subsystem of the compiler that has the 
 ability to
 run code *after* it has been generated (either via parsing the 
 source
 code directly, or via expanding templates). That is to say, the 
 program
 is now in a (semi)runnable form, but the compiler hasn't output 
 the
 object files / executables yet. Basically kinda like an "early
 execution" of your program, if you will. Since this happens in a
 conceptually later stage than template expansion, whatever's 
 going on in
 CTFE generally can't be fed back into template parameters. 
 Template
 expansion must have been completed before CTFE even comes into 
 play.

 I think part of the confusion may stem from the liberal use of 
 the term
 "compile-time", which may have given a false impression that if 
 entity X
 is known "at compile-time", then it should be usable by 
 construct Y
 which requires stuff that's known "at compile-time". This 
 oversimplifies
 the process of compilation; in reality, "compile-time" consists 
 of
 several distinct conceptual phases (in implementation this may 
 not
 necessarily be true literally, but it helps to understand the 
 conceptual
 stages). What generally happens is that the compiler needs to:

 1) Read the text representation of the source code, break that 
 down
 into tokens ("lexing"), and based on the sequence of tokens, 
 construct
 an abstract syntax tree that represents the structure of the 
 source code
 ("parsing");

 2) Translate the syntax tree into an internal representation 
 (IR) of the
 program that more-or-less maps to the machine code that will 
 eventually
 be output;

 3) Translate the IR into actual instructions the target CPU can
 understand, and output that as the object file / executable.

 Conceptually-speaking, template expansion comes somewhere 
 between (1)
 and (2): it's a mechanism for producing large numbers of 
 very-similar
 subtrees of the syntax tree without requiring the programmer to 
 do lots
 and lots of typing. CTFE is roughly somewhere between (2) and 
 (3): the
 program isn't fully compiled yet, but enough translation has 
 been done
 that it is now in a form that can actually be run (by an 
 interpreter).
 In order to get to this point, template expansion must have been
 completed first, since otherwise your syntax tree isn't 
 complete yet, so
 by definition it can't have been translated into a runnable 
 form yet.
 But since template expansion must have already taken place, 
 that means
 you can't feed CTFE values back into templates -- since 
 otherwise
 template expansion couldn't have already taken place!

 So, long story short, template parameters must be known at 
 "text-level",
 if we can coin a term for that, the compiler must be able to 
 reduce said
 parameters into a concrete type name or a literal of a built-in 
 type.
 CTFE, on the other hand, works after template expansion has 
 completed
 (conceptually speaking), and so is essentially a kind of "early
 execution" of your program. CTFE variables and the like are
 "interpreter-level", so they can't be passed back into template
 parameters that expect "text-level" input.

 The actual details of what the compiler does is, of course, 
 quite a bit
 more complex than the above (over)simplified description, but 
 generally
 speaking, if you stick by the rule that CTFE values can't be 
 fed back
 into the templating system, you will experience much less 
 frustration.


 T



Thanks, this has made it much clearer.

Something like

foreach(a; StrSplit!(s))
    foreach(b; StrSplit !(a))

does work because the second StrSplit uses a "ctfe-time variable" 
instead of a "template-time variable".

My logic was:

1. first StrSplit resolved
2. first foreach evaluated
3. second StrSplit resolved
4. second foreach evaluated

while it actually is

1. first StrSplit resolved
2. second StrSplit resolved
3. foreach's resolved

because template expansion happens before any ctfe expansion.

I guess I was thinking the compiler would be smart enough to 
interleave template expansion and ctfe code(which would be much 
more powerfull).

Effectively template expansion is a sort of pre-processing to 
ctfe code and must be static as far as ctfe's go.

Does this make sense?
Jul 17 2013
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jul 18, 2013 at 07:23:57AM +0200, JS wrote:
[...]

 Thanks, this has made it much clearer.
 
 Something like
 
 foreach(a; StrSplit!(s))
    foreach(b; StrSplit !(a))
 
 does work because the second StrSplit uses a "ctfe-time variable"
 instead of a "template-time variable".
 
 My logic was:
 
 1. first StrSplit resolved
 2. first foreach evaluated
 3. second StrSplit resolved
 4. second foreach evaluated
 
 while it actually is
 
 1. first StrSplit resolved
 2. second StrSplit resolved
 3. foreach's resolved
 
 because template expansion happens before any ctfe expansion.
 
 I guess I was thinking the compiler would be smart enough to
 interleave template expansion and ctfe code(which would be much more
 powerfull).
 
 Effectively template expansion is a sort of pre-processing to ctfe
 code and must be static as far as ctfe's go.


[...]

That's one way to think of it, yes.

As for interleaving template expansion vs. ctfe evaluation, the compiler
*does* do that to some extent.  For example, this code does work:

	// Function that can be evaluated by CTFE
	int func(int x) pure {
		int sum;
		foreach (i; 0..x) {
			sum += i;
		}
		return sum;
	}

	// Force CTFE evaluation
	enum myConst = func(10);

	// Template that requires an int parameter.
	template MyTemplate(int x) {
		enum MyTemplate = x+10;
	}

	// Instantiate template with enum produced by CTFE.
	pragma(msg, MyTemplate!myConst);

	void main() {}

The reason this works is because the compiler is smart enough to figure
out that myConst requires func, so it first compiles func far enough to
be CTFE-evaluable, then it evaluates func to produce the value of
myConst, and then myConst is used to instantiate MyTemplate. You could
think of it as the compiler compiling different parts of the program at
different rates, so func has been compiled into a runnable state, but
the pragma(msg) line is still at the template expansion state.

Note, however, that the template-before-CTFE limitation still applies:
func can't require template expansion while it's running; it must be
entirely compilable into runnable state before CTFE can evaluate it.
Other parts of the program can still remain at the template-expansion
stage, so they can take some CTFE-produced values as template
parameters. But you can't make any reverse dependencies / loops.  You're
OK if you already have runnable code that can then produce template
parameters for other code, but you can't run CTFE and template expansion
simultaneously in the *same* code.

So the compiler is actually pretty smart about reordering these things,
but the fundamental limitation of template-before-CTFE still applies to
each individual code unit. After all, you can't run code that hasn't
been fully expanded by the template system yet.


T

-- 
Winners never quit, quitters never win. But those who never quit AND never win
are idiots.
Jul 17 2013
next sibling parent "JS" <js.mdnq gmail.com> writes:
On Thursday, 18 July 2013 at 05:57:48 UTC, H. S. Teoh wrote:

 On Thu, Jul 18, 2013 at 07:23:57AM +0200, JS wrote:
 [...]

 Thanks, this has made it much clearer.
 
 Something like
 
 foreach(a; StrSplit!(s))
    foreach(b; StrSplit !(a))
 
 does work because the second StrSplit uses a "ctfe-time 
 variable"
 instead of a "template-time variable".
 
 My logic was:
 
 1. first StrSplit resolved
 2. first foreach evaluated
 3. second StrSplit resolved
 4. second foreach evaluated
 
 while it actually is
 
 1. first StrSplit resolved
 2. second StrSplit resolved
 3. foreach's resolved
 
 because template expansion happens before any ctfe expansion.
 
 I guess I was thinking the compiler would be smart enough to
 interleave template expansion and ctfe code(which would be 
 much more
 powerfull).
 
 Effectively template expansion is a sort of pre-processing to 
 ctfe
 code and must be static as far as ctfe's go.


 [...]

 That's one way to think of it, yes.

 As for interleaving template expansion vs. ctfe evaluation, the 
 compiler
 *does* do that to some extent.  For example, this code does 
 work:

 	// Function that can be evaluated by CTFE
 	int func(int x) pure {
 		int sum;
 		foreach (i; 0..x) {
 			sum += i;
 		}
 		return sum;
 	}

 	// Force CTFE evaluation
 	enum myConst = func(10);

 	// Template that requires an int parameter.
 	template MyTemplate(int x) {
 		enum MyTemplate = x+10;
 	}

 	// Instantiate template with enum produced by CTFE.
 	pragma(msg, MyTemplate!myConst);

 	void main() {}

 The reason this works is because the compiler is smart enough 
 to figure
 out that myConst requires func, so it first compiles func far 
 enough to
 be CTFE-evaluable, then it evaluates func to produce the value 
 of
 myConst, and then myConst is used to instantiate MyTemplate. 
 You could
 think of it as the compiler compiling different parts of the 
 program at
 different rates, so func has been compiled into a runnable 
 state, but
 the pragma(msg) line is still at the template expansion state.

 Note, however, that the template-before-CTFE limitation still 
 applies:
 func can't require template expansion while it's running; it 
 must be
 entirely compilable into runnable state before CTFE can 
 evaluate it.
 Other parts of the program can still remain at the 
 template-expansion
 stage, so they can take some CTFE-produced values as template
 parameters. But you can't make any reverse dependencies / 
 loops.  You're
 OK if you already have runnable code that can then produce 
 template
 parameters for other code, but you can't run CTFE and template 
 expansion
 simultaneously in the *same* code.

 So the compiler is actually pretty smart about reordering these 
 things,
 but the fundamental limitation of template-before-CTFE still 
 applies to
 each individual code unit. After all, you can't run code that 
 hasn't
 been fully expanded by the template system yet.


 T


Thanks. Your reply was extremely helpful and I think I'll have 
less pms over ctfe's.
Jul 17 2013
prev sibling parent reply "JS" <js.mdnq gmail.com> writes:
Since you do such a good job a explaining things, would you mind 
informing me what the difference between alias T, T and string T 
used as template parameters are?

template A!(T) { }
template A!(alias T) { }
template A!(string T) { }


The string version is suppose to be a specialization of A that is 
called when A is passed a string? e.g., A!("...") calls the 
string version? (this is my assumption but I seem to run into 
problems where sometimes other versions are called)

The alias T version seems to accept a symbol alias(a sort of 
redirection/substitution is made but no actual evaluation is made 
on the symbol).

The T version accepts types and compile type 
constants/literals(like a string).


This is what I understand them to do... but always get into 
trouble where something wants an alias while another thing wants 
a type.

moduleName complains a lot because sometimes I pass it a built in 
type like double , sometimes a string(by accident but I want it 
to return the string itself), and sometimes I pass a user type.

I have to rectify all 3 usages into one common template so I wrap 
moduleName into, say ModuleName.

template ModuleName(T) { return (isBasicType!T) ? "" : 
moduleName!(T); }
template ModuleName(string s) { return s; }

but sometimes, it seems the T version is called when I pass it a 
string.

Error: template instance moduleName!(string) does not match 
template declaration moduleName(alias T)

when I use

template ModuleName(T)
{
     static if (isValueType!T)
         enum ModuleName = "";
     else static if (isString!T)
         enum ModuleName = T;
     else
         enum ModuleName = StripStr!(std.traits.moduleName!T, `"`, 
" ");
}

template ModuleName(string s) { enum ModuleName = s; }

StripStr is just a template that strips the ends of a string. 
isValueType is a template wrapper to test T for a value 
type(scalar, void, string... *all* types that do not have/need 
module resolution)

The error suggests that I'm passing a string into the 
std.traits.moduleName call but this shouldn't happen?
Jul 18 2013
next sibling parent "JS" <js.mdnq gmail.com> writes:
Ok, I see, I am actually passing it a string type(not a string 
literal) to the template so the first is chosen. I guess my 
isValuetype isn't working properly ;/
Jul 18 2013
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jul 18, 2013 at 12:07:58PM +0200, JS wrote:

 Since you do such a good job a explaining things, would you mind
 informing me what the difference between alias T, T and string T
 used as template parameters are?
 
 template A!(T) { }


T here is a type name. So you can pass in 'int', 'string', etc., or any
user-defined type.



 template A!(alias T) { }


T here is a "symbol", that is, the name of a variable, or a function
literal, etc. Basically an entry in the compiler's symbol table.



 template A!(string T) { }


[...]

T here is a string value. So either a string literal or a string value
known at compile-time. Note that this should not be confused with the
string *type*. That is:

	A!("abc")	// matches template A!(string T)
	A!(string)	// matches template A!(T)

It's probably not a good idea to overload a template on an alias
parameter *and* a string parameter, though, because alias can refer to
*any* symbol, including string literals. You'll either get an ambiguity
error, or the compiler may do something unexpected.

You're probably asking what "any symbol" includes. Perhaps this code
might be enlightening:

	template Templ(alias A) {
		pragma(msg, typeof(A));
		enum Templ = 1;
	}

	void main() {
		int x;
		int delegate(int x) dg;

		auto dummy1 = Templ!("a");
		auto dummy2 = Templ!(x);
		auto dummy3 = Templ!((int z) => z*2);
		auto dummy4 = Templ!(dg);
		auto dummy5 = Templ!(123);

		// Compile error: types are not symbols, so they won't
		// match the alias parameter:
		//auto dummy6 = Templ!(int);
	}

The compiler output is:

	string
	int
	int function(int z) pure nothrow  safe
	int delegate(int x)
	int

That is to say, "alias A" picked up everything ranging from local
variables to string literals to function literals. The only thing it
didn't pick up is types.

(As to *why* alias parameters behave in this way and why that might be
useful, that's a whole 'nother topic. :))


T

-- 
Everybody talks about it, but nobody does anything about it!  -- Mark Twain
Jul 18 2013
parent "JS" <js.mdnq gmail.com> writes:
On Thursday, 18 July 2013 at 19:11:55 UTC, H. S. Teoh wrote:

 On Thu, Jul 18, 2013 at 12:07:58PM +0200, JS wrote:

 Since you do such a good job a explaining things, would you 
 mind
 informing me what the difference between alias T, T and string 
 T
 used as template parameters are?
 
 template A!(T) { }


 T here is a type name. So you can pass in 'int', 'string', 
 etc., or any
 user-defined type.



 template A!(alias T) { }


 T here is a "symbol", that is, the name of a variable, or a 
 function
 literal, etc. Basically an entry in the compiler's symbol table.



 template A!(string T) { }


 [...]

 T here is a string value. So either a string literal or a 
 string value
 known at compile-time. Note that this should not be confused 
 with the
 string *type*. That is:

 	A!("abc")	// matches template A!(string T)
 	A!(string)	// matches template A!(T)

 It's probably not a good idea to overload a template on an alias
 parameter *and* a string parameter, though, because alias can 
 refer to
 *any* symbol, including string literals. You'll either get an 
 ambiguity
 error, or the compiler may do something unexpected.

 You're probably asking what "any symbol" includes. Perhaps this 
 code
 might be enlightening:

 	template Templ(alias A) {
 		pragma(msg, typeof(A));
 		enum Templ = 1;
 	}

 	void main() {
 		int x;
 		int delegate(int x) dg;

 		auto dummy1 = Templ!("a");
 		auto dummy2 = Templ!(x);
 		auto dummy3 = Templ!((int z) => z*2);
 		auto dummy4 = Templ!(dg);
 		auto dummy5 = Templ!(123);

 		// Compile error: types are not symbols, so they won't
 		// match the alias parameter:
 		//auto dummy6 = Templ!(int);
 	}

 The compiler output is:

 	string
 	int
 	int function(int z) pure nothrow  safe
 	int delegate(int x)
 	int

 That is to say, "alias A" picked up everything ranging from 
 local
 variables to string literals to function literals. The only 
 thing it
 didn't pick up is types.

 (As to *why* alias parameters behave in this way and why that 
 might be
 useful, that's a whole 'nother topic. :))



You did it again! Thanks for the accurate and details response! I 
don't mind there being a difference between T and alias T as long 
as I know what it is ;) (although it does require two templates 
do deal with both cases and can increase template explosion).

Thank again.
Jul 18 2013