www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - bug in foreach continue

reply Hussien <SH glory.com> writes:
foreach (y; aliasSeqOf!["a", "b", "c"])
{
static if (y == "a") { }
else
  pragma(msg, y);
}

works but

foreach (y; aliasSeqOf!["a", "b", "c"])
{
static if (y == "a") continue
pragma(msg, y);
}

fails.

Seems like continue needs to be static aware.

This leads to subtle bugs where one thinks the continue 
statement(like in in a runtime foreach loop, should jump to to 
the next iteration, but it doesn't.

The runtime versions of the code work identically, and so should 
the static versions.
Mar 16
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 17 March 2017 at 01:34:52 UTC, Hussien wrote:
 Seems like continue needs to be static aware.
That's not a bug, pragma is triggered when the code is *compiled*, not when it is run. The code is compiled, even if it is skipped over by a continue.
Mar 16
parent reply Hussien <SH glory.com> writes:
On Friday, 17 March 2017 at 01:41:47 UTC, Adam D. Ruppe wrote:
 On Friday, 17 March 2017 at 01:34:52 UTC, Hussien wrote:
 Seems like continue needs to be static aware.
That's not a bug, pragma is triggered when the code is *compiled*, not when it is run. The code is compiled, even if it is skipped over by a continue.
My point is, that it shouldnt! a static if should have a static continue in some way. It is confusing to be thinking in terms of the compile time code and (static if) and use a ambiguous statement(continue) and it not be compile time. Either one needs a static version of continue or a warning when used inside a static if. As is, continue doesn't even apply to the static case, so it shouldn't be allowed. A static foreach with a dynamic continue is illogical yet no warning is emitted.
Mar 16
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, March 17, 2017 01:55:19 Hussien via Digitalmars-d-learn wrote:
 On Friday, 17 March 2017 at 01:41:47 UTC, Adam D. Ruppe wrote:
 On Friday, 17 March 2017 at 01:34:52 UTC, Hussien wrote:
 Seems like continue needs to be static aware.
That's not a bug, pragma is triggered when the code is *compiled*, not when it is run. The code is compiled, even if it is skipped over by a continue.
My point is, that it shouldnt! a static if should have a static continue in some way. It is confusing to be thinking in terms of the compile time code and (static if) and use a ambiguous statement(continue) and it not be compile time. Either one needs a static version of continue or a warning when used inside a static if. As is, continue doesn't even apply to the static case, so it shouldn't be allowed. A static foreach with a dynamic continue is illogical yet no warning is emitted.
I tend to agree with this. If the foreach is static, and continue and break are just going to be ignored, then they should just be illegal. Allowing them is just going to confuse people. Now, making it so that they actually work statically has some interesting possibilities, but that would fall apart as soon as you have any constructs that would use continue or break (e.g. a loop or switch statement) inside the static foreach, and it might break code in rare cases. So, we're probably better off just making them illegal. But having them be legal just seems disingenious, since they don't do anything. - Jonathan M Davis
Mar 17
parent reply Michael <michael toohuman.io> writes:
On Friday, 17 March 2017 at 11:30:48 UTC, Jonathan M Davis wrote:
 On Friday, March 17, 2017 01:55:19 Hussien via 
 Digitalmars-d-learn wrote:

 I tend to agree with this. If the foreach is static, and 
 continue and break are just going to be ignored, then they 
 should just be illegal. Allowing them is just going to confuse 
 people. Now, making it so that they actually work statically 
 has some interesting possibilities, but that would fall apart 
 as soon as you have any constructs that would use continue or 
 break (e.g. a loop or switch statement) inside the static 
 foreach, and it might break code in rare cases. So, we're 
 probably better off just making them illegal. But having them 
 be legal just seems disingenious, since they don't do anything.

 - Jonathan M Davis
What exactly IS happening in the case of a continue in a static-if? I could sort of imagine that maybe if you were expecting the loop to be unrolled, that you then have a continue statement in the correct part of the unrolled loop. But I take it this isn't what's happening?
Mar 17
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Friday, March 17, 2017 11:53:41 Michael via Digitalmars-d-learn wrote:
 On Friday, 17 March 2017 at 11:30:48 UTC, Jonathan M Davis wrote:
 On Friday, March 17, 2017 01:55:19 Hussien via
 Digitalmars-d-learn wrote:

 I tend to agree with this. If the foreach is static, and
 continue and break are just going to be ignored, then they
 should just be illegal. Allowing them is just going to confuse
 people. Now, making it so that they actually work statically
 has some interesting possibilities, but that would fall apart
 as soon as you have any constructs that would use continue or
 break (e.g. a loop or switch statement) inside the static
 foreach, and it might break code in rare cases. So, we're
 probably better off just making them illegal. But having them
 be legal just seems disingenious, since they don't do anything.

 - Jonathan M Davis
What exactly IS happening in the case of a continue in a static-if? I could sort of imagine that maybe if you were expecting the loop to be unrolled, that you then have a continue statement in the correct part of the unrolled loop. But I take it this isn't what's happening?
Okay. I completely misunderstood what was happening based on the previous posts. For instance, from this test import std.meta; import std.stdio; void main() { foreach(str; ["hello", "world", "foo", "bar"]) { foreach(i; AliasSeq!(0, 1, 2, 3, 4)) { static if(i / 2 == 0) writefln("%s: %s", i, str); else { writefln("%s: skipping %s", i, str); break; } } writeln(); } } it looks like break and continue _are_ used at compile time, since it prints 0: hello 1: hello 2: skipping hello 0: world 1: world 2: skipping world 0: foo 1: foo 2: skipping foo 0: bar 1: bar 2: skipping bar whereas if the break were just compiled in to be run at runtime, you never would have seen any of the strings past "hello" in the outer loop get printed. So, the issue is _not_ that continue and break are always treated as runtime constructs or outright ignored. And if we have import std.meta; import std.stdio; void main() { foreach(str; AliasSeq!("a", "b", "c")) { static if(str == "a") continue; writeln(str); } } it prints b c because when the loop is unrolled, wirteln isn't compiled in (and in fact, if you compile with -w, the code won't compile at all, because writeln is unreachable in the "a" iteration). On the other hand, if we have import std.meta; import std.stdio; void main() { foreach(str; AliasSeq!("a", "b", "c")) { static if(str == "a") continue; pragma(msg, str); } } then we get a b c which is what the OP was complaining about. Now, as to whether that's a bug or not, I don't know. pragma(msg, ...) prints whenever it's compiled, and as shown by the compilation error when you compile in the -w case, the code after the static if _is_ compiled in. It just ends up not being run. Certainly, the continue and break are _not_ skipped like I originally understood to be the case. So, that's not the bug. If there _is_ a bug, it's that the rest of the code in the loop after the static if is compiled after the continue, and I don't know that that is a bug. Remember that this is basically doing loop unrolling, which is not necessarily the same thing as you would expect from a "static foreach." It's not like we're building a string mixin here. It's more like the same code is being copy-pasted over and over but potentially with some tweaks on a given iteration. So, I'm not at all convinced that this is a bug, but I do agree that the semantics are a bit wonky. Regardless, based on what happens with the writeln case, it's clearly the case that you should be using an else branch if you want to cleanly use a continue in a foreach like this. - Jonathan M Davis
Mar 17
next sibling parent reply Hussien <SH glory.com> writes:
On Friday, 17 March 2017 at 13:10:23 UTC, Jonathan M Davis wrote:
 On Friday, March 17, 2017 11:53:41 Michael via 
 Digitalmars-d-learn wrote:
 On Friday, 17 March 2017 at 11:30:48 UTC, Jonathan M Davis 
 wrote:
 On Friday, March 17, 2017 01:55:19 Hussien via 
 Digitalmars-d-learn wrote:

 I tend to agree with this. If the foreach is static, and 
 continue and break are just going to be ignored, then they 
 should just be illegal. Allowing them is just going to 
 confuse people. Now, making it so that they actually work 
 statically has some interesting possibilities, but that 
 would fall apart as soon as you have any constructs that 
 would use continue or break (e.g. a loop or switch 
 statement) inside the static foreach, and it might break 
 code in rare cases. So, we're probably better off just 
 making them illegal. But having them be legal just seems 
 disingenious, since they don't do anything.

 - Jonathan M Davis
What exactly IS happening in the case of a continue in a static-if? I could sort of imagine that maybe if you were expecting the loop to be unrolled, that you then have a continue statement in the correct part of the unrolled loop. But I take it this isn't what's happening?
Okay. I completely misunderstood what was happening based on the previous posts. For instance, from this test import std.meta; import std.stdio; void main() { foreach(str; ["hello", "world", "foo", "bar"]) { foreach(i; AliasSeq!(0, 1, 2, 3, 4)) { static if(i / 2 == 0) writefln("%s: %s", i, str); else { writefln("%s: skipping %s", i, str); break; } } writeln(); } } it looks like break and continue _are_ used at compile time, since it prints 0: hello 1: hello 2: skipping hello 0: world 1: world 2: skipping world 0: foo 1: foo 2: skipping foo 0: bar 1: bar 2: skipping bar whereas if the break were just compiled in to be run at runtime, you never would have seen any of the strings past "hello" in the outer loop get printed. So, the issue is _not_ that continue and break are always treated as runtime constructs or outright ignored. And if we have
Yes, but you have a nested foreach loop. One runtime and one compile time. The break goes with the runtime loop... but NORMAL programming logic tells us that the break goes with the loop that it exists in.
     foreach(str; ["hello", "world", "foo", "bar"])
     {
         foreach(i; AliasSeq!(0, 1, 2, 3, 4))
         {
             static if(i / 2 == 0)
                 writefln("%s: %s", i, str);
             else
             {
                 writefln("%s: skipping %s", i, str);
                 break;
             }
         }
         writeln();
     }
reduces too
     foreach(str; ["hello", "world", "foo", "bar"])
     {
writefln("%s: %s", 0, str); writefln("%s: %s", 1, str); writefln("%s: skipping %s", 2, str); break; writefln("%s: skipping %s", 3, str); break; writefln("%s: %s", 4, str); break;
         writeln();
     }
And I'm sure this is not what you intended(The first break is good enough). Not only does the unrolled loop not make sense, the standard logic any normal programmer without knowing the ambiguity of foreach loop in D would assume that the break in the inner most loop is for the inner most foreach. This can create very complex and subtle bugs. It gets worse the more complex the mixing of the runtime and compile time programming is. Static foreach loops should use static break and static continue... "Birds of a feather flock together"... If one needs to use a runtime break in a compile time loop, then some special mechanism could be added to disambiguate the two. Mixing compile time and runtime identifiers is a very bad idea! It creates very subtle ambiguities that will cause very serious bugs in the future when D becomes more widely used. At that point, it will probably be too late to be changed(as it will then break code that took in to account the ambiguity). One can say that all break's and continues are runtime... which is how it currently is. That is fine, but a warning should be issued because it goes against meta programming logic. When I think of meta programming, I don't start thinking using a completely different set of rules... especially since most of the syntax and semantics is between runtime and compile time is identical.
 import std.meta;
 import std.stdio;

 void main()
 {
     foreach(str; AliasSeq!("a", "b", "c"))
     {
         static if(str == "a")
             continue;
         writeln(str);
     }
 }

 it prints

 b
 c

 because when the loop is unrolled, wirteln isn't compiled in 
 (and in fact, if you compile with -w, the code won't compile at 
 all, because writeln is unreachable in the "a" iteration).

 On the other hand, if we have

 import std.meta;
 import std.stdio;

 void main()
 {
     foreach(str; AliasSeq!("a", "b", "c"))
     {
         static if(str == "a")
             continue;
         pragma(msg, str);
     }
 }

 then we get

 a
 b
 c

 which is what the OP was complaining about. Now, as to whether 
 that's a bug or not, I don't know. pragma(msg, ...) prints 
 whenever it's compiled, and as shown by the compilation error 
 when you compile in the -w case, the code after the static if 
 _is_ compiled in. It just ends up not being run. Certainly, the 
 continue and break are _not_ skipped like I originally 
 understood to be the case. So, that's not the bug. If there 
 _is_ a bug, it's that the rest of the code in the loop after 
 the static if is compiled after the continue, and I don't know 
 that that is a bug. Remember that this is basically doing loop 
 unrolling, which is not necessarily the same thing as you would 
 expect from a "static foreach." It's not like we're building a 
 string mixin here. It's more like the same code is being 
 copy-pasted over and over but potentially with some tweaks on a 
 given iteration.

 So, I'm not at all convinced that this is a bug, but I do agree 
 that the semantics are a bit wonky. Regardless, based on what 
 happens with the writeln case, it's clearly the case that you 
 should be using an else branch if you want to cleanly use a 
 continue in a foreach like this.

 - Jonathan M Davis
A bug or not is irrelevant. There is an issue with the design of for loops between runtime and compile time versions. There is overlap in syntax and it creates ambiguity and this leads to REAL bugs... sometimes very hard to track down. When someone is scanning a complex piece of code and they run across something similar to the above. They are not going to think of the continue and break's as being dynamic and having complex fall through behavior. They are going to expect them to continue or break out of the inner most loop that they are called from. That is the definition of them: "A break statement results in the termination of the statement to which it applies ( switch , for , do , or while ). A continue statement is used to end the current loop iteration and return control to the loop statement." "The break statement "jumps out" of a loop. The continue statement "jumps over" one iteration in the loop." It's bad enough foreach is ambiguous... we can't now if it's static or not unless we analyze the object it loops over. A better syntax would be to introduce a static syntax for static constructs: $foreach = static/compile time foreach $continue = static/compile time continue $break = static/compile time break $if = static/compile time if $while $do void $foo(int) = static/compile time function. (not necessarily needed because of CTFE but it is explicit. Such a syntax(not saying that $ should be used, just had to use something for demonstration purposes) is not ambiguous. When someone sees $continue, they know EXACTLY what it is doing. What has been done is this: Black: 1. of the very darkest color owing to the absence of or complete absorption of light; the opposite of white. "black smoke" synonyms: dark, pitch-black, jet-black, coal-black, ebony, sable, inky "a black horse" antonyms: white (of the sky or night) completely dark owing to nonvisibility of the sun, moon, or stars. "the sky was moonless and black" synonyms: unlit, dark, starless, moonless, wan; More literarytenebrous, Stygian "a black night" antonyms: clear, bright deeply stained with dirt. "his clothes were absolutely black" (of a plant or animal) dark in color as distinguished from a lighter variety. "Japanese black pine" synonyms: dark, pitch-black, jet-black, coal-black, ebony, sable, inky "a black horse" antonyms: white (of coffee or tea) served without milk or cream. of or denoting the suits spades and clubs in a deck of cards. (of a ski run) of the highest level of difficulty, as indicated by black markers positioned along it. 2. of any human group having dark-colored skin, especially of African or Australian Aboriginal ancestry. And what will eventually have to happen is that the term black will have to be banned as a racist word... simply because of the ambiguity. It is like all words, people are confused by them because of their different meanings. It's best to be explicit as necessary, and in this case, I think D is not explicit enough. The convenience factor created by ambiguity will cause more problems than it solves down the road for a select group of people. A non-ambiguous syntax could be added on top of what currently exists and eventually the old syntax could be depreciated... If continue/break are the only problems then some other, simpler solution, could be found. I think a warning would be good enough. e.g., "Warning: continue used in static foreach loop." which brings attention to the issue. Since no one likes warnings, a pragma(-warn, C34324) could be used(or whatever D's mechanism is to specific appropriate warnings).
Mar 17
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 17 March 2017 at 13:53:58 UTC, Hussien wrote:
 Yes, but you have a nested foreach loop. One runtime and one 
 compile time. The break goes with the runtime loop... but 
 NORMAL programming logic tells us that the break goes with the 
 loop that it exists in.
It did. It broke the loop counting the numbers, but continued the loop of strings normally. People have requested a `static foreach` before, but that request has not yet been granted.
 They are going to expect them to continue or break out of the 
 inner most loop that they are called from.
That is exactly what they do, and exactly what happened in all the examples posted.
Mar 17
parent reply Hussien <SH glory.com> writes:
On Friday, 17 March 2017 at 14:27:25 UTC, Adam D. Ruppe wrote:
 On Friday, 17 March 2017 at 13:53:58 UTC, Hussien wrote:
 Yes, but you have a nested foreach loop. One runtime and one 
 compile time. The break goes with the runtime loop... but 
 NORMAL programming logic tells us that the break goes with the 
 loop that it exists in.
It did. It broke the loop counting the numbers, but continued the loop of strings normally. People have requested a `static foreach` before, but that request has not yet been granted.
 They are going to expect them to continue or break out of the 
 inner most loop that they are called from.
That is exactly what they do, and exactly what happened in all the examples posted.
You're failing to see what I am talking about. I understand the compiled code is correct and functions as it should. What I am talking about is how the statement static if (x) continue; pragma(msg, "called"); vs static if (x) { } else pragma(msg, "not called"); They are both semantically equivalent e.g., if (x) continue; foo(); vs if (x) {} else { foo(); } Both do the same thing in loops when used in the context I originally gave. e.g., using a continue to skip over elements of an interation. Works great in normally programming! BUT in static programming, the continue FUNCTIONS as a runtime component... just as you say. BUT that is not how one interprets it on the static level. The logic for if (x) continue; foo(); vs if (x) {} else { foo(); } and the logic for static if (x) continue; foo(); vs static if (x) {} else { foo(); } should be identical. Just because we are programming on two different levels doesn't change the logic(or shouldn't). The problem, of course, is the continue does exactly what you claim it does. But when I am meta programming I am thinking in terms of meta programming. I program exactly the same way as I do in runtime programming except for the obvious shifts that are required. But foreach is a foreach. In runtime it works on runtime stuff and at compile time it works on compile time stuff. Effectively I have runtime_foreach and static_foreach in my head and I magically disambiguate when required. But there is no such concept for continue. It is impossible to "continue" or even "break" a static for each loop. That is what I am addressing. SO, because I had a mental glitch and I used continue "thinking" it would work for the static case, e.g., I assumed it was a static_continue... when no such thing exists, the code compiled and did something I did not expect! YES, It was my fault! BUT the point is that D has that ambiguity. AND because the flow structures for runtime and compile time are virtually identical, the mistake is easy to make! Even if not, if the mistake does occur, it will be a difficult thing to track down in real world cases that are complex. That is the only point I am making! It has nothing to do with what D actually does, but how it does it. A continue or break should create a warning when used in the context of a static foreach. OR change the freaken syntax of static semantics vs runtime. Making them identical is ridiculously and only leads to these kinds of problems. Sure, it works in 99% of the cases, but so will a new syntax. My point is not what D does specifically in compiling source code, or that it does it right or wrong. I am talking about the interpretation by a human of the code. $ #$ ($*%**%*#( ( !) $_ %%#$#$ # ($!* is some code to some yet to be created programming language... but it doesn't mean that humans will be able to interpret it properly. I am talking about how humans interpret things. Do you really think that a non-D programmer that sees the code foreach(x; [1,2,3]) foreach(y; aliasSeqOf![1,2,3]) static if (y == 1) continue; pragma(msg, y) is going to be able to interpret the code correctly? Because if they use a rewrite rule, which is GENERALLY VALID in all programming languages: foreach(x; [1,2,3]) foreach(y; aliasSeqOf![1,2,3]) static if (y == 1) {} else { pragma(msg, y) } (again, this should be the same as the first case, because we can do that at runtime and it is valid... people write code like that) Yet one gets a different result. Hence, if they are programming some NASA rocket, and make that mistake and the rocket goes left instead of dropping it's boosters, then what? Who's fault? Is it D's fault for creating ambiguity between static and runtime programming? Of course it is, because It sets the foundation for what is to come. Again, my point is not about D compiling the above code, but about the syntax/grammar/language issue it creates conceptually. You may not have that problem because you are familiar enough with D's meta programming but to someone that isn't, they have to go on what they know, and there is a problem conceptually they will have to learn about. It shouldn't be that way. It is a problem of D treating apples as oranges. foreach != foreach... does that make sense? Sure, if you already learned, but if you are learning or have a brain fart, too bad for you... This is D's approach: "Meta programming - The same as programming... well almost". The almost part is what gets people killed.
Mar 17
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 17 March 2017 at 15:14:08 UTC, Hussien wrote:
 What I am talking about is
If you want to add a new feature, `static foreach`, that has static continue and static break, I can get behind that, but that's a new feature, not a bug in the existing feature. You think it is something it isn't. `static if` is unique in D. Nothing else works like it. There is no static switch, no static foreach. Regular switch and foreach can work on compile time values, but they don't actually change their behavior like static if does, they work the same as always. The best you get is that the current value of the foreach can inherit the compile-time staticness of what it is looping, allowing you to use it in a few more places (notably in static if), but the loop construct itself is the same. That's it. It does not do special flow control, it does not change scopes or compile requirements. A new `static foreach` that DOES do those things, and works in places `static if` currently works like outside functions, would be a cool new feature. But until that's added, stop thinking of "static foreach" as being a special thing in the language now. It isn't there. It is just regular foreach and thinking of it as that, always, will make things a lot easier. Then you can join the calls for a new `static foreach` that explicitly IS compile time! And we'll be on the same page there.
Mar 17
prev sibling parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Fri, Mar 17, 2017 at 03:14:08PM +0000, Hussien via Digitalmars-d-learn wrote:
[...]
 What I am talking about is
 
 how the statement
 
 static if (x) continue;
 pragma(msg, "called");
 
 vs
 
 static if (x) { } else
 pragma(msg, "not called");
 
 
 They are both semantically equivalent
This appears to be yet another case of the term "compile-time" causing confusion, because it's actually an ambiguous term. I actually think it's a bad term that should be replaced by something more precise; its ambiguity has caused confusion on the part of D learners more often than not. There are actually (at least) TWO distinct phases of compilation that are conventionally labelled "compile time": 1) Template expansion / AST manipulation, and: 2) CTFE (compile-time function evaluation). Not clearly understanding the distinction between the two often leads to confusion and frustration at why the compiler isn't doing "what I want". This confusion is confounded by the fact that these two do mutually interact in any non-trivial metaprogramming D code, often in rather complex ways. But the fundamental thing to understand here is: Template expansion / AST manipulation must be completed *before* CTFE can run. Basically, CTFE is essentially a D interpreter embedded inside the compiler, that simulates computation at runtime. As such, it requires the AST to be fully compiled, as in, the compiler is ready to generate object code for it, before CTFE can work. This is because it makes no sense to generate code on an incomplete / partial AST. Furthermore, once a piece of code has made it to the CTFE stage, its AST has already been processed, and it's now compiled into an internal representation (analogous to bytecode), so AST-manipulating constructs no longer make any sense. In the CTFE stage, there is no such thing as an AST anymore. Constructs like `static if` and `pragma(msg)` belong to the AST manipulation stage. `static if` changes the effective AST that the compiler sees when it's generating code, and pragma(msg) is essentially a debugging construct that says "print this message if this part of the AST makes it past the template expansion phase". This is why code like this one doesn't do what you might think it does: int ctfeFunc(bool b) { if (b) { pragma(msg, "hello"); return 1; } else { pragma(msg, "goodbye"); return 2; } } The compiler will print *both* "hello" and "goodbye", because the pragma(msg) is evaluated *when the AST of this function is processed*. The variable 'b' doesn't even exist at that point, because ASTs don't have the concept of a 'variable' -- 'b' is just an identifier node in the tree. And 'if' is just another node in the tree that represents a control-flow directive. Change that to 'static if', however, and the compiler sees it differently: int ctfeFunc(bool b) { static if (b) { pragma(msg, "hello"); return 1; } else { pragma(msg, "goodbye"); return 2; } } This code won't work, because 'b' is a variable, but variables don't exist at the AST manipulation stage. So you have to turn 'b' into a so-called "compile-time argument" (boy I hate that term "compile-time", it totally is not precise enough to convey the meaning here): int ctfeFunc(bool b)() { static if (b) { pragma(msg, "hello"); return 1; } else { pragma(msg, "goodbye"); return 2; } } Do not be deceived by the appearances; the story here is far more involved than "since static if is a `compile-time` construct, obviously it needs the condition to be made from `compile-time` arguments". You have to understand that when you move 'b' into the so-called compile-time parameter list, you're essentially saying "here's a template that, given a boolean value, produces an AST tree according to the following pattern". If you instantiate ctfeFunc!true, then what effectively happens is that the compiler sees this AST: int ctfeFunc!true() { return 1; } So it prints "hello". Note that the else branch is NOT EVEN SEEN past the AST stage. It practically doesn't exist at that point. The pragma(msg) is also not seen past that point: it is consumed in the AST manipulation stage. The compiler sees the static if, evaluates the condition, and basically prunes the else branch off, then sees the pragma(msg) and emits the message (i.e., acknowledging "yes this part of the AST makes it past the AST manipulation stage"). The pragma(msg) is pruned from the AST after that. By the time CTFE runs on this function, all the CTFE interpreter sees is "return 1;". It doesn't see the "static if" nor the "pragma(msg)", because those things are AST manipulation constructs; past the AST stage such things don't exist anymore. What complicates this 2-stage picture, though, and probably what doesn't help the confusion with the term "compile-time", is that the compiler is smart enough to perform CTFE on-demand. Meaning that it's valid to do this: int ctfeFunc(bool b) { /* N.B.: "runtime" parameter! */ if (b) return 1; else return 2; } enum b = true; static if (ctfeFunc(b) == 1) struct S { int x; } else struct S { int y; } Note that the static if here has a condition that depends on the output of a CTFE function. This appears be a reversal of the AST manipulation / CTFE stages, but strictly speaking that's actually not the case. What actually happens is: 1) The compiler sees the declaration of ctfeFunc(), produces an AST for it. Note that this means the AST of ctfeFunc gets finalised here -- any static if's, templates, etc., are expanded into the "final" AST for this function. 2) Then the compiler sees the static if, and enters the AST phase *for this part of the code*. It says, I need to know the value of ctfeFunc(b) in order to know which struct declaration to use -- so it *emits code* for ctfeFunc(), and then runs the CTFE engine on the resulting code. Then it takes the output of that evaluation to make the decision. The key point here is that ctfeFunc has *already* passed the AST manipulation stage while the struct declaration is still in the AST stage. The compiler is clever enough to sequence the processing of these two parts of the code so that it's able to compute the static if condition. But the principle of AST manipulation coming before CTFE still applies -- it's not possible for the CTFE engine to interpret ctfeFunc if its AST is still being manipulated (because if the AST is not finalized yet, there isn't any code to interpret!), and it's not possible for the AST manipulation stage to read the value of a CTFE computation that's being run on the same piece of code (because CTFE can't run in the first place while the AST is still being manipulated, and if the code is already ready for CTFE to interpret, that means the AST has already been finalized, you can't change it anymore). Coming back to your loop: static if (x) continue; foo(); In the AST manipulation stage, the compiler evaluates x, and if x is true, then the effective AST is: continue; foo(); By the time it gets to CTFE, it has already "forgotten" that there was such a thing as a static if in the source code. So it will interpret the "continue" unconditionally. Whereas if you wrote: static if (x) {} else { foo(); } in the AST manipulation stage, if x is true, then the effective AST is: {} and if x is false, then the effective AST is: foo(); which is what you intended. T -- A mathematician is a device for turning coffee into theorems. -- P. Erdos
Mar 17
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 03/17/2017 12:05 PM, H. S. Teoh via Digitalmars-d-learn wrote:

 1) Template expansion / AST manipulation, and:

 2) CTFE (compile-time function evaluation).

 Not clearly understanding the distinction between the two often leads to
 confusion and frustration at why the compiler isn't doing "what I want".
I always find your posts very informative and very well articulated. Where is the book already? :) Or blog posts? Ali
Mar 17
next sibling parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Fri, Mar 17, 2017 at 02:52:39PM -0700, Ali Çehreli via Digitalmars-d-learn
wrote:
 On 03/17/2017 12:05 PM, H. S. Teoh via Digitalmars-d-learn wrote:
 
 1) Template expansion / AST manipulation, and:
 
 2) CTFE (compile-time function evaluation).
 
 Not clearly understanding the distinction between the two often
 leads to confusion and frustration at why the compiler isn't doing
 "what I want".
I always find your posts very informative and very well articulated. Where is the book already? :) Or blog posts?
[...] Haha, I don't think I'm up for writing a book... and I don't really keep a blog either. But perhaps a writeup on wiki.dlang.org is in order. This particular topic, I think, is something somebody *should* write about, because it seems to trip up newbies quite often. Maybe sometimes even seasoned D users get tripped up, too (though they tend to know better where the problem is and how to work around it). And this isn't the first time I wrote about it. I guess it's time to consolidate what I've written into a wiki article. T -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
Mar 18
parent Mike Parker <aldacron gmail.com> writes:
On Sunday, 19 March 2017 at 00:18:15 UTC, H. S. Teoh wrote:

 Haha, I don't think I'm up for writing a book... and I don't 
 really keep a blog either.  But perhaps a writeup on 
 wiki.dlang.org is in order.

 This particular topic, I think, is something somebody *should* 
 write about, because it seems to trip up newbies quite often.  
 Maybe sometimes even seasoned D users get tripped up, too 
 (though they tend to know better where the problem is and how 
 to work around it).  And this isn't the first time I wrote 
 about it.  I guess it's time to consolidate what I've written 
 into a wiki article.
Ahem. https://dlang.org/blog/ I'm always ready to talk about guest posts at aldacron gmail.com :-)
Mar 18
prev sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Saturday, March 18, 2017 17:18:15 H. S. Teoh via Digitalmars-d-learn 
wrote:
 On Fri, Mar 17, 2017 at 02:52:39PM -0700, Ali Çehreli via Digitalmars-d-
learn wrote:
 On 03/17/2017 12:05 PM, H. S. Teoh via Digitalmars-d-learn wrote:
 1) Template expansion / AST manipulation, and:

 2) CTFE (compile-time function evaluation).

 Not clearly understanding the distinction between the two often
 leads to confusion and frustration at why the compiler isn't doing
 "what I want".
I always find your posts very informative and very well articulated. Where is the book already? :) Or blog posts?
[...] Haha, I don't think I'm up for writing a book... and I don't really keep a blog either. But perhaps a writeup on wiki.dlang.org is in order. This particular topic, I think, is something somebody *should* write about, because it seems to trip up newbies quite often. Maybe sometimes even seasoned D users get tripped up, too (though they tend to know better where the problem is and how to work around it). And this isn't the first time I wrote about it. I guess it's time to consolidate what I've written into a wiki article.
For the most part, I understand compile-time vs runtime quite well - in particular, I don't find templates vs CTFE to be all that complicated once you get into it. I find it to usually be straightforward and obvious, but there are corner cases where it gets harder to understand, and in particular, foreach with compile-time constructs gets weird enough that - to me at least - it stops being straightforward. So, to a great extent, with foreach and compile-time constructs, my understanding is primarily limited to what I need to work in order to do the stuff that I do. And I'm obviously not a D newbie. For newbies, the whole thing is going to be far harder to understand, and for whatever reason, the fact that a function run with CTFE can't use the template parameters beyond what can be done at runtime throws off a lot of people. - Jonathan M Davis
Mar 18
prev sibling next sibling parent Hussien <SH glory.com> writes:
On Friday, 17 March 2017 at 19:05:20 UTC, H. S. Teoh wrote:
 On Fri, Mar 17, 2017 at 03:14:08PM +0000, Hussien via 
 Digitalmars-d-learn wrote: [...]
 [...]
This appears to be yet another case of the term "compile-time" causing confusion, because it's actually an ambiguous term. I actually think it's a bad term that should be replaced by something more precise; its ambiguity has caused confusion on the part of D learners more often than not. [...]
Thanks for agreeing with me ;)
Mar 17
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 17 March 2017 at 19:05:20 UTC, H. S. Teoh wrote:
 There are actually (at least) TWO distinct phases of 
 compilation that are conventionally labelled "compile time":

 1) Template expansion / AST manipulation, and:

 2) CTFE (compile-time function evaluation).

 [ ... ]
 	Template expansion / AST manipulation must be completed 
 *before*
 	CTFE can run.
Only the templates that the ctfe relies on.
 [ ... ]
 This is because it makes no sense to generate code on an 
 incomplete / partial AST.
This is not exactly true whenever you use ctfe to generate a source string that you later mix-in. You are working with a partial ast.
 Furthermore, once a piece of code has made it to the CTFE 
 stage, its AST has already been processed, and it's now 
 compiled into an internal representation (analogous to 
 bytecode), so AST-manipulating constructs no longer make any 
 sense.
Yes.
 In the CTFE stage, there is no such thing as an AST anymore.
There is an AST. It's just already processed.
Mar 21
prev sibling parent reply Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 17 March 2017 at 19:05:20 UTC, H. S. Teoh wrote:
 There are actually (at least) TWO distinct phases of 
 compilation that are conventionally labelled "compile time":

 1) Template expansion / AST manipulation, and:

 2) CTFE (compile-time function evaluation).
This was an awesome explanation, please do put it on the wiki or as a pull request for the website.
Mar 22
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Wed, Mar 22, 2017 at 11:12:06PM +0000, Jesse Phillips via
Digitalmars-d-learn wrote:
 On Friday, 17 March 2017 at 19:05:20 UTC, H. S. Teoh wrote:
 There are actually (at least) TWO distinct phases of compilation
 that are conventionally labelled "compile time":
 
 1) Template expansion / AST manipulation, and:
 
 2) CTFE (compile-time function evaluation).
This was an awesome explanation, please do put it on the wiki or as a pull request for the website.
I'm still working on that. :-) But the article is turning out to be quite a bit longer than I anticipated -- there is just so much I want to say! The draft is on the wiki but I don't really want to post the link just yet because it still needs lots of editing before it's ready for public consumption. T -- Dogs have owners ... cats have staff. -- Krista Casada
Mar 22
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 03/22/2017 05:39 PM, H. S. Teoh via Digitalmars-d-learn wrote:

 the article is turning out to be quite a bit longer
I said "book"; didn't I? :) Ali
Mar 22
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 23 March 2017 at 00:39:56 UTC, H. S. Teoh wrote:
 I'm still working on that. :-)
Hey, can you at least put scare quotes around "static foreach" each time it is used? There's no such thing as static foreach and while we might call foreach over a typetuple "static", it isn't a precise term and invites additional confusion.
Mar 23
parent "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Thu, Mar 23, 2017 at 05:11:49PM +0000, Adam D. Ruppe via Digitalmars-d-learn
wrote:
 On Thursday, 23 March 2017 at 00:39:56 UTC, H. S. Teoh wrote:
 I'm still working on that. :-)
Hey, can you at least put scare quotes around "static foreach" each time it is used? There's no such thing as static foreach and while we might call foreach over a typetuple "static", it isn't a precise term and invites additional confusion.
Done. I've gotten the draft into a semi-acceptable state, though I'm still not fully satisfied with it just yet. But perhaps now is a good time to solicit input from forum members, so here's the link to the draft: https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time T -- It always amuses me that Windows has a Safe Mode during bootup. Does that mean that Windows is normally unsafe?
Mar 24
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 17 March 2017 at 13:10:23 UTC, Jonathan M Davis wrote:
 it looks like break and continue _are_ used at compile time, 
 since it prints
They are working exactly the same way as any other loop. The fact that it is unrolled and the dead code removed from the binary is an implementation detail that doesn't change the semantics of the loop, just like any other loop unroll optimization. `continue` is more like `goto` than it is like `static if`, even when looping over AliasSeq.
 whereas if the break were just compiled in to be run at 
 runtime, you never would have seen any of the strings past 
 "hello" in the outer loop get printed.
No, they did a normal break at runtime. But they are breaking the INNER loop, so the outer loop continues. The generated code simply removed the dead portions since the optimizer realized there was no point carrying it around. You are totally overthinking this, loops still work the same way regardless of what they are iterating over. The only special thing about an AliasSeq thing is that the current value is available at compile time, so it is legal to use in more places (then the compiler implementation generates the code differently to make it happen, but that's the same idea as passing -funroll-loops or -O3 and getting different code, they don't change how break works.)
 Remember that this is basically doing loop unrolling, which is 
 not necessarily the same thing as you would expect from a 
 "static foreach." It's not like we're building a string mixin 
 here.
Yes.
Mar 17
prev sibling parent Daniel Kozak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
Dne 17.3.2017 v 02:34 Hussien via Digitalmars-d-learn napsal(a):

 foreach (y; aliasSeqOf!["a", "b", "c"])
 {
 static if (y == "a") { }
 else
  pragma(msg, y);
 }

 works but

 foreach (y; aliasSeqOf!["a", "b", "c"])
 {
 static if (y == "a") continue
 pragma(msg, y);
 }

 fails.

 Seems like continue needs to be static aware.

 This leads to subtle bugs where one thinks the continue statement(like 
 in in a runtime foreach loop, should jump to to the next iteration, 
 but it doesn't.

 The runtime versions of the code work identically, and so should the 
 static versions.
No it is not a bug, it works as I would expected: import std.meta; import std.stdio; void main() { foreach (y; aliasSeqOf!(["a", "b", "c"])) { static if (y == "a") continue; else writeln(y); pragma(msg, y); } }
Mar 16