www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is old style compile-time foreach redundant?

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Is 'static foreach' sufficient for all needs or is there any value for 
regular foreach over compile-time sequences?

Code unrelated to the question:

import std.stdio;

void main() {
     // Old style compile-time foreach. This still works
     // when 'static' is uncommented below.
     import std.meta : AliasSeq;
     /* static */ foreach (i; AliasSeq!(1, "hello", 2)) {
         writeln(i);
     }

     // Proper 'static foreach'.
     import std.range : iota;
     import std.algorithm : map;
     static foreach (i; 3.iota.map!(a => a * 10)) {
         writeln(i);
     }
}

Ali
Jan 06 2018
next sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?
There is, as far as I've seen, there's nothing old foreach-over-tuple can do that new-style static foreach can't. The big difference is static foreach doesn't introduce a scope, which is great when that's what you want, and only requires an additional set of braces otherwise. In some cases, especially when combined with mixins, this new behavior might manifest bugs at some later point. Simplified example: mixin template declareInts(names...) { static foreach (name; names) { mixin("int "~name~";"); } } As long as you use it with non-duplicate names, this will work great. It will fail the moment a junior programmer decides he wants two ints with the same name. Real-world examples will of course be more involved. Old foreach is however not going away any time soon - there's far too many codebases out there that use the feature. -- Simen
Jan 06 2018
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?

 Code unrelated to the question:

 import std.stdio;

 void main() {
     // Old style compile-time foreach. This still works
     // when 'static' is uncommented below.
     import std.meta : AliasSeq;
     /* static */ foreach (i; AliasSeq!(1, "hello", 2)) {
         writeln(i);
     }

     // Proper 'static foreach'.
     import std.range : iota;
     import std.algorithm : map;
     static foreach (i; 3.iota.map!(a => a * 10)) {
         writeln(i);
     }
 }

 Ali
No it's not. When you can use the old style do so. Since it puts less stress on the compiler in the general case.
Jan 06 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via Digitalmars-d-learn
wrote:
 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali ehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?
[...]
 No it's not.
 When you can use the old style do so. Since it puts less stress on the
 compiler in the general case.
Really? Based on a recent post by Jonathan Davis, the new static foreach actually runs faster in certain use cases. T -- No! I'm not in denial!
Jan 06 2018
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 7 January 2018 at 01:08:44 UTC, H. S. Teoh wrote:
 On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via 
 Digitalmars-d-learn wrote:
 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?
[...]
 No it's not.
 When you can use the old style do so. Since it puts less 
 stress on the
 compiler in the general case.
Really? Based on a recent post by Jonathan Davis, the new static foreach actually runs faster in certain use cases. T
That might be true. If you are hitting some constant factor, however the big-o for static foreach is worse then for tuple foreach.
Jan 06 2018
parent Seb <seb wilzba.ch> writes:
On Sunday, 7 January 2018 at 02:17:02 UTC, Stefan Koch wrote:
 On Sunday, 7 January 2018 at 01:08:44 UTC, H. S. Teoh wrote:
 On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via 
 Digitalmars-d-learn wrote:
 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli 
 wrote:
 Is 'static foreach' sufficient for all needs or is there 
 any value for regular foreach over compile-time sequences?
[...]
 No it's not.
 When you can use the old style do so. Since it puts less 
 stress on the
 compiler in the general case.
Really? Based on a recent post by Jonathan Davis, the new static foreach actually runs faster in certain use cases. T
That might be true. If you are hitting some constant factor, however the big-o for static foreach is worse then for tuple foreach.
FWIW there isn't any measurable performance penalty / difference. We moved to use static foreach on Phobos recently: https://github.com/dlang/phobos/pull/5989
Jan 12 2018
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 04:55 PM, Stefan Koch wrote:

 When you can use the old style do so. Since it puts less stress on the
 compiler in the general case.
My question is related to how to update my book. If the difference is mainly about performance, I think I will cover 'static foreach' but also mention old style foreach as a note. Ali
Jan 06 2018
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 7 January 2018 at 01:52:10 UTC, Ali Çehreli wrote:
 On 01/06/2018 04:55 PM, Stefan Koch wrote:

 When you can use the old style do so. Since it puts less
stress on the
 compiler in the general case.
My question is related to how to update my book. If the difference is mainly about performance, I think I will cover 'static foreach' but also mention old style foreach as a note. Ali
How about doing sth. similar like for DIP1003? https://dlang.org/spec/contracts.html#pre_post_contracts
 Since DIP1003 has been applied the actual function body starts 
 with do. In the past, body was used, and could still be 
 encountered in old code bases.
Jan 06 2018
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 06:20 PM, Seb wrote:

 How about doing sth. similar like for DIP1003?
 
 https://dlang.org/spec/contracts.html#pre_post_contracts
Already done! :) https://bitbucket.org/acehreli/ddili/commits/2f10c048c2940a49263319d0c23b0ad661449f3e Ali
Jan 06 2018
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 10:53 PM, Ali Çehreli wrote:
 On 01/06/2018 06:20 PM, Seb wrote:
 
 How about doing sth. similar like for DIP1003?

 https://dlang.org/spec/contracts.html#pre_post_contracts
Already done! :) https://bitbucket.org/acehreli/ddili/commits/2f10c048c2940a492633 9d0c23b0ad661449f3e
What I meant is, yes, I will do it similar to the body-do change as you suggested. Ali
Jan 06 2018
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/6/18 6:25 PM, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any value for 
 regular foreach over compile-time sequences?
If you use continues or breaks, then you need to switch to gotos if using static foreach, as it does not support them directly. -Steve
Jan 07 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/6/18 6:25 PM, Ali ehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?
If you use continues or breaks, then you need to switch to gotos if using static foreach, as it does not support them directly.
[...] Are you sure? I was under te impression that it does support continues and breaks -- but only if they are labelled, because of a syntactic ambiguity otherwise. T -- "Hi." "'Lo."
Jan 08 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/8/18 9:27 AM, H. S. Teoh wrote:
 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/6/18 6:25 PM, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?
If you use continues or breaks, then you need to switch to gotos if using static foreach, as it does not support them directly.
[...] Are you sure? I was under te impression that it does support continues and breaks -- but only if they are labelled, because of a syntactic ambiguity otherwise.
I thought it only worked for constructs outside the static foreach (like switch). testing... Nope, doesn't work. The ambiguity is that if you have a breakable or continuable construct outside a static foreach (e.g. switch), then you may believe that the break statement is affecting the foreach (in fact, that is how tuple-foreach works), but you are actually affecting the outer construct. The extra requirement is to help you realize the implication. It may be removed in the future. I may have been misleading when I made my first comment. What I mean is that you *can't* break or continue a static foreach, even with labels. However, you *can* do it to a standard foreach over a tuple. This may be one reason you want to use a tuple-foreach over a static foreach. -Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 10:57:03AM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/8/18 9:27 AM, H. S. Teoh wrote:
 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/6/18 6:25 PM, Ali Çehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any
 value for regular foreach over compile-time sequences?
If you use continues or breaks, then you need to switch to gotos if using static foreach, as it does not support them directly.
[...] Are you sure? I was under te impression that it does support continues and breaks -- but only if they are labelled, because of a syntactic ambiguity otherwise.
I thought it only worked for constructs outside the static foreach (like switch). testing... Nope, doesn't work.
Grrr... I thought it did, but you're right, attempting to break the static foreach with a label gets this compile error: ----- test.d(7): Error: enclosing label FE for break not found -----
 The ambiguity is that if you have a breakable or continuable construct
 outside a static foreach (e.g. switch), then you may believe that the
 break statement is affecting the foreach (in fact, that is how
 tuple-foreach works), but you are actually affecting the outer
 construct.
Yes, that's the ambiguity I was referring to. :-)
 The extra requirement is to help you realize the implication. It may
 be removed in the future.
I vaguely remember Timon mentioning something about implementing static break / static continue, and somehow I thought the labelled break / labelled continue was supposed to be it. Or at least, they are stand-ins until static break/continue are implemented. Is that no longer on the table?
 I may have been misleading when I made my first comment. What I mean
 is that you *can't* break or continue a static foreach, even with
 labels. However, you *can* do it to a standard foreach over a tuple.
 This may be one reason you want to use a tuple-foreach over a static
 foreach.
[...] Actually, that's wrong too. Tuple-foreach does not interpret break/continue either. Here's a proof: alias Seq(A...) = A; foreach (i; Seq!(0, 1, 2, 3)) { static if (i==2) break; static assert(i < 3); // will fail on the 4th iteration } What actually happens is that all iterations are unrolled, then the unreachable iterations are elided by the optimizer during codegen. The foreach itself is not affected by break/continue at all. T -- Дерево держится корнями, а человек - друзьями.
Jan 09 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/9/18 11:35 AM, H. S. Teoh wrote:
 On Tue, Jan 09, 2018 at 10:57:03AM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 I may have been misleading when I made my first comment. What I mean
 is that you *can't* break or continue a static foreach, even with
 labels. However, you *can* do it to a standard foreach over a tuple.
 This may be one reason you want to use a tuple-foreach over a static
 foreach.
[...] Actually, that's wrong too. Tuple-foreach does not interpret break/continue either. Here's a proof: alias Seq(A...) = A; foreach (i; Seq!(0, 1, 2, 3)) { static if (i==2) break; static assert(i < 3); // will fail on the 4th iteration } What actually happens is that all iterations are unrolled, then the unreachable iterations are elided by the optimizer during codegen. The foreach itself is not affected by break/continue at all.
A break or continue is simply a goto underneath. A goto in an unrolled loop isn't much different than a goto in a um... rolled loop :) It's just that there are copies of each loop body, and the gotos need copies of the labels. So no, it's not "interpreted" by the foreach statement, but the foreach statement provides the anchors for the goto label targets. e.g.: int x; foreach(i; Seq!(0, 1, 2, 3)) { x += i; static if(i % 2) continue; x *= i; } => int x; { x += 0; x *= 0; } { x += 1; goto label1; x *= 1; } { label1: x += 2; x *= 2; } { x += 3; goto label2; x *= 3; } label2: And then of course, the optimizer weeds out the unreachable statements. Doing this with static foreach wouldn't be as pleasant. You'd have to branch the entire loop body, or use a goto in the case of a break. -Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 02:24:11PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
[...]
 A break or continue is simply a goto underneath. A goto in an unrolled
 loop isn't much different than a goto in a um... rolled loop :) It's
 just that there are copies of each loop body, and the gotos need
 copies of the labels.
 
 So no, it's not "interpreted" by the foreach statement, but the
 foreach statement provides the anchors for the goto label targets.
True. [...]
 And then of course, the optimizer weeds out the unreachable
 statements.  Doing this with static foreach wouldn't be as pleasant.
 You'd have to branch the entire loop body, or use a goto in the case
 of a break.
[...] If there were a hypothetical `static continue` or `static break` that's recognized by the static foreach unroller, we could in theory automate this branching in the compiler itself, e.g., by deleting the AST nodes that would be skipped. Of course, the syntax need not be `static continue`; if there were a way to overload `break LABEL;` for the same purpose, i.e., have the static foreach unroller inspect the label to see if it is referring to the static foreach itself, then this would work. But I don't know the static foreach implementation enough to be able to tell whether this is actually possible at the time static foreach is processed, or whether there may be some chicken-and-egg problem with inspecting the target of a break/continue before semantic or whatever. T -- Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.
Jan 09 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/9/18 2:31 PM, H. S. Teoh wrote:
 On Tue, Jan 09, 2018 at 02:24:11PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 [...]
 A break or continue is simply a goto underneath. A goto in an unrolled
 loop isn't much different than a goto in a um... rolled loop :) It's
 just that there are copies of each loop body, and the gotos need
 copies of the labels.

 So no, it's not "interpreted" by the foreach statement, but the
 foreach statement provides the anchors for the goto label targets.
True. [...]
 And then of course, the optimizer weeds out the unreachable
 statements.  Doing this with static foreach wouldn't be as pleasant.
 You'd have to branch the entire loop body, or use a goto in the case
 of a break.
[...] If there were a hypothetical `static continue` or `static break` that's recognized by the static foreach unroller, we could in theory automate this branching in the compiler itself, e.g., by deleting the AST nodes that would be skipped. Of course, the syntax need not be `static continue`; if there were a way to overload `break LABEL;` for the same purpose, i.e., have the static foreach unroller inspect the label to see if it is referring to the static foreach itself, then this would work. But I don't know the static foreach implementation enough to be able to tell whether this is actually possible at the time static foreach is processed, or whether there may be some chicken-and-egg problem with inspecting the target of a break/continue before semantic or whatever.
Yeah, I think in terms of static foreach, it's not a straightforward problem. Because the compiler may not know enough information at the time to figure out whether it should still keep generating code. For example: static foreach(i; 0 .. 5) { if(i == 3) static break; static assert(i < 3); } How does it know whether the static break should be "executed" at compile-time if it hasn't evaluated the if-statement? The code would have to have no runtime branches to make sure that static break can be evaluated at compile-time. And this still puts it at a disadvantage when compared to tuple-foreach, at least as far as break/continue are concerned. -Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 03:26:32PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/9/18 2:31 PM, H. S. Teoh wrote:
[...]
 If there were a hypothetical `static continue` or `static break`
 that's recognized by the static foreach unroller, we could in theory
 automate this branching in the compiler itself, e.g., by deleting
 the AST nodes that would be skipped.  Of course, the syntax need not
 be `static continue`; if there were a way to overload `break LABEL;`
 for the same purpose, i.e., have the static foreach unroller inspect
 the label to see if it is referring to the static foreach itself,
 then this would work.
 
 But I don't know the static foreach implementation enough to be able
 to tell whether this is actually possible at the time static foreach
 is processed, or whether there may be some chicken-and-egg problem
 with inspecting the target of a break/continue before semantic or
 whatever.
Yeah, I think in terms of static foreach, it's not a straightforward problem. Because the compiler may not know enough information at the time to figure out whether it should still keep generating code. For example: static foreach(i; 0 .. 5) { if(i == 3) static break; static assert(i < 3); } How does it know whether the static break should be "executed" at compile-time if it hasn't evaluated the if-statement? The code would have to have no runtime branches to make sure that static break can be evaluated at compile-time.
[...] Static foreach does not (and should not!) evaluate a runtime branch, because this is before CTFE even happens. CTFE cannot happen until the static foreach has been fully unrolled, so it doesn't even make sense to talk about evaluating the if-statement at this point. For your example to make sense, you'd have to use static if, then the break would be possible, and non-problematic. Of course, that still doesn't solve the problem of what static break is supposed to do from inside a runtime branch. I'm tempted to say that static break should mean "delete all subsequent nodes from the AST that follows this node in depth-first traversal order", so your example above would be transformed into: if (0 == 3) {} // all subsequent iterations deleted because the static break is unconditionally compiled (it has nothing to do with the runtime branch). You'd have to use static if to make it conditionally-compiled and thus not instantly aborting the loop. Such semantics would be logically consistent, but unfortunately rather counterintuitive at first glance. T -- If the comments and the code disagree, it's likely that *both* are wrong. -- Christopher
Jan 09 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.01.2018 22:04, H. S. Teoh wrote:
 	if (0 == 3) {}
 	// all subsequent iterations deleted
 
 because the static break is unconditionally compiled (it has nothing to
 do with the runtime branch).  You'd have to use static if to make it
 conditionally-compiled and thus not instantly aborting the loop.
 
 Such semantics would be logically consistent, but unfortunately rather
 counterintuitive at first glance.
I think "if (0 == 3) { static break; }" should be a compile-time error.
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 10, 2018 at 12:18:46AM +0100, Timon Gehr via Digitalmars-d-learn
wrote:
 On 09.01.2018 22:04, H. S. Teoh wrote:
 	if (0 == 3) {}
 	// all subsequent iterations deleted
 
 because the static break is unconditionally compiled (it has nothing
 to do with the runtime branch).  You'd have to use static if to make
 it conditionally-compiled and thus not instantly aborting the loop.
 
 Such semantics would be logically consistent, but unfortunately
 rather counterintuitive at first glance.
I think "if (0 == 3) { static break; }" should be a compile-time error.
That's also a possible solution, perhaps a better solution than what I described. Make it so that static break/continue cannot be nested inside runtime conditionals. That should exclude all of the pathological cases, hopefully. T -- The diminished 7th chord is the most flexible and fear-instilling chord. Use it often, use it unsparingly, to subdue your listeners into submission!
Jan 09 2018
parent Seb <seb wilzba.ch> writes:
On Tuesday, 9 January 2018 at 23:27:42 UTC, H. S. Teoh wrote:
 On Wed, Jan 10, 2018 at 12:18:46AM +0100, Timon Gehr via 
 Digitalmars-d-learn wrote:
 On 09.01.2018 22:04, H. S. Teoh wrote:
 [...]
I think "if (0 == 3) { static break; }" should be a compile-time error.
That's also a possible solution, perhaps a better solution than what I described. Make it so that static break/continue cannot be nested inside runtime conditionals. That should exclude all of the pathological cases, hopefully. T
FWIW I recently bumped into a problem where `static break` would be _really_ useful: https://github.com/dlang/dmd/pull/7577#discussion_r159175229
Jan 09 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, January 08, 2018 06:27:12 H. S. Teoh via Digitalmars-d-learn 
wrote:
 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
 On 1/6/18 6:25 PM, Ali ehreli wrote:
 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?
If you use continues or breaks, then you need to switch to gotos if using static foreach, as it does not support them directly.
[...] Are you sure? I was under te impression that it does support continues and breaks -- but only if they are labelled, because of a syntactic ambiguity otherwise.
It does support them if they're labeled. I did it just the other day. Originally, that code was not using static foreach (though in both cases, it was over an AliasSeq of template arguments), and I'd used a normal break (which compiles just fine with a non-static foreach) and been very confused about why my tests were failing. since for whetever reason, it didn't occur to me when I was writing it that break would break out of the foreach (probably since I was thinking of it as a compile-time thing and not as a loop that would be run, but I don't know; it seems kind of dumb in retrospect either way). Since my tests failed miserably, I realized something was wrong and fixed it (though man was it confusing to begin with), but I was later able to switch that foreach to static foreach without any breakage. However, if I remove the label now, then it won't compile, since it's a static foreach. So, if I'd used a static foreach from the get-go, it would have actually caught a bug for me before I even ran my tests. But regardless, labeled break definitely works within a static foreach, and I expect that a labeled continue does as well, but I haven't tried it. - Jonathan M Davis
Jan 08 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/8/18 3:07 PM, Jonathan M Davis wrote:
 But regardless, labeled break definitely works within a static foreach, and
 I expect that a labeled continue does as well, but I haven't tried it.
I didn't mean it that way, see my reply to H. -Steve
Jan 09 2018