www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - foreach thoughts

reply Manu <turkeyman gmail.com> writes:
--089e01182c7e8e6e6304efe9e5f1
Content-Type: text/plain; charset=UTF-8

So, the last few days, 2 things have been coming up constantly.

foreach is like, the best thing about D. But I often get caught by 2 little
details that result in my having to create a bunch more visual noise.

1. A termination condition (ie, while)

foreach(t; things) iterates each thing, but it's common in traditional for
loops to have an && in the second 'while' term, to add an additional
termination condition.
for(i=0; i<things.length && things[i].someCondition; ++i)

Or with foreach:
foreach(i, t; things)
{
  if(t.someCondition)
    break;
  ...
}

I often feel like I want something like this where I would have the
opportunity to add that additional term I lose with a traditional for loop:
  foreach(i, t; things; t.someCondition)


2. A filter

The other thing is the ability to skip uninteresting elements. This is
typically performed with the first line of the loop testing a condition,
and then continue:
foreach(i, t; things)
{
  if(!t.isInteresting)
    continue;
  ...
}


I'm finding in practise that at least one of these seems to pop up on the
vast majority of loops I'm writing. It produces a lot of visual noise, and
I'm finding it's quite a distraction from the otherwise relative tidiness
of my code.

How have others dealt with this? I wonder if it's worth exploring a 3rd
term in the foreach statement?

I've tried to approach the problem with std.algorithm, but I find the
std.algorithm statement to be much more noisy and usually longer when the
loops are sufficiently simple (as they usually are in my case, which is why
the trivial conditions are so distracting by contrast).
I also find that the exclamation mark overload in typical std.algorithm
statements tends to visually obscure the trivial condition I'm wanting to
insert in the first place.
It's also really hard to debug std.algorithm statements, you can't step the
debugger anymore.

Also, I have to import std.algorithm, which then imports the universe... >_<

--089e01182c7e8e6e6304efe9e5f1
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">So, the last few days, 2 things have been coming up consta=
ntly.<div><br></div><div>foreach is like, the best thing about D. But I oft=
en get caught by 2 little details that result in my having to create a bunc=
h more visual noise.</div>
<div><br></div><div>1. A termination condition (ie, while)</div><div><br></=
div><div>foreach(t; things) iterates each thing, but it&#39;s common in tra=
ditional for loops to have an &amp;&amp; in the second &#39;while&#39; term=
, to add an additional termination condition.</div>
<div>for(i=3D0; i&lt;things.length &amp;&amp; things[i].someCondition; ++i)=
</div><div><br></div><div>Or with foreach:</div><div>foreach(i, t; things)<=
/div><div>{</div><div>=C2=A0 if(t.someCondition)</div><div>=C2=A0 =C2=A0 br=
eak;</div><div>
=C2=A0 ...</div><div>}</div><div><br></div><div>I often feel like I want so=
mething like this where I would have the opportunity to add that additional=
 term I lose with a traditional for loop:</div><div>=C2=A0 foreach(i, t; th=
ings; t.someCondition)</div>
<div><br></div><div><br></div><div>2. A filter</div><div><br></div><div>The=
 other thing is the ability to skip uninteresting elements. This is typical=
ly performed with the first line of the loop testing a condition, and then =
continue:</div>
<div>foreach(i, t; things)</div><div>{</div><div>=C2=A0 if(!t.isInteresting=
)</div><div>=C2=A0 =C2=A0 continue;</div><div>=C2=A0 ...</div><div>}</div><=
div><br></div><div><br></div><div>I&#39;m finding in practise that at least=
 one of these seems to pop up on the vast majority of loops I&#39;m writing=
. It produces a lot of visual noise, and I&#39;m finding it&#39;s quite a d=
istraction from the otherwise relative tidiness of my code.</div>
<div><br></div><div>How have others dealt with this? I wonder if it&#39;s w=
orth exploring a 3rd term in the foreach statement?</div><div><br></div><di=
v>I&#39;ve tried to approach the problem with std.algorithm, but I find the=
 std.algorithm statement to be much more noisy and usually longer when the =
loops are sufficiently simple (as they usually are in my case, which is why=
 the trivial conditions are so distracting by contrast).</div>
<div>I also find that the exclamation mark overload in typical std.algorith=
m statements tends to visually obscure the trivial condition I&#39;m wantin=
g to insert in the first place.</div><div>It&#39;s also really hard to debu=
g std.algorithm statements, you can&#39;t step the debugger anymore.</div>
<div><br></div><div>Also, I have to import std.algorithm, which then import=
s the universe... &gt;_&lt;</div></div>

--089e01182c7e8e6e6304efe9e5f1--
Jan 14 2014
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 14 January 2014 at 08:23:05 UTC, Manu wrote:
 So, the last few days, 2 things have been coming up constantly.

 foreach is like, the best thing about D. But I often get caught 
 by 2 little
 details that result in my having to create a bunch more visual 
 noise.

 1. A termination condition (ie, while)

 foreach(t; things) iterates each thing, but it's common in 
 traditional for
 loops to have an && in the second 'while' term, to add an 
 additional
 termination condition.
 for(i=0; i<things.length && things[i].someCondition; ++i)

 Or with foreach:
 foreach(i, t; things)
 {
   if(t.someCondition)
     break;
   ...
 }

 I often feel like I want something like this where I would have 
 the
 opportunity to add that additional term I lose with a 
 traditional for loop:
   foreach(i, t; things; t.someCondition)

until
 2. A filter

 The other thing is the ability to skip uninteresting elements. 
 This is
 typically performed with the first line of the loop testing a 
 condition,
 and then continue:
 foreach(i, t; things)
 {
   if(!t.isInteresting)
     continue;
   ...
 }


 I'm finding in practise that at least one of these seems to pop 
 up on the
 vast majority of loops I'm writing. It produces a lot of visual 
 noise, and
 I'm finding it's quite a distraction from the otherwise 
 relative tidiness
 of my code.

filter
 How have others dealt with this? I wonder if it's worth 
 exploring a 3rd
 term in the foreach statement?

We have the tool to build ranges that does what you ask for and feed foreach with them.
Jan 14 2014
prev sibling next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 14 January 2014 at 08:23:05 UTC, Manu wrote:
 1. A termination condition (ie, while)

 foreach(t; things) iterates each thing, but it's common in 
 traditional for
 loops to have an && in the second 'while' term, to add an 
 additional
 termination condition.
 for(i=0; i<things.length && things[i].someCondition; ++i)

 Or with foreach:
 foreach(i, t; things)
 {
   if(t.someCondition)
     break;
   ...
 }

foreach(t; things.until!(t => t.someCondition)) { } Unfortunately foreach over a range does not automatically support an index loop variable. We could add something like std.range.enumerate to support this, but I think it's a common enough requirement that a language amendment is warranted (there are some subtleties involved in implementing it though - specifically when combined with automatic tuple expansion).
 2. A filter

 The other thing is the ability to skip uninteresting elements. 
 This is
 typically performed with the first line of the loop testing a 
 condition,
 and then continue:
 foreach(i, t; things)
 {
   if(!t.isInteresting)
     continue;
   ...
 }

foreach(t; things.filter!(t => t.isInteresting)) { } Ditto about the index loop variable.
 I've tried to approach the problem with std.algorithm, but I 
 find the
 std.algorithm statement to be much more noisy and usually 
 longer when the
 loops are sufficiently simple (as they usually are in my case, 
 which is why
 the trivial conditions are so distracting by contrast).

The two examples above look a *lot* cleaner and less noisy (declarative!) to me than the imperative approach using if-break or if-continue.
 I also find that the exclamation mark overload in typical 
 std.algorithm
 statements tends to visually obscure the trivial condition I'm 
 wanting to
 insert in the first place.

You'll have to get used to the exclamation mark, otherwise you'll never be able to fully appreciate D's generic programming. I quite like it - I don't think there's anything objectively ugly about it.
 Also, I have to import std.algorithm, which then imports the 
 universe... >_<

This is fixable. We shouldn't reach for language changes to compensate for library deficiencies.
Jan 14 2014
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-01-14 09:36, Jakob Ovrum wrote:

 You'll have to get used to the exclamation mark, otherwise you'll never
 be able to fully appreciate D's generic programming. I quite like it - I
 don't think there's anything objectively ugly about it.

Or, in this case, fix the compiler to be able to inline delegates. Or is there any other advantage of using an alias parameter instead of a delegate? -- /Jacob Carlborg
Jan 14 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 01/14/2014 11:09 AM, Jacob Carlborg wrote:
 On 2014-01-14 09:36, Jakob Ovrum wrote:

 You'll have to get used to the exclamation mark, otherwise you'll never
 be able to fully appreciate D's generic programming. I quite like it - I
 don't think there's anything objectively ugly about it.

Or, in this case, fix the compiler to be able to inline delegates. Or is there any other advantage of using an alias parameter instead of a delegate?

1. It is likely to result in faster code when inlining fails. Eg. it will often not allocate when the delegate would, due to nested template instantiation simplifying escape analysis. 2. IFTI limitations. Eg. the following cannot work with the current language unless 'map' is specialized to ranges of int: [1,2,3].map(x=>2*x)
Jan 14 2014
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01/14/2014 09:43 AM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 objectively ugly

 because "!" implies a boolean expression, but that is
 off-topic.

This is the binary usage of "!". It does not imply a boolean expression similarly to how a * b does not imply an expression on pointers.
Jan 14 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01/14/2014 04:40 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 14 January 2014 at 15:34:40 UTC, Dominikus Dittes Scherkl
 wrote:
 I think it is. It would be far worse if the semantics were related.

No. Array sub script and array initialization are both array-like, so no confusion. Same with "+" for concatenation (if you don't do implicit casting) and addition, they are both additive-like.

Nope. They are both monoid operations. Addition is commutative, and it is conventional in abstract algebra to use "+" to denote commutative operations only. eg. http://en.wikipedia.org/wiki/Abelian_group#Notation . assert("123abc" == "123" + "abc"); hurts my eyes.
 The human mind categorize in a fuzzy fashion.

Jan 14 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 01/14/2014 05:50 PM, "Ola Fosheim Grøstad" 
<ola.fosheim.grostad+dlang gmail.com>" wrote:
 On Tuesday, 14 January 2014 at 16:32:55 UTC, Timon Gehr wrote:
 assert("123abc" == "123" + "abc"); hurts my eyes.

Doesn't matter.

Thanks.
Jan 14 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/14/14 1:04 AM, Manu wrote:
 /agree completely.
 This is nice, I didn't think of writing statements like that :)
 That's precisely the sort of suggestion I was hoping for. I'll continue
 like this.

I think I died and got to heaven. Andrei
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 08:36:53 UTC, Jakob Ovrum wrote:
 You'll have to get used to the exclamation mark, otherwise 
 you'll never be able to fully appreciate D's generic 
 programming. I quite like it - I don't think there's anything 
 objectively ugly about it.

You subjectively think that there is nothing objectively ugly about it? :-) It is objectively ugly because "!" implies a boolean expression, but that is off-topic. I agree that chaining of filters and sorting rules is a good solution, provided that you have a high level optimizer capable of transforming the chain into something optimal. You basically need some sort of term-rewriting.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c2c4847cfa5504efea7956
Content-Type: text/plain; charset=UTF-8

On 14 January 2014 18:36, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 14 January 2014 at 08:23:05 UTC, Manu wrote:

 1. A termination condition (ie, while)

 foreach(t; things) iterates each thing, but it's common in traditional for
 loops to have an && in the second 'while' term, to add an additional
 termination condition.
 for(i=0; i<things.length && things[i].someCondition; ++i)

 Or with foreach:
 foreach(i, t; things)
 {
   if(t.someCondition)
     break;
   ...
 }

foreach(t; things.until!(t => t.someCondition)) { } Unfortunately foreach over a range does not automatically support an index loop variable. We could add something like std.range.enumerate to support this, but I think it's a common enough requirement that a language amendment is warranted (there are some subtleties involved in implementing it though - specifically when combined with automatic tuple expansion). 2. A filter
 The other thing is the ability to skip uninteresting elements. This is
 typically performed with the first line of the loop testing a condition,
 and then continue:
 foreach(i, t; things)
 {
   if(!t.isInteresting)
     continue;
   ...
 }

foreach(t; things.filter!(t => t.isInteresting)) { } Ditto about the index loop variable. I've tried to approach the problem with std.algorithm, but I find the
 std.algorithm statement to be much more noisy and usually longer when the
 loops are sufficiently simple (as they usually are in my case, which is
 why
 the trivial conditions are so distracting by contrast).

The two examples above look a *lot* cleaner and less noisy (declarative!) to me than the imperative approach using if-break or if-continue.

/agree completely. This is nice, I didn't think of writing statements like that :) That's precisely the sort of suggestion I was hoping for. I'll continue like this. --001a11c2c4847cfa5504efea7956 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 18:36, Jakob Ovrum <span dir=3D"ltr">&lt;<a href=3D"mailto:j= akobovrum gmail.com" target=3D"_blank">jakobovrum gmail.com</a>&gt;</span> = wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, 14 January 201= 4 at 08:23:05 UTC, Manu wrote:<br> </div><div class=3D"im"><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> 1. A termination condition (ie, while)<br> <br> foreach(t; things) iterates each thing, but it&#39;s common in traditional = for<br> loops to have an &amp;&amp; in the second &#39;while&#39; term, to add an a= dditional<br> termination condition.<br> for(i=3D0; i&lt;things.length &amp;&amp; things[i].someCondition; ++i)<br> <br> Or with foreach:<br> foreach(i, t; things)<br> {<br> =C2=A0 if(t.someCondition)<br> =C2=A0 =C2=A0 break;<br> =C2=A0 ...<br> }<br> </blockquote> <br></div> foreach(t; things.until!(t =3D&gt; t.someCondition))<br> {<br> }<br> <br> Unfortunately foreach over a range does not automatically support an index = loop variable. We could add something like std.range.enumerate to support t= his, but I think it&#39;s a common enough requirement that a language amend= ment is warranted (there are some subtleties involved in implementing it th= ough - specifically when combined with automatic tuple expansion).<div clas= s=3D"im"> <br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> 2. A filter<br> <br> The other thing is the ability to skip uninteresting elements. This is<br> typically performed with the first line of the loop testing a condition,<br=

foreach(i, t; things)<br> {<br> =C2=A0 if(!t.isInteresting)<br> =C2=A0 =C2=A0 continue;<br> =C2=A0 ...<br> }<br> </blockquote> <br></div> foreach(t; things.filter!(t =3D&gt; t.isInteresting))<br> {<br> }<br> <br> Ditto about the index loop variable.<div class=3D"im"><br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;ve tried to approach the problem with std.algorithm, but I find the<b= r> std.algorithm statement to be much more noisy and usually longer when the<b= r> loops are sufficiently simple (as they usually are in my case, which is why= <br> the trivial conditions are so distracting by contrast).<br> </blockquote> <br></div> The two examples above look a *lot* cleaner and less noisy (declarative!) t= o me than the imperative approach using if-break or if-continue.</blockquot= e><div><br></div><div>/agree completely.</div><div>This is nice, I didn&#39= ;t think of writing statements like that :)</div> <div>That&#39;s precisely the sort of suggestion I was hoping for. I&#39;ll= continue like this.</div></div></div></div> --001a11c2c4847cfa5504efea7956--
Jan 14 2014
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 14 January 2014 at 08:43:37 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 14 January 2014 at 08:36:53 UTC, Jakob Ovrum wrote:
 You'll have to get used to the exclamation mark, otherwise 
 you'll never be able to fully appreciate D's generic 
 programming. I quite like it - I don't think there's anything 
 objectively ugly about it.

You subjectively think that there is nothing objectively ugly about it? :-) It is objectively ugly because "!" implies a boolean expression, but that is off-topic.

This argument is stupid. It's the same argument as the famous "Yeah, well, that's just, like, your opinion, man", or the "please put `I think...` in front of all your sentences!" argument. The burden of proof is on the person who first claims it's deficient. Saying that I can't think of anything objectively wrong with it serves the purpose of inviting Manu to provide some kind of argument, as I can't think of anything *obviously* wrong about it that goes unsaid. It's common to overload tokens in programming languages, and it's usually only a problem for beginners who aren't used to the particular language's choice of overloads yet (Ruby is good example of a language with rather extreme token reuse) - humans are pretty good at context-sensitive parsing. From a character-by-character perspective it's particularly common, with the bitwise shift operators having nothing to do with comparisons, bitwise xor having nothing to do with exponents etc. Regardless of whether binary ! is "ugly" or not, it's still better than introducing language features left and right to avoid templates, and it's still better than C++'s template instantiation syntax :)
 I agree that chaining of filters and sorting rules is a good 
 solution, provided that you have a high level optimizer capable 
 of transforming the chain into something optimal. You basically 
 need some sort of term-rewriting.

LDC and GDC are capable of unravelling the (fairly thin) abstraction. All it requires is the ability to inline direct function calls to small functions - the aforementioned compilers always have this capability for templated functions. DMD is hit and miss, but I think there was a recent improvement to its inliner... luckily this is still the domain of micro-optimization.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b3a8674d2866304efea853e
Content-Type: text/plain; charset=UTF-8

On 14 January 2014 19:04, Manu <turkeyman gmail.com> wrote:

 On 14 January 2014 18:36, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 14 January 2014 at 08:23:05 UTC, Manu wrote:

 1. A termination condition (ie, while)

 foreach(t; things) iterates each thing, but it's common in traditional
 for
 loops to have an && in the second 'while' term, to add an additional
 termination condition.
 for(i=0; i<things.length && things[i].someCondition; ++i)

 Or with foreach:
 foreach(i, t; things)
 {
   if(t.someCondition)
     break;
   ...
 }

foreach(t; things.until!(t => t.someCondition)) { } Unfortunately foreach over a range does not automatically support an index loop variable. We could add something like std.range.enumerate to support this, but I think it's a common enough requirement that a language amendment is warranted (there are some subtleties involved in implementing it though - specifically when combined with automatic tuple expansion). 2. A filter
 The other thing is the ability to skip uninteresting elements. This is
 typically performed with the first line of the loop testing a condition,
 and then continue:
 foreach(i, t; things)
 {
   if(!t.isInteresting)
     continue;
   ...
 }

foreach(t; things.filter!(t => t.isInteresting)) { } Ditto about the index loop variable. I've tried to approach the problem with std.algorithm, but I find the
 std.algorithm statement to be much more noisy and usually longer when the
 loops are sufficiently simple (as they usually are in my case, which is
 why
 the trivial conditions are so distracting by contrast).

The two examples above look a *lot* cleaner and less noisy (declarative!) to me than the imperative approach using if-break or if-continue.

/agree completely. This is nice, I didn't think of writing statements like that :) That's precisely the sort of suggestion I was hoping for. I'll continue like this.

Can anyone comment on the codegen when using these statements? Is it identical to my reference 'if' statement? Liberal use of loops like this will probably obliterate unoptimised performance... :/ --047d7b3a8674d2866304efea853e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 19:04, Manu <span dir=3D"ltr">&lt;<a href=3D"mailto:turkeyma= n gmail.com" target=3D"_blank">turkeyman gmail.com</a>&gt;</span> wrote:<br=
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1=

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div=
<div class=3D"h5">On 14 January 2014 18:36, Jakob Ovrum <span dir=3D"ltr">=

ail.com</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>On Tuesday, 14 January 2014 at 08:23:05= UTC, Manu wrote:<br> </div><div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bor= der-left:1px #ccc solid;padding-left:1ex"> 1. A termination condition (ie, while)<br> <br> foreach(t; things) iterates each thing, but it&#39;s common in traditional = for<br> loops to have an &amp;&amp; in the second &#39;while&#39; term, to add an a= dditional<br> termination condition.<br> for(i=3D0; i&lt;things.length &amp;&amp; things[i].someCondition; ++i)<br> <br> Or with foreach:<br> foreach(i, t; things)<br> {<br> =C2=A0 if(t.someCondition)<br> =C2=A0 =C2=A0 break;<br> =C2=A0 ...<br> }<br> </blockquote> <br></div> foreach(t; things.until!(t =3D&gt; t.someCondition))<br> {<br> }<br> <br> Unfortunately foreach over a range does not automatically support an index = loop variable. We could add something like std.range.enumerate to support t= his, but I think it&#39;s a common enough requirement that a language amend= ment is warranted (there are some subtleties involved in implementing it th= ough - specifically when combined with automatic tuple expansion).<div> <br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> 2. A filter<br> <br> The other thing is the ability to skip uninteresting elements. This is<br> typically performed with the first line of the loop testing a condition,<br=

foreach(i, t; things)<br> {<br> =C2=A0 if(!t.isInteresting)<br> =C2=A0 =C2=A0 continue;<br> =C2=A0 ...<br> }<br> </blockquote> <br></div> foreach(t; things.filter!(t =3D&gt; t.isInteresting))<br> {<br> }<br> <br> Ditto about the index loop variable.<div><br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;ve tried to approach the problem with std.algorithm, but I find the<b= r> std.algorithm statement to be much more noisy and usually longer when the<b= r> loops are sufficiently simple (as they usually are in my case, which is why= <br> the trivial conditions are so distracting by contrast).<br> </blockquote> <br></div> The two examples above look a *lot* cleaner and less noisy (declarative!) t= o me than the imperative approach using if-break or if-continue.</blockquot= e><div><br></div></div></div><div>/agree completely.</div><div>This is nice= , I didn&#39;t think of writing statements like that :)</div> <div>That&#39;s precisely the sort of suggestion I was hoping for. I&#39;ll= continue like this.</div></div></div></div> </blockquote></div><br></div><div class=3D"gmail_extra">Can anyone comment = on the codegen when using these statements? Is it identical to my reference= &#39;if&#39; statement?</div><div class=3D"gmail_extra">Liberal use of loo= ps like this will probably obliterate unoptimised performance... :/<br> </div></div> --047d7b3a8674d2866304efea853e--
Jan 14 2014
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 14 January 2014 at 09:06:23 UTC, Jakob Ovrum wrote:

 It's common to overload tokens in programming languages, and 
 it's usually only a problem for beginners who aren't used to 
 the particular language's choice of overloads yet (Ruby is good 
 example of a language with rather extreme token reuse) - humans 
 are pretty good at context-sensitive parsing. From a 
 character-by-character perspective it's particularly common, 
 with the bitwise shift operators having nothing to do with 
 comparisons, bitwise xor having nothing to do with exponents 
 etc.

BTW, I am not implying Manu is a beginner, I suspect he had something more specific in mind when he mentioned the instantiation syntax.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e01160266c39a9204efea9a58
Content-Type: text/plain; charset=UTF-8

On 14 January 2014 19:09, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 14 January 2014 at 09:06:23 UTC, Jakob Ovrum wrote:

  It's common to overload tokens in programming languages, and it's usually
 only a problem for beginners who aren't used to the particular language's
 choice of overloads yet (Ruby is good example of a language with rather
 extreme token reuse) - humans are pretty good at context-sensitive parsing.
 From a character-by-character perspective it's particularly common, with
 the bitwise shift operators having nothing to do with comparisons, bitwise
 xor having nothing to do with exponents etc.

BTW, I am not implying Manu is a beginner, I suspect he had something more specific in mind when he mentioned the instantiation syntax.

Thank you... I was going to comment, but I restrained myself ;) --089e01160266c39a9204efea9a58 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 19:09, Jakob Ovrum <span dir=3D"ltr">&lt;<a href=3D"mailto:j= akobovrum gmail.com" target=3D"_blank">jakobovrum gmail.com</a>&gt;</span> = wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, 14 January 201= 4 at 09:06:23 UTC, Jakob Ovrum wrote:<br> <br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> It&#39;s common to overload tokens in programming languages, and it&#39;s u= sually only a problem for beginners who aren&#39;t used to the particular l= anguage&#39;s choice of overloads yet (Ruby is good example of a language w= ith rather extreme token reuse) - humans are pretty good at context-sensiti= ve parsing. From a character-by-character perspective it&#39;s particularly= common, with the bitwise shift operators having nothing to do with compari= sons, bitwise xor having nothing to do with exponents etc.<br> </blockquote> <br></div> BTW, I am not implying Manu is a beginner, I suspect he had something more = specific in mind when he mentioned the instantiation syntax.<br> </blockquote></div><br></div><div class=3D"gmail_extra">Thank you... I was = going to comment, but I restrained myself ;)</div></div> --089e01160266c39a9204efea9a58--
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e01538848c8e8aa04efeab819
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 14 January 2014 19:06, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 14 January 2014 at 08:43:37 UTC, Ola Fosheim Gr=C3=B8stad wro=

 On Tuesday, 14 January 2014 at 08:36:53 UTC, Jakob Ovrum wrote:

 You'll have to get used to the exclamation mark, otherwise you'll never
 be able to fully appreciate D's generic programming. I quite like it - =



 don't think there's anything objectively ugly about it.

You subjectively think that there is nothing objectively ugly about it? :-) It is objectively ugly because "!" implies a boolean expression, but that is off-topic.

This argument is stupid. It's the same argument as the famous "Yeah, well=

 that's just, like, your opinion, man", or the "please put `I think...` in
 front of all your sentences!" argument.

 The burden of proof is on the person who first claims it's deficient.
 Saying that I can't think of anything objectively wrong with it serves th=

 purpose of inviting Manu to provide some kind of argument, as I can't thi=

 of anything *obviously* wrong about it that goes unsaid.

 It's common to overload tokens in programming languages, and it's usually
 only a problem for beginners who aren't used to the particular language's
 choice of overloads yet (Ruby is good example of a language with rather
 extreme token reuse) - humans are pretty good at context-sensitive parsin=

 From a character-by-character perspective it's particularly common, with
 the bitwise shift operators having nothing to do with comparisons, bitwis=

 xor having nothing to do with exponents etc.

 Regardless of whether binary ! is "ugly" or not, it's still better than
 introducing language features left and right to avoid templates, and it's
 still better than C++'s template instantiation syntax :)

Personally, I generally like the '!' syntax, except when it's in conjunction with lambda's where it can kinda ruins the statements a bit. I agree that chaining of filters and sorting rules is a good solution,
 provided that you have a high level optimizer capable of transforming th=


 chain into something optimal. You basically need some sort of
 term-rewriting.

LDC and GDC are capable of unravelling the (fairly thin) abstraction. All it requires is the ability to inline direct function calls to small functions - the aforementioned compilers always have this capability for templated functions. DMD is hit and miss, but I think there was a recent improvement to its inliner... luckily this is still the domain of micro-optimization.

I'm quite concerned about unoptimised performance too. I know that's unusual, but I'm often left wondering what to do about it. Imagine, iterating over an array, if I use .filter(lambda) or something, there's now an additional 2 function calls at least per iteration, as opposed to the original zero. If this is idiomatic (I'd like to think it should be since it's fairly tidy), then we commit to a serious performance problem in unoptimised code. Liberal use of .empty and things too is a problem in unoptimised code... :/ --089e01538848c8e8aa04efeab819 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 19:06, Jakob Ovrum <span dir=3D"ltr">&lt;<a href=3D"mailto:j= akobovrum gmail.com" target=3D"_blank">jakobovrum gmail.com</a>&gt;</span> = wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, 14 January 201= 4 at 08:43:37 UTC, Ola Fosheim Gr=C3=B8stad wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> On Tuesday, 14 January 2014 at 08:36:53 UTC, Jakob Ovrum wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> You&#39;ll have to get used to the exclamation mark, otherwise you&#39;ll n= ever be able to fully appreciate D&#39;s generic programming. I quite like = it - I don&#39;t think there&#39;s anything objectively ugly about it.<br> </blockquote> <br> You subjectively think that there is nothing objectively ugly about it? :-)= It is objectively ugly because &quot;!&quot; implies a boolean expression,= but that is off-topic.<br> </blockquote> <br></div> This argument is stupid. It&#39;s the same argument as the famous &quot;Yea= h, well, that&#39;s just, like, your opinion, man&quot;, or the &quot;pleas= e put `I think...` in front of all your sentences!&quot; argument.<br> <br> The burden of proof is on the person who first claims it&#39;s deficient. S= aying that I can&#39;t think of anything objectively wrong with it serves t= he purpose of inviting Manu to provide some kind of argument, as I can&#39;= t think of anything *obviously* wrong about it that goes unsaid.<br> <br> It&#39;s common to overload tokens in programming languages, and it&#39;s u= sually only a problem for beginners who aren&#39;t used to the particular l= anguage&#39;s choice of overloads yet (Ruby is good example of a language w= ith rather extreme token reuse) - humans are pretty good at context-sensiti= ve parsing. From a character-by-character perspective it&#39;s particularly= common, with the bitwise shift operators having nothing to do with compari= sons, bitwise xor having nothing to do with exponents etc.<br> <br> Regardless of whether binary ! is &quot;ugly&quot; or not, it&#39;s still b= etter than introducing language features left and right to avoid templates,= and it&#39;s still better than C++&#39;s template instantiation syntax :)<= /blockquote> <div><br></div><div>Personally, I generally like the &#39;!&#39; syntax, ex= cept when it&#39;s in conjunction with lambda&#39;s where it can kinda ruin= s the statements a bit.</div><div><br></div><div><br></div><blockquote clas= s=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pad= ding-left:1ex"> <div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I agree that chaining of filters and sorting rules is a good solution, prov= ided that you have a high level optimizer capable of transforming the chain= into something optimal. You basically need some sort of term-rewriting.<br=

</blockquote> <br></div> LDC and GDC are capable of unravelling the (fairly thin) abstraction. All i= t requires is the ability to inline direct function calls to small function= s - the aforementioned compilers always have this capability for templated = functions.<br> <br> DMD is hit and miss, but I think there was a recent improvement to its inli= ner... luckily this is still the domain of micro-optimization.<br> </blockquote></div><br></div><div class=3D"gmail_extra">I&#39;m quite conce= rned about unoptimised performance too. I know that&#39;s unusual, but I&#3= 9;m often left wondering what to do about it.</div><div class=3D"gmail_extr= a"> <br></div><div class=3D"gmail_extra">Imagine, iterating over an array, if I= use .filter(lambda) or something, there&#39;s now an additional 2 function= calls at least per iteration, as opposed to the original zero. If this is = idiomatic (I&#39;d like to think it should be since it&#39;s fairly tidy), = then we commit to a serious performance problem in unoptimised code. Libera= l use of .empty and things too is a problem in unoptimised code... :/</div> </div> --089e01538848c8e8aa04efeab819--
Jan 14 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
I also think proposed syntax does not give any advantage, not 
worth any language change. If there are any optimization issues 
with std.algorithm those must be fixed instead. Idiomatic D is 
all about ranges so such use case must be as efficient a possible.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--001a11c1bce829588c04efead155
Content-Type: text/plain; charset=UTF-8

On 14 January 2014 19:22, Dicebot <public dicebot.lv> wrote:

 I also think proposed syntax does not give any advantage, not worth any
 language change. If there are any optimization issues with std.algorithm
 those must be fixed instead. Idiomatic D is all about ranges so such use
 case must be as efficient a possible.

Perhaps __forceinline in the future may create some opportunities to do these things without creating a crap load of superfluous function calls. --001a11c1bce829588c04efead155 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 19:22, Dicebot <span dir=3D"ltr">&lt;<a href=3D"mailto:publi= c dicebot.lv" target=3D"_blank">public dicebot.lv</a>&gt;</span> wrote:<br>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I also think proposed syntax does not give any advantage, not worth any lan= guage change. If there are any optimization issues with std.algorithm those= must be fixed instead. Idiomatic D is all about ranges so such use case mu= st be as efficient a possible.<br> </blockquote></div><br></div><div class=3D"gmail_extra">Perhaps __forceinli= ne in the future may create some opportunities to do these things without c= reating a crap load of superfluous function calls.</div></div> --001a11c1bce829588c04efead155--
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 09:06:23 UTC, Jakob Ovrum wrote:
 Regardless of whether binary ! is "ugly" or not, it's still 
 better than introducing language features left and right to 
 avoid templates, and it's still better than C++'s template 
 instantiation syntax :)

C++'s templates are horrible for other reasons! D's metaprogramming is much more readable (except for template instantiation, of course ;^)
 LDC and GDC are capable of unravelling the (fairly thin) 
 abstraction. All it requires is the ability to inline direct 
 function calls to small functions - the aforementioned 
 compilers always have this capability for templated functions.

But what happens when you chain multiple modifying functions? A high level optimizer can have heuristics to transform a complex expression into some kind of normal form which the lower level have heuristics to deal with.
 DMD is hit and miss, but I think there was a recent improvement 
 to its inliner... luckily this is still the domain of 
 micro-optimization.

Not really! I think you need high level optimization in order to have an efficient generic programming environment. Like, if you want to encapsulate generators. (e.g. if you ask a factory for a filtered and sorted list, which returns a generator, then you need more filters, then you send it to another function which is generic and applies sort again etc)
Jan 14 2014
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 14 January 2014 at 09:21:59 UTC, Manu wrote:

 Personally, I generally like the '!' syntax, except when it's in
 conjunction with lambda's where it can kinda ruins the 
 statements a bit.

I know what you mean, but I think it's more due to the extra parentheses than the exclamation mark, in which case UFCS really helps, e.g. array.map!(...).filter!(...).copy(sink); When this is not an option (due to passing more than one argument), splitting the expression into multiple lines still helps.
 I'm quite concerned about unoptimised performance too. I know 
 that's
 unusual, but I'm often left wondering what to do about it.

 Imagine, iterating over an array, if I use .filter(lambda) or 
 something,
 there's now an additional 2 function calls at least per 
 iteration, as
 opposed to the original zero. If this is idiomatic (I'd like to 
 think it
 should be since it's fairly tidy), then we commit to a serious 
 performance
 problem in unoptimised code. Liberal use of .empty and things 
 too is a
 problem in unoptimised code... :/

I think it is the price we pay for choosing function-based generic programming (that is, it's not a problem for mixin-based generics). Idiomatic C++ has the same compromise (except, of course, the mixin approach is only available as the less powerful text preprocessor).
Jan 14 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 14 January 2014 at 09:28:49 UTC, Manu wrote:
 Perhaps __forceinline in the future may create some 
 opportunities to do
 these things without creating a crap load of superfluous 
 function calls.

I don't think this is the case for __forceinline - it is a problem with generic optimizer logic if such trivial wrappers don't get inlined. Former is specialized power tool, latter benefits every single program out there.
Jan 14 2014
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Manu:

 So, the last few days, 2 things have been coming up constantly.

 foreach is like, the best thing about D. But I often get caught 
 by 2 little
 details that result in my having to create a bunch more visual 
 noise.

I am against the suggested changes, because they are too much not-orthogonal. One of the few things that I'd like to change in foreach is support for anonymous loops: foreach(;0..10) I'd like an enumerate() in Phobos. I think the current tuple unpacking in foreach is a very limited and partially broken feature that should be deprecated as soon as possible (and later replaced with something better and more general). I think the ! for algorithms and ranges add noise, sometimes I forget them, and I'd like to not need them, but I think they are not going away. So what I'd like are more specific and more clean error messages when I forget them. As D programs use more and more algorithm UFCS chains, D compilers will need to optimize that kind of code better, adding specific high-level and mid-level optimizations (rewrite rules, deforestations, etc). Bye, bearophile
Jan 14 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 01/14/2014 11:09 AM, Tobias Pankrath wrote:
 On Tuesday, 14 January 2014 at 09:55:42 UTC, bearophile wrote:
 As D programs use more and more algorithm UFCS chains, D compilers
 will need to optimize that kind of code better, adding specific
 high-level and mid-level optimizations (rewrite rules, deforestations,
 etc).

You came up with deforestation several times now, so I digged up this paper from wikipedia http://homepages.inf.ed.ac.uk/wadler/papers/deforest/deforest.ps I skimmed it and it seems that the advantage is elimination of intermediate data structures (i.e. lists), which is std.algorithm achieves by design. Could you elaborate how deforestation might apply to std.algorithm?

I guess it would be about eliminating intermediate arrays.
Jan 14 2014
prev sibling next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Tuesday, 14 January 2014 at 09:55:42 UTC, bearophile wrote:
 As D programs use more and more algorithm UFCS chains, D 
 compilers will need to optimize that kind of code better, 
 adding specific high-level and mid-level optimizations (rewrite 
 rules, deforestations, etc).

You came up with deforestation several times now, so I digged up this paper from wikipedia http://homepages.inf.ed.ac.uk/wadler/papers/deforest/deforest.ps I skimmed it and it seems that the advantage is elimination of intermediate data structures (i.e. lists), which is std.algorithm achieves by design. Could you elaborate how deforestation might apply to std.algorithm?
Jan 14 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 14 January 2014 at 09:07:43 UTC, Manu wrote:
 Can anyone comment on the codegen when using these statements? 
 Is it
 identical to my reference 'if' statement?
 Liberal use of loops like this will probably obliterate 
 unoptimised
 performance... :/

In my experience, unoptimised performance will suffer due to the various function calls and intermediate structs which will be generated naively as-written. Generic range programming in D depends heavily on optimisers to denest everything, which gdc and ldc do manage quite successfully. Am I correct in thinking that you want (near)playably fast debug builds? I think that will require some extra thought to achieve in idiomatic D compared to your usual C++ work. pragma(optimise) anyone?
Jan 14 2014
prev sibling next sibling parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Tuesday, 14 January 2014 at 13:04:58 UTC, Timon Gehr wrote:
 On 01/14/2014 11:09 AM, Jacob Carlborg wrote:
 On 2014-01-14 09:36, Jakob Ovrum wrote:

 You'll have to get used to the exclamation mark, otherwise 
 you'll never
 be able to fully appreciate D's generic programming. I quite 
 like it - I
 don't think there's anything objectively ugly about it.

Or, in this case, fix the compiler to be able to inline delegates. Or is there any other advantage of using an alias parameter instead of a delegate?

1. It is likely to result in faster code when inlining fails. Eg. it will often not allocate when the delegate would, due to nested template instantiation simplifying escape analysis. 2. IFTI limitations. Eg. the following cannot work with the current language unless 'map' is specialized to ranges of int: [1,2,3].map(x=>2*x)

3. Lazy ranges don't have to carry around delegates. 4. Algorithms can specialize depending on the argument function. However, this is currently only possibly when string lambdas are used... (e.g. std.algorithm.startsWith does this) 5. Alias parameters receiving function templates have the benefit of being able to instantiate the function template argument multiple times with different template arguments. This is murky territory when it comes to function literals with inferred parameter types though - they are not guaranteed to be implemented in terms of function templates.
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 12:55:45 UTC, Timon Gehr wrote:
 This is the binary usage of "!". It does not imply a boolean 
 expression similarly to how a * b does not imply an expression 
 on pointers.

Are you claiming that the C dereference-operator is a good usability design? If so we really have nothing to discuss! :)
Jan 14 2014
prev sibling next sibling parent "Meta" <jared771 gmail.com> writes:
On Tuesday, 14 January 2014 at 14:27:02 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 14 January 2014 at 12:55:45 UTC, Timon Gehr wrote:
 This is the binary usage of "!". It does not imply a boolean 
 expression similarly to how a * b does not imply an expression 
 on pointers.

Are you claiming that the C dereference-operator is a good usability design? If so we really have nothing to discuss! :)

I'm not sure if you're genuinely misunderstanding him or trying as hard as possible to intentionally misrepresent his point.
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 14:38:18 UTC, Meta wrote:
 I'm not sure if you're genuinely misunderstanding him or trying 
 as hard as possible to intentionally misrepresent his point.

I don't think unary/binary is a good enough distinction when the semantics are orthogonal.
Jan 14 2014
prev sibling next sibling parent "Dominikus Dittes Scherkl" writes:
On Tuesday, 14 January 2014 at 15:02:19 UTC, Ola Fosheim Grøstad 
wrote:
 On Tuesday, 14 January 2014 at 14:38:18 UTC, Meta wrote:
 I'm not sure if you're genuinely misunderstanding him or 
 trying as hard as possible to intentionally misrepresent his 
 point.

I don't think unary/binary is a good enough distinction when the semantics are orthogonal.

I think it is. It would be far worse if the semantics were related.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0116026684ae7604efeff6dd
Content-Type: text/plain; charset=UTF-8

On 14 January 2014 20:25, John Colvin <john.loughran.colvin gmail.com>wrote:

 On Tuesday, 14 January 2014 at 09:07:43 UTC, Manu wrote:

 Can anyone comment on the codegen when using these statements? Is it
 identical to my reference 'if' statement?
 Liberal use of loops like this will probably obliterate unoptimised
 performance... :/

In my experience, unoptimised performance will suffer due to the various function calls and intermediate structs which will be generated naively as-written. Generic range programming in D depends heavily on optimisers to denest everything, which gdc and ldc do manage quite successfully. Am I correct in thinking that you want (near)playably fast debug builds? I think that will require some extra thought to achieve in idiomatic D compared to your usual C++ work. pragma(optimise) anyone?

Sorry, that was 2 separate points although it didn't look like it. The first question was assuming full optimisation, is it equivalent to the if statement I demonstrate with full optimisation in practise? (I'll do some tests, but people must have already experimented in various circumstances, and identified patterns?) The second comment about non-optimised builds; you are correct, it's important that debug builds remain useful/playable. I have worked in codebases where debug builds run in the realm of 1-2 fps, literally unplayable and entirely useless.. This is a MASSIVE productivity hinderance. By contrast, at krome, we took great pride in that our debug build ran at around 10-15fps (actually playable, and quite a unique achievement which did not go unappreciated), which meant we could actually debug complex bugs while still being able to reproduce them. This was almost entirely attributed to our (unpopular) insistence on writing our engine in C instead of C++. Sadly, I don't see D being any better than C++ in this case. It's likely worse than C++, since it's more convenient, and therefore more compelling to use these meta features. I'd like to know if anybody can see any path towards inlined lambda's/literal's/micro-functions in non-optimised builds? I guess the first question is, is this a problem that is even worth addressing? How many people are likely to object in principle? --089e0116026684ae7604efeff6dd Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 4 January 2014 20:25, John Colvin <span dir=3D"ltr">&lt;<a href=3D"mailto:j= ohn.loughran.colvin gmail.com" target=3D"_blank">john.loughran.colvin gmail= .com</a>&gt;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">On Tuesday, 14 January 201= 4 at 09:07:43 UTC, Manu wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Can anyone comment on the codegen when using these statements? Is it<br> identical to my reference &#39;if&#39; statement?<br> Liberal use of loops like this will probably obliterate unoptimised<br> performance... :/<br> </blockquote> <br></div> In my experience, unoptimised performance will suffer due to the various fu= nction calls and intermediate structs which will be generated naively as-wr= itten. Generic range programming in D depends heavily on optimisers to dene= st everything, which gdc and ldc do manage quite successfully.<br> <br> Am I correct in thinking that you want (near)playably fast debug builds? I = think that will require some extra thought to achieve in idiomatic D compar= ed to your usual C++ work. pragma(optimise) anyone?<br> </blockquote></div><br></div><div class=3D"gmail_extra">Sorry, that was 2 s= eparate points although it didn&#39;t look like it.</div><div class=3D"gmai= l_extra">The first question was assuming full optimisation, is it equivalen= t to the if statement I demonstrate with full optimisation in practise? (I&= #39;ll do some tests, but people must have already experimented in various = circumstances, and identified patterns?)</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">The second = comment about non-optimised builds; you are correct, it&#39;s important tha= t debug builds remain useful/playable.</div><div class=3D"gmail_extra">I ha= ve worked in codebases where debug builds run in the realm of 1-2 fps, lite= rally unplayable and entirely useless.. This is a MASSIVE productivity hind= erance.</div> <div class=3D"gmail_extra">By contrast, at krome, we took great pride in th= at our debug build ran at around 10-15fps (actually playable, and quite a u= nique achievement which did not go unappreciated), which meant we could act= ually debug complex bugs while still being able to reproduce them.</div> <div class=3D"gmail_extra">This was almost entirely attributed to our (unpo= pular) insistence on writing our engine in C instead of C++.</div><div clas= s=3D"gmail_extra">Sadly, I don&#39;t see D being any better than C++ in thi= s case. It&#39;s likely worse than C++, since it&#39;s more convenient, and= therefore more compelling to use these meta features.</div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">I&#39;d lik= e to know if anybody can see any path towards inlined lambda&#39;s/literal&= #39;s/micro-functions in non-optimised builds?</div><div class=3D"gmail_ext= ra"> I guess the first question is, is this a problem that is even worth address= ing? How many people are likely to object in principle?</div></div> --089e0116026684ae7604efeff6dd--
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 15:34:40 UTC, Dominikus Dittes 
Scherkl wrote:
 I think it is. It would be far worse if the semantics were 
 related.

No. Array sub script and array initialization are both array-like, so no confusion. Same with "+" for concatenation (if you don't do implicit casting) and addition, they are both additive-like. The human mind categorize in a fuzzy fashion.
Jan 14 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 14 January 2014 at 15:37:21 UTC, Manu wrote:
 Sorry, that was 2 separate points although it didn't look like 
 it.
 The first question was assuming full optimisation, is it 
 equivalent to the
 if statement I demonstrate with full optimisation in practise? 
 (I'll do
 some tests, but people must have already experimented in various
 circumstances, and identified patterns?)

Don't think so. LDC is best at doing such transformation per my observations but it still major are for improvements in general.
 I'd like to know if anybody can see any path towards inlined
 lambda's/literal's/micro-functions in non-optimised builds?
 I guess the first question is, is this a problem that is even 
 worth
 addressing? How many people are likely to object in principle?

I'd object against it general. Important trait of such builds is resulting code gen that matches original source code as much as possible. Mass inlining will kill it. But I don't see why you use "debug" and "non-optimised" as synonyms here. For example, in DMD "-release" vs "-debug" are orthogonal to "-O". I see no problems with doing optimised debug build.
Jan 14 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
Some experimental data.

---- test program ----

auto input()
{
     int[] arr;

     foreach (i; 0..6)
     {
         import std.stdio;
         int x;
         readf(" %d ", &x);
         arr ~= x;
     }

     return arr;
}

int main()
{
     import std.algorithm;

     auto arr = input();
     int sum;

     // version algo
     foreach (elem; arr.filter!( x => x % 2 ))
     {
         sum += elem;
     }

     // version raw
     foreach (elem; arr)
     {
         if (elem % 2)
             sum += elem;
     }

     return sum;
}

------

Disassemblies for latest LDC (ldmd2 -O -release -inline):

$ cat test-ldc-algo.dump
0000000000402eb0 <_Dmain>:
   402eb0:	50                   	push   %rax
   402eb1:	e8 fa fe ff ff       	callq  402db0 <_D4test5inputFZAi>
   402eb6:	48 89 c1             	mov    %rax,%rcx
   402eb9:	31 c0                	xor    %eax,%eax
   402ebb:	48 85 c9             	test   %rcx,%rcx
   402ebe:	74 4d                	je     402f0d <_Dmain+0x5d>
   402ec0:	f6 02 01             	testb  $0x1,(%rdx)
   402ec3:	75 0b                	jne    402ed0 <_Dmain+0x20>
   402ec5:	48 83 c2 04          	add    $0x4,%rdx
   402ec9:	48 ff c9             	dec    %rcx
   402ecc:	75 f2                	jne    402ec0 <_Dmain+0x10>
   402ece:	eb 3d                	jmp    402f0d <_Dmain+0x5d>
   402ed0:	31 c0                	xor    %eax,%eax
   402ed2:	eb 34                	jmp    402f08 <_Dmain+0x58>
   402ed4:	66 66 66 2e 0f 1f 84 	data32 data32 nopw 
%cs:0x0(%rax,%rax,1)
   402edb:	00 00 00 00 00
   402ee0:	03 02                	add    (%rdx),%eax
   402ee2:	66 66 66 66 66 2e 0f 	data32 data32 data32 data32 nopw 
%cs:0x0(%rax,%rax,1)
   402ee9:	1f 84 00 00 00 00 00
   402ef0:	48 85 c9             	test   %rcx,%rcx
   402ef3:	74 1a                	je     402f0f <_Dmain+0x5f>
   402ef5:	48 83 f9 01          	cmp    $0x1,%rcx
   402ef9:	74 12                	je     402f0d <_Dmain+0x5d>
   402efb:	48 ff c9             	dec    %rcx
   402efe:	f6 42 04 01          	testb  $0x1,0x4(%rdx)
   402f02:	48 8d 52 04          	lea    0x4(%rdx),%rdx
   402f06:	74 e8                	je     402ef0 <_Dmain+0x40>
   402f08:	48 85 c9             	test   %rcx,%rcx
   402f0b:	75 d3                	jne    402ee0 <_Dmain+0x30>
   402f0d:	5a                   	pop    %rdx
   402f0e:	c3                   	retq
   402f0f:	bf e0 0a 68 00       	mov    $0x680ae0,%edi
   402f14:	be 9e 01 00 00       	mov    $0x19e,%esi
   402f19:	e8 c2 c2 02 00       	callq  42f1e0 <_d_array_bounds>
   402f1e:	66 90                	xchg   %ax,%ax

$ cat test-ldc-raw.dump
0000000000402eb0 <_Dmain>:
   402eb0:	50                   	push   %rax
   402eb1:	e8 fa fe ff ff       	callq  402db0 <_D4test5inputFZAi>
   402eb6:	48 89 c1             	mov    %rax,%rcx
   402eb9:	31 c0                	xor    %eax,%eax
   402ebb:	48 85 c9             	test   %rcx,%rcx
   402ebe:	74 17                	je     402ed7 <_Dmain+0x27>
   402ec0:	8b 3a                	mov    (%rdx),%edi
   402ec2:	89 fe                	mov    %edi,%esi
   402ec4:	c1 e6 1f             	shl    $0x1f,%esi
   402ec7:	c1 fe 1f             	sar    $0x1f,%esi
   402eca:	21 fe                	and    %edi,%esi
   402ecc:	01 f0                	add    %esi,%eax
   402ece:	48 83 c2 04          	add    $0x4,%rdx
   402ed2:	48 ff c9             	dec    %rcx
   402ed5:	75 e9                	jne    402ec0 <_Dmain+0x10>
   402ed7:	5a                   	pop    %rdx
   402ed8:	c3                   	retq
   402ed9:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)

So it does inline but end result is still less optimal 
assembly-wise.
Jan 14 2014
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 14 January 2014 at 16:32:55 UTC, Timon Gehr wrote:
 assert("123abc" == "123" + "abc"); hurts my eyes.

Doesn't matter. The human brain does not work with algebra, it's not a logic engine. It is a fuzzy abstraction engine. Besides, in unary notation you get "111" for 3, "11111" for 5 etc, but this is rather off-topic...
Jan 14 2014
prev sibling next sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
foreach(thought; thoughts)
  if(!thought.isInteresting)
    delete thought;
Jan 14 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 14 January 2014 at 09:07:43 UTC, Manu wrote:
 Can anyone comment on the codegen when using these statements? 
 Is it
 identical to my reference 'if' statement?
 Liberal use of loops like this will probably obliterate 
 unoptimised
 performance... :/

I can't comment for every compiler, but when using LLVM as backend, here is what to expect. LLVM is really good at inlining. However, it inline mostly bottom-up, which mean that the front of the subrange will be inlined in the outer front, popFront of the subrange will be inlined in the popFront of the outer range, etc... You'll generally have pretty good performance, but 2 bad scenarios can happen. For most ranges, the compiler will find good optimization between front/popFront/empty and friends. But the way this stuff will be inlined, you'll get bigger and bigger front/popFront/empty until they all are inlined in the loop and everything is simplified away by the optimizer. If you wrap enough range into one another, you'll reach a point where the compiler will consider that these method became too big to be good candidate to inline. Things should improve over time as LLVM team is working on a better top down inliner. The second caveat is automatic heap promotion of the closure frame pointer. Right now, optimizers aren't really good at heap to stack promotion in D as they mostly don't understand the runtime (LDC have some progress in that direction, but this is still quite simplistic). That mean that you can end up with unwanted heap allocation. Realistically, it is possible for a compiler to see through any reasonably size range code, but for now, it is kind of lacking in some directions.
Jan 14 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Tuesday, 14 January 2014 at 16:31:00 UTC, Dicebot wrote:
 So it does inline but end result is still less optimal 
 assembly-wise.

The problem, by the way, seems to be that the program actually contains two loops that LLVM cannot merge: One in the filter() constructor (skipping the initial run of even elements), and the actual foreach loop in main(). David
Jan 14 2014
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Tuesday, 14 January 2014 at 18:38:27 UTC, David Nadlinger 
wrote:
 On Tuesday, 14 January 2014 at 16:31:00 UTC, Dicebot wrote:
 So it does inline but end result is still less optimal 
 assembly-wise.

The problem, by the way, seems to be that the program actually contains two loops that LLVM cannot merge: One in the filter() constructor (skipping the initial run of even elements), and the actual foreach loop in main(). David

I wonder if there is some way the language could help with that in the common case. For example, provide an opApply for all non-infinite ranges that can be used with foreach instead of front/empty/popFront. I imagine opApply is much easier to inline and optimise than trying to piece range operations and state back together to see the underlying loop.
Jan 14 2014
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 14 January 2014 at 19:05:22 UTC, Peter Alexander 
wrote:
 On Tuesday, 14 January 2014 at 18:38:27 UTC, David Nadlinger 
 wrote:
 On Tuesday, 14 January 2014 at 16:31:00 UTC, Dicebot wrote:
 So it does inline but end result is still less optimal 
 assembly-wise.

The problem, by the way, seems to be that the program actually contains two loops that LLVM cannot merge: One in the filter() constructor (skipping the initial run of even elements), and the actual foreach loop in main(). David

I wonder if there is some way the language could help with that in the common case. For example, provide an opApply for all non-infinite ranges that can be used with foreach instead of front/empty/popFront. I imagine opApply is much easier to inline and optimise than trying to piece range operations and state back together to see the underlying loop.

Actually, you could give an opApply for infinite ranges too. foreach loops can have breaks, and opApply knows how to handle them.
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e013c6fa05736a204eff83869
Content-Type: text/plain; charset=UTF-8

On 15 January 2014 02:16, Dicebot <public dicebot.lv> wrote:

 On Tuesday, 14 January 2014 at 15:37:21 UTC, Manu wrote:

 Sorry, that was 2 separate points although it didn't look like it.
 The first question was assuming full optimisation, is it equivalent to the
 if statement I demonstrate with full optimisation in practise? (I'll do
 some tests, but people must have already experimented in various
 circumstances, and identified patterns?)

Don't think so. LDC is best at doing such transformation per my observations but it still major are for improvements in general.

So you're saying that using these primitives like until! and filter! will not produce the same code as my if statement when optimised? That's worrying. I'll have to try it out. I'd like to know if anybody can see any path towards inlined
 lambda's/literal's/micro-functions in non-optimised builds?
 I guess the first question is, is this a problem that is even worth
 addressing? How many people are likely to object in principle?

I'd object against it general. Important trait of such builds is resulting code gen that matches original source code as much as possible. Mass inlining will kill it. But I don't see why you use "debug" and "non-optimised" as synonyms here. For example, in DMD "-release" vs "-debug" are orthogonal to "-O". I see no problems with doing optimised debug build.

You can't step a -O build, or inspect the value of most variables. So you can't really debug. --089e013c6fa05736a204eff83869 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 5 January 2014 02:16, Dicebot <span dir=3D"ltr">&lt;<a href=3D"mailto:publi= c dicebot.lv" target=3D"_blank">public dicebot.lv</a>&gt;</span> wrote:<br>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <div class=3D"im">On Tuesday, 14 January 2014 at 15:37:21 UTC, Manu wrote:<= br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Sorry, that was 2 separate points although it didn&#39;t look like it.<br> The first question was assuming full optimisation, is it equivalent to the<= br> if statement I demonstrate with full optimisation in practise? (I&#39;ll do= <br> some tests, but people must have already experimented in various<br> circumstances, and identified patterns?)<br> </blockquote> <br></div> Don&#39;t think so. LDC is best at doing such transformation per my observa= tions but it still major are for improvements in general.</blockquote><div>= <br></div><div>So you&#39;re saying that using these primitives like until!= and filter! will not produce the same code as my if statement when optimis= ed?</div> <div>That&#39;s worrying. I&#39;ll have to try it out.</div><div><br></div>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> I&#39;d like to know if anybody can see any path towards inlined<br> lambda&#39;s/literal&#39;s/micro-<u></u>functions in non-optimised builds?<= br> I guess the first question is, is this a problem that is even worth<br> addressing? How many people are likely to object in principle?<br> </blockquote> <br></div> I&#39;d object against it general. Important trait of such builds is result= ing code gen that matches original source code as much as possible. Mass in= lining will kill it. But I don&#39;t see why you use &quot;debug&quot; and = &quot;non-optimised&quot; as synonyms here. For example, in DMD &quot;-rele= ase&quot; vs &quot;-debug&quot; are orthogonal to &quot;-O&quot;. I see no = problems with doing optimised debug build.<br> </blockquote></div><br></div><div class=3D"gmail_extra">You can&#39;t step = a -O build, or inspect the value of most variables. So you can&#39;t really= debug.</div></div> --089e013c6fa05736a204eff83869--
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--047d7b2e4d5227dab104eff8542c
Content-Type: text/plain; charset=UTF-8

On 15 January 2014 03:08, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org
 wrote:

 On 1/14/14 1:04 AM, Manu wrote:

 /agree completely.
 This is nice, I didn't think of writing statements like that :)
 That's precisely the sort of suggestion I was hoping for. I'll continue
 like this.

I think I died and got to heaven.

The problem is I initially tried to use an all-out std.algorithm approach, with map and reduce and stuff... --047d7b2e4d5227dab104eff8542c Content-Type: text/html; charset=UTF-8 <div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 15 January 2014 03:08, Andrei Alexandrescu <span dir="ltr">&lt;<a href="mailto:SeeWebsiteForEmail erdani.org" target="_blank">SeeWebsiteForEmail erdani.org</a>&gt;</span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 1/14/14 1:04 AM, Manu wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> /agree completely.<br> This is nice, I didn&#39;t think of writing statements like that :)<br> That&#39;s precisely the sort of suggestion I was hoping for. I&#39;ll continue<br> like this.<br> </blockquote> <br></div> I think I died and got to heaven.</blockquote><div><br></div><div>The problem is I initially tried to use an all-out std.algorithm approach, with map and reduce and stuff...</div></div></div></div> --047d7b2e4d5227dab104eff8542c--
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0112cb9eba531004eff85cf7
Content-Type: text/plain; charset=UTF-8

On 15 January 2014 02:56, Daniel Murphy <yebbliesnospam gmail.com> wrote:

 foreach(thought; thoughts)
  if(!thought.isInteresting)
    delete thought;

That's a redundant filter, your code effectively does nothing ;) --089e0112cb9eba531004eff85cf7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 5 January 2014 02:56, Daniel Murphy <span dir=3D"ltr">&lt;<a href=3D"mailto= :yebbliesnospam gmail.com" target=3D"_blank">yebbliesnospam gmail.com</a>&g= t;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex">foreach(thought; thoughts)<br> =C2=A0if(!thought.isInteresting)<br> =C2=A0 =C2=A0delete thought;<br> </blockquote></div><br></div><div class=3D"gmail_extra">That&#39;s a redund= ant filter, your code effectively does nothing ;)</div></div> --089e0112cb9eba531004eff85cf7--
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d0445178faf1ae604eff869d3
Content-Type: text/plain; charset=UTF-8

On 15 January 2014 02:30, Dicebot <public dicebot.lv> wrote:

 Some experimental data.

 ---- test program ----
 ...
 So it does inline but end result is still less optimal assembly-wise.

Right, thanks for that. I'm quite surprised by how bad that turned out actually, and with LDC, which is usually the best at optimising that sort of thing. Need to do some intensive experimentation... but this is a bit concerning. Cheers. --f46d0445178faf1ae604eff869d3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 5 January 2014 02:30, Dicebot <span dir=3D"ltr">&lt;<a href=3D"mailto:publi= c dicebot.lv" target=3D"_blank">public dicebot.lv</a>&gt;</span> wrote:<br>= <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Some experimental data.<br> <br> ---- test program ----<br>...<br> So it does inline but end result is still less optimal assembly-wise.<br> </blockquote></div><br></div><div class=3D"gmail_extra">Right, thanks for t= hat.</div><div class=3D"gmail_extra">I&#39;m quite surprised by how bad tha= t turned out actually, and with LDC, which is usually the best at optimisin= g that sort of thing.</div> <div class=3D"gmail_extra">Need to do some intensive experimentation... but= this is a bit concerning.</div><div class=3D"gmail_extra"><br></div><div c= lass=3D"gmail_extra">Cheers.</div></div> --f46d0445178faf1ae604eff869d3--
Jan 14 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 15 January 2014 at 01:42:07 UTC, Manu wrote:
 Right, thanks for that.
 I'm quite surprised by how bad that turned out actually, and 
 with LDC,
 which is usually the best at optimising that sort of thing.
 Need to do some intensive experimentation... but this is a bit 
 concerning.

GCC might do some loop merging, I haven't checked. It's just that this pattern doesn't tend to appear too much in traditional C/C++ code (after all, who splits up their loops into two parts just for fun?), so it could be that the LLVM people just never really bothered to write a pass to merge single loops that have been split (as opposed to classical loop fusion, where the loop ranges are the same, but the operations performed/target data different). Maybe it would be possible to implement something like this fairly easily using the existing LLVM loop analyses though. David
Jan 14 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--f46d044280c098a9a404eff94e6d
Content-Type: text/plain; charset=UTF-8

On 15 January 2014 12:22, David Nadlinger <code klickverbot.at> wrote:

 On Wednesday, 15 January 2014 at 01:42:07 UTC, Manu wrote:

 Right, thanks for that.
 I'm quite surprised by how bad that turned out actually, and with LDC,
 which is usually the best at optimising that sort of thing.
 Need to do some intensive experimentation... but this is a bit concerning.

GCC might do some loop merging, I haven't checked. It's just that this pattern doesn't tend to appear too much in traditional C/C++ code (after all, who splits up their loops into two parts just for fun?), so it could be that the LLVM people just never really bothered to write a pass to merge single loops that have been split (as opposed to classical loop fusion, where the loop ranges are the same, but the operations performed/target data different). Maybe it would be possible to implement something like this fairly easily using the existing LLVM loop analyses though.

Okay. As long as the problem is understood and a solution seems realistic. The closure allocation is also an important problem to fix, but that one's been on the list a long time. --f46d044280c098a9a404eff94e6d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 5 January 2014 12:22, David Nadlinger <span dir=3D"ltr">&lt;<a href=3D"mail= to:code klickverbot.at" target=3D"_blank">code klickverbot.at</a>&gt;</span=
 wrote:<br>

x #ccc solid;padding-left:1ex"><div class=3D"im">On Wednesday, 15 January 2= 014 at 01:42:07 UTC, Manu wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> Right, thanks for that.<br> I&#39;m quite surprised by how bad that turned out actually, and with LDC,<= br> which is usually the best at optimising that sort of thing.<br> Need to do some intensive experimentation... but this is a bit concerning.<= br> </blockquote> <br></div> GCC might do some loop merging, I haven&#39;t checked. It&#39;s just that t= his pattern doesn&#39;t tend to appear too much in traditional C/C++ code (= after all, who splits up their loops into two parts just for fun?), so it c= ould be that the LLVM people just never really bothered to write a pass to = merge single loops that have been split (as opposed to classical loop fusio= n, where the loop ranges are the same, but the operations performed/target = data different).<br> <br> Maybe it would be possible to implement something like this fairly easily u= sing the existing LLVM loop analyses though.</blockquote><div><br></div><di= v>Okay. As long as the problem is understood and a solution seems realistic= .</div> <div>The closure allocation is also an important problem to fix, but that o= ne&#39;s been on the list a long time.</div></div></div></div> --f46d044280c098a9a404eff94e6d--
Jan 14 2014
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 15, 2014 at 11:38:13AM +1000, Manu wrote:
 On 15 January 2014 02:56, Daniel Murphy <yebbliesnospam gmail.com> wrote:
 
 foreach(thought; thoughts)
  if(!thought.isInteresting)
    delete thought;

That's a redundant filter, your code effectively does nothing ;)

And 'delete' is deprecated. :-P T -- Bomb technician: If I'm running, try to keep up.
Jan 14 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 15 January 2014 at 02:22:22 UTC, David Nadlinger 
wrote:
 On Wednesday, 15 January 2014 at 01:42:07 UTC, Manu wrote:
 Right, thanks for that.
 I'm quite surprised by how bad that turned out actually, and 
 with LDC,
 which is usually the best at optimising that sort of thing.
 Need to do some intensive experimentation... but this is a bit 
 concerning.

GCC might do some loop merging, I haven't checked. It's just that this pattern doesn't tend to appear too much in traditional C/C++ code (after all, who splits up their loops into two parts just for fun?), so it could be that the LLVM people just never really bothered to write a pass to merge single loops that have been split (as opposed to classical loop fusion, where the loop ranges are the same, but the operations performed/target data different). Maybe it would be possible to implement something like this fairly easily using the existing LLVM loop analyses though. David

Actually this is a useful optimization technique when you need to skip a lot of objects that way. See http://www.onversity.com/load/d-loop.pdf
Jan 14 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 15 January 2014 at 07:08:14 UTC, deadalnix wrote:
 Actually this is a useful optimization technique when you need 
 to skip a lot of objects that way. See 
 http://www.onversity.com/load/d-loop.pdf

Well, yes, expect for the fact that the case discussed here isn't an instance of the optimization. Even the "pre-skip" loop performs two conditional jumps per array element. And worse, the form the loop is in causes LLVM to miss two additional optimization opportunities, compared to the plain C (D) version: First, LLVM actually optimizes away the even/odd conditional branch in the plain version, replacing it with a shl/sar/and combination. The rotated loop structure in the filter version seems to cause the optimizer to miss that opportunity. And second, as a consequence the loop in the filter version is no longer a good candidate for vectorization, whereas LLVM will happily emit AVX/… instructions in the plain case if you let it. David
Jan 15 2014
prev sibling next sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 14 Jan 2014 19:07:33 +1000
schrieb Manu <turkeyman gmail.com>:

 On 14 January 2014 19:04, Manu <turkeyman gmail.com> wrote:
 
 On 14 January 2014 18:36, Jakob Ovrum <jakobovrum gmail.com> wrote:

 On Tuesday, 14 January 2014 at 08:23:05 UTC, Manu wrote:

 1. A termination condition (ie, while)

 foreach(t; things) iterates each thing, but it's common in traditional
 for
 loops to have an && in the second 'while' term, to add an additional
 termination condition.
 for(i=0; i<things.length && things[i].someCondition; ++i)

 Or with foreach:
 foreach(i, t; things)
 {
   if(t.someCondition)
     break;
   ...
 }

foreach(t; things.until!(t => t.someCondition)) { } Unfortunately foreach over a range does not automatically support an index loop variable. We could add something like std.range.enumerate to support this, but I think it's a common enough requirement that a language amendment is warranted (there are some subtleties involved in implementing it though - specifically when combined with automatic tuple expansion). 2. A filter
 The other thing is the ability to skip uninteresting elements. This is
 typically performed with the first line of the loop testing a condition,
 and then continue:
 foreach(i, t; things)
 {
   if(!t.isInteresting)
     continue;
   ...
 }

foreach(t; things.filter!(t => t.isInteresting)) { } Ditto about the index loop variable. I've tried to approach the problem with std.algorithm, but I find the
 std.algorithm statement to be much more noisy and usually longer when the
 loops are sufficiently simple (as they usually are in my case, which is
 why
 the trivial conditions are so distracting by contrast).

The two examples above look a *lot* cleaner and less noisy (declarative!) to me than the imperative approach using if-break or if-continue.

/agree completely. This is nice, I didn't think of writing statements like that :) That's precisely the sort of suggestion I was hoping for. I'll continue like this.

Can anyone comment on the codegen when using these statements? Is it identical to my reference 'if' statement? Liberal use of loops like this will probably obliterate unoptimised performance... :/

For a moment, I thought the performance crew lost you. :) D's foreach is awesome. Remember this benchmark? http://togototo.wordpress.com/2013/08/23/benchmarks-round-two-parallel-go-rust-d-scala-and-nimrod/ The top entry's performance is due to how well LDC2 optimized the foreach, otherwise the code is quite the same as the C++ version. -- Marco
Jan 15 2014
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
--089e0112cb9e0a746a04f00a2df3
Content-Type: text/plain; charset=UTF-8

On 16 January 2014 05:56, Marco Leise <Marco.Leise gmx.de> wrote:

 For a moment, I thought the performance crew lost you. :)

Haha, don't worry, that'll never happen! ;) I'm just trying to be more open minded in this code I'm writing. I bring a lot of C/C++ baggage to my D code, so I'm trying to write it in the most idiomatic D way I can, and then I'll do a thorough performance analysis later, and see how far south it went (and what the optimisers we able to do in practise)... Experience from C/C++ makes me over-cautious wrt performance these days. Modern optimisers are more reliable than the ones I've spent a decade trying to wrestle to do what I want them to. That said though, I still believe there's value in writing explicitly fast code. If you're quite direct, then it doesn't rely so much on an optimiser, and more importantly, it will continue to run well-ish in non-optimised builds. I'm really wrestling with that; it's not a property I'm comfortable to give up, and all these little templates everywhere that do trivial little things are probably going to interfere with non-optimised performance in a way that exceeds my worst nightmares in C++. D's foreach is awesome. Remember this benchmark?
 http://togototo.wordpress.com/2013/08/23/benchmarks-round-two-parallel-go-rust-d-scala-and-nimrod/
 The top entry's performance is due to how well LDC2 optimized
 the foreach, otherwise the code is quite the same as the C++
 version.

Indeed, and I had just presumed that LDC would properly optimise these little functions we're discussing. Rather surprised to learn it doesn't. Hopefully it's not so hard to improve with some time. --089e0112cb9e0a746a04f00a2df3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On 1= 6 January 2014 05:56, Marco Leise <span dir=3D"ltr">&lt;<a href=3D"mailto:M= arco.Leise gmx.de" target=3D"_blank">Marco.Leise gmx.de</a>&gt;</span> wrot= e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l= eft:1px #ccc solid;padding-left:1ex"> <div class=3D"im"><br> </div>For a moment, I thought the performance crew lost you. :)<br></blockq= uote><div><br></div><div>Haha, don&#39;t worry, that&#39;ll never happen! ;= )</div><div><br></div><div>I&#39;m just trying to be more open minded in th= is code I&#39;m writing.</div> <div>I bring a lot of C/C++ baggage to my D code, so I&#39;m trying to writ= e it in the most idiomatic D way I can, and then I&#39;ll do a thorough per= formance analysis later, and see how far south it went (and what the optimi= sers we able to do in practise)...</div> <div>Experience from C/C++ makes me over-cautious wrt performance these day= s. Modern optimisers are more reliable than the ones I&#39;ve spent a decad= e trying to wrestle to do what I want them to.</div><div><br></div><div> That said though, I still believe there&#39;s value in writing explicitly f= ast code. If you&#39;re quite direct, then it doesn&#39;t rely so much on a= n optimiser, and more importantly, it will continue to run well-ish in non-= optimised builds. I&#39;m really wrestling with that; it&#39;s not a proper= ty I&#39;m comfortable to give up, and all these little templates everywher= e that do trivial little things are probably going to interfere with non-op= timised performance in a way that exceeds my worst nightmares in C++.</div> <div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex= ;border-left:1px #ccc solid;padding-left:1ex"> D&#39;s foreach is awesome. Remember this benchmark?<br> <a href=3D"http://togototo.wordpress.com/2013/08/23/benchmarks-round-two-pa= rallel-go-rust-d-scala-and-nimrod/" target=3D"_blank">http://togototo.wordp= ress.com/2013/08/23/benchmarks-round-two-parallel-go-rust-d-scala-and-nimro= d/</a><br> The top entry&#39;s performance is due to how well LDC2 optimized<br> the foreach, otherwise the code is quite the same as the C++<br> version.<br></blockquote><div><br></div><div>Indeed, and I had just presume= d that LDC would properly optimise these little functions we&#39;re discuss= ing. Rather surprised to learn it doesn&#39;t.</div><div>Hopefully it&#39;s= not so hard to improve with some time.</div> </div></div></div> --089e0112cb9e0a746a04f00a2df3--
Jan 15 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 15 January 2014 at 01:28:26 UTC, Manu wrote:
 You can't step a -O build, or inspect the value of most 
 variables. So you
 can't really debug.

The problem with optimizations is that code mutates a lot. Can you expect the debugger to step through inlined functions? Loop merging can be even more evil in this regard.
Jan 16 2014