www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Expression evaluation order

reply "David Nadlinger" <code klickverbot.at> writes:
There is a test in the DMD testsuite that verifies that the 
evaluation order in the following statement

  a()[] = b()[] + c()[];

is b, c, a.

I'm trying to figure out why this regressed with the 2.063 merge 
in LDC, but (where) is this specified in the first place?

David
Jun 13 2013
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, June 14, 2013 01:09:18 David Nadlinger wrote:
 There is a test in the DMD testsuite that verifies that the
 evaluation order in the following statement
 
 a()[] = b()[] + c()[];
 
 is b, c, a.
 
 I'm trying to figure out why this regressed with the 2.063 merge
 in LDC, but (where) is this specified in the first place?

Walter has expressed a desire in the past to make it so that D requires that the evaluation order of arguments be left-to-right in order to avoid bugs, and while these aren't exactly function arguments, they're basically function arguments to a built-in function called +, so I could definitely see Walter wanting to require that the evaluation order be left-to-right. However, while Walter has expressed an interest in making it a requirement, AFAIK, it has never officially become one and the spec says nothing on the matter. - Jonathan M Davis
Jun 13 2013
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/14/2013 02:13 AM, Jonathan M Davis wrote:
 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd expect
 the same of any operator which is right-associative, ...

Why? Associativity is unrelated to evaluation order.
Jun 14 2013
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/15/2013 04:21 AM, Jonathan M Davis wrote:
 On Saturday, June 15, 2013 01:46:49 Timon Gehr wrote:
 On 06/14/2013 02:13 AM, Jonathan M Davis wrote:
 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd
 expect the same of any operator which is right-associative, ...

Why? Associativity is unrelated to evaluation order.

It seems natural to me that the stuff on the associated side would be evaluated before the stuff on the other, but that doesn't mean that it's the best way to go. It's just what I would have expected. So, if there's a good reason to do it differently, then I don't necessarily have a problem with that. - Jonathan M Davis

Well, I can't think of a _good_ reason now. :) Since Walter once stated that evaluation is supposed to be LTR, this is what I have implemented for CTFE. The stack-based bytecode interpreter already relies on having the address-related information pushed before the value, and changing the evaluation order would therefore imply a little work.
Jun 15 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/15/2013 07:44 PM, Timon Gehr wrote:
 ...
 It seems natural to me that the stuff on the associated side would be
 evaluated
 before the stuff on the other, but that doesn't mean that it's the
 best way to
 go. It's just what I would have expected. So, if there's a good reason
 to do
 it differently, then I don't necessarily have a problem with that.

 - Jonathan M Davis

Well, I can't think of a _good_ reason now. :) ...

Correction: The conditional operator is right-associative, but evaluation necessarily proceeds LTR.
Jun 15 2013
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 6/13/2013 11:14 PM, monarch_dodra wrote:
 3) specifically unspecified
    3.5) error when compiler sees ambiguity

 I'm personally in favor of 3, with some 3.5 as a warning.

Having a warning for that is "boy who cried wolf", as most functions have side effects and any expression like: f() + g() would generate such a warning.
Jun 15 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:

 Walter has expressed a desire in the past to make it so that D 
 requires that the evaluation order of arguments be left-to-right
 in order to avoid bugs,

Considering how much important this feature is, I think it should have priority over several other things. Bye, bearophile
Jun 13 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:
 Walter has expressed a desire in the past to make it so that D 
 requires that
 the evaluation order of arguments be left-to-right in order to 
 avoid bugs, and
 while these aren't exactly function arguments, …

Actually, behind the scenes all three are arguments to *one* function call: the druntime array op implementation. Anyway, my question was mainly about the assignment operator, which is obviously not expected to behave lexically left-to-right by that test case. David
Jun 13 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, June 14, 2013 02:02:23 David Nadlinger wrote:
 On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:
 Walter has expressed a desire in the past to make it so that D
 requires that
 the evaluation order of arguments be left-to-right in order to
 avoid bugs, and
 while these aren't exactly function arguments, …

Actually, behind the scenes all three are arguments to *one* function call: the druntime array op implementation. Anyway, my question was mainly about the assignment operator, which is obviously not expected to behave lexically left-to-right by that test case.

I don't see how the assignment operator could possibly work left-to-right, though I could see how there might be a question with regards to whether what's on the left of the assignment operator would be evaluated before what's on the right when what's on the left is an expression rather than a variable. Personally, I would expect it to fully evaluate the right-hand side of an assignment expression before evaluating anything on the left, and I'd expect the same of any operator which is right-associative, but I'm not aware of anything official on the matter. All the discussions on the matter that I'm aware of have referred specifically to function arguments being left-to-right and said nothing about operators. - Jonathan M Davis
Jun 13 2013
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 14 June 2013 at 00:13:37 UTC, Jonathan M Davis wrote:
 I don't see how the assignment operator could possibly work 
 left-to-right,
 though I could see how there might be a question with regards 
 to whether
 what's on the left of the assignment operator would be 
 evaluated before what's
 on the right when what's on the left is an expression rather 
 than a variable.

SDC compute the address of the reciever, then the value on the right, and finally store it to the address. I don't see any major issue here.
Jun 13 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 June 2013 at 00:13:37 UTC, Jonathan M Davis wrote:
 On Friday, June 14, 2013 02:02:23 David Nadlinger wrote:
 On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis 
 wrote:
 Walter has expressed a desire in the past to make it so that 
 D
 requires that
 the evaluation order of arguments be left-to-right in order 
 to
 avoid bugs, and
 while these aren't exactly function arguments, …

Actually, behind the scenes all three are arguments to *one* function call: the druntime array op implementation. Anyway, my question was mainly about the assignment operator, which is obviously not expected to behave lexically left-to-right by that test case.

I don't see how the assignment operator could possibly work left-to-right

What do you mean? assignment requires two arguments: the *reference* of lhs, and value/reference or rhs. Regardless of if lhs is a value or expression, the compiler still needs to know *where* to assign to. I don't see any reasons why the compiler couldn't choose lhs first instead. *as* a matter of fact, that's how gcc does it for C++: //---- int main() { int i = 0; ++i = ++i; assert(i == 2); //passes } //---- Left to right First, evaluate lhs: i has the value 1. then evaluate rhs, which will now have the value 2. assign 2 to i. * i == 2 * right to left: First, evalutate rhs: i has the value 1, an rhs 1. Evaluate lhs: i is now 2. But then assign rhs (1) to lhs. * i == 1 * So yeah, left to right evaluation is completely coherent, and a valid scheme. Heck, you'll notice this *also* passes with DMD, so dmd *also* evaluates opEqual left to right. So I guess this "surprising behavior" is the evaluation order we want to enforce? --------- I remember the discussion: http://forum.dlang.org/thread/bniaxycuguviwfdtojzf forum.dlang.org For the record, I don't remember there being a consensus... at all... There were 3.5 positions: 1) left to right evaluation 2) right to left evaluation 3) specifically unspecified 3.5) error when compiler sees ambiguity I'm personally in favor of 3, with some 3.5 as a warning. Evaluation order, even when defined, remains obscure once associative operations come into play, and still bite you in the ass anyways, so I don't see anything good out of forcing it one way or the other anyways.
Jun 13 2013
prev sibling next sibling parent reply Iain Buclaw <ibuclaw ubuntu.com> writes:
On 14 June 2013 00:09, David Nadlinger <code klickverbot.at> wrote:
 There is a test in the DMD testsuite that verifies that the evaluation order
 in the following statement

  a()[] = b()[] + c()[];

 is b, c, a.

 I'm trying to figure out why this regressed with the 2.063 merge in LDC, but
 (where) is this specified in the first place?

 David

Interesting, that certainly doesn't flag up as a regression in the 2.063 merge in GDC... :o) However yes, the behaviour relies on the order the sucky X86 ABI pushes arguments onto the stack (which for this array op is from right to left). Whereas on *all* other architectures it will execute the parameters in left to right order, which would be a, c, b in this case. -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 14 2013
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/14/2013 01:04 PM, bearophile wrote:
 ...
 (And I think the right order for that is b, c, a).
 ...

Why?
Jun 14 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Iain Buclaw:

 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is 
 from right
 to left).  Whereas on *all* other architectures it will execute 
 the
 parameters in left to right order, which would be a, c, b in 
 this case.

D needs the same standard order of evaluations for all expressions on all compilers, regardless of the CPUs. The only other acceptable alternative is to statically forbid code that risks having variable results. (And I think the right order for that is b, c, a). Bye, bearophile
Jun 14 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Friday, 14 June 2013 at 11:04:04 UTC, bearophile wrote:
 (And I think the right order for that is b, c, a).

Could you derive a set of rules from/compatible with that thought? ;) David
Jun 14 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
monarch_dodra:

 *as* a matter of fact, that's how gcc does it for C++:

 //----
 int main()
 {
     int i = 0;
     ++i = ++i;
     assert(i == 2); //passes
 }
 //----

 Left to right
 First, evaluate lhs: i has the value 1.
 then evaluate rhs, which will now have the value 2.
 assign 2 to i.
 * i == 2 *

 right to left:
 First, evalutate rhs: i has the value 1, an rhs 1.
 Evaluate lhs: i is now 2.
 But then assign rhs (1) to lhs.
 * i == 1 *

 So yeah, left to right evaluation is completely coherent, and a 
 valid scheme.

I think the results of your code is not defined by the C++ standard, so in practice writing that code is a programmer's mistake. And it's a mistake to design a modern programming language that accepts code that gives undefined results. To be considered a grown up language D needs define only one semantics, or forbid code like that.
 Evaluation order, even when defined, remains obscure once 
 associative operations come into play, and still bite you in 
 the ass anyways,

Even when the programmer isn't able to know what the result will be, it's essential for the D code to give the same result on all CPUs and on all compilers. Bye, bearophile
Jun 14 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On 14 June 2013 12:04, bearophile <bearophileHUGS lycos.com> wrote:
 Iain Buclaw:


 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is from right
 to left).  Whereas on *all* other architectures it will execute the
 parameters in left to right order, which would be a, c, b in this case.

D needs the same standard order of evaluations for all expressions on all compilers, regardless of the CPUs. The only other acceptable alternative is to statically forbid code that risks having variable results. (And I think the right order for that is b, c, a).

Right... and the way to do that would be to evaluate the argument before calling the array op. eg: ref _tmp1 = b()[]; ref _tmp2 = c()[]; ref _tmp3 = a()[]; _tmp3 = _arrayOp(_tmp3, _tmp2, _tmp1); -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 14 2013
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
On Thursday, 13 June 2013 at 23:25:42 UTC, Jonathan M Davis wrote:
 Walter has expressed a desire in the past to make it so that D 
 requires that
 the evaluation order of arguments be left-to-right in order to 
 avoid bugs, and
 while these aren't exactly function arguments, they're 
 basically function
 arguments to a built-in function called +, so I could 
 definitely see Walter
 wanting to require that the evaluation order be left-to-right. 
 However, while
 Walter has expressed an interest in making it a requirement, 
 AFAIK, it has
 never officially become one and the spec says nothing on the 
 matter.

 - Jonathan M Davis

I would like to cast my vote for this feature. It is very nice to have, and it would prevent a few bugs from popping up.
Jun 14 2013
prev sibling next sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 14 Jun 2013 01:09:18 +0200
schrieb "David Nadlinger" <code klickverbot.at>:

 There is a test in the DMD testsuite that verifies that the 
 evaluation order in the following statement
 
   a()[] = b()[] + c()[];
 
 is b, c, a.
 
 I'm trying to figure out why this regressed with the 2.063 merge 
 in LDC, but (where) is this specified in the first place?
 
 David

Related old discussions: http://bugzilla.gdcproject.org/show_bug.cgi?id=8 http://forum.dlang.org/thread/bniaxycuguviwfdtojzf forum.dlang.org (I knew we discussed this before ;-)
Jun 14 2013
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Saturday, June 15, 2013 01:46:49 Timon Gehr wrote:
 On 06/14/2013 02:13 AM, Jonathan M Davis wrote:
 ...
 Personally, I would expect it to fully evaluate the right-hand side of an
 assignment expression before evaluating anything on the left, and I'd
 expect the same of any operator which is right-associative, ...

Why? Associativity is unrelated to evaluation order.

It seems natural to me that the stuff on the associated side would be evaluated before the stuff on the other, but that doesn't mean that it's the best way to go. It's just what I would have expected. So, if there's a good reason to do it differently, then I don't necessarily have a problem with that. - Jonathan M Davis
Jun 14 2013
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Friday, 14 June 2013 at 09:59:07 UTC, Iain Buclaw wrote:
 Interesting, that certainly doesn't flag up as a regression in 
 the
 2.063 merge in GDC... :o)

 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is 
 from right
 to left).  Whereas on *all* other architectures it will execute 
 the
 parameters in left to right order, which would be a, c, b in 
 this
 case.

In LDC, the actual evaluation order is completely separate from the target ABI (you simply pass the SSA vales to a function call in LLVM, and you can decide the order in which to create them yourself). Turns out that for array operations, we actually need to reverse the order compared to our normal one (left to right), as their signatures have specifically been chosen for the evaluation order in DMD that is intertwined with the backend ABI/function call implementation details. David
Jun 15 2013
prev sibling next sibling parent Iain Buclaw <ibuclaw ubuntu.com> writes:
On 15 June 2013 18:16, David Nadlinger <code klickverbot.at> wrote:
 On Friday, 14 June 2013 at 09:59:07 UTC, Iain Buclaw wrote:
 Interesting, that certainly doesn't flag up as a regression in the
 2.063 merge in GDC... :o)

 However yes, the behaviour relies on the order the sucky X86 ABI
 pushes arguments onto the stack (which for this array op is from right
 to left).  Whereas on *all* other architectures it will execute the
 parameters in left to right order, which would be a, c, b in this
 case.

In LDC, the actual evaluation order is completely separate from the target ABI (you simply pass the SSA vales to a function call in LLVM, and you can decide the order in which to create them yourself). Turns out that for array operations, we actually need to reverse the order compared to our normal one (left to right), as their signatures have specifically been chosen for the evaluation order in DMD that is intertwined with the backend ABI/function call implementation details.

We don't have that luxury, but as I said in the last thread brought up on this some time ago, for extern(D) we just make temporaries and compound them all together (_a = a(), _b = b(), _c = c(), _arrayop(_a, _b, _c)); There's no reason why this can't be done for extern (C) - other than it will break a load of code that relies on X86 ABI behaviour (such as what array operations apparently do...) -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Jun 15 2013
prev sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 15 June 2013 at 17:50:39 UTC, Walter Bright wrote:
 On 6/13/2013 11:14 PM, monarch_dodra wrote:
 3) specifically unspecified
   3.5) error when compiler sees ambiguity

 I'm personally in favor of 3, with some 3.5 as a warning.

Having a warning for that is "boy who cried wolf", as most functions have side effects and any expression like: f() + g() would generate such a warning.

Well, why would that throw a warning? I didn't mean expressions that *could* raise an ambiguity, but when the compiler *catches* that there *will* be ambiguity.
Jun 15 2013