www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Always false float comparisons

reply Walter Bright <newshound2 digitalmars.com> writes:
Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

will be true.

So, should the compiler emit a warning for the former case?
May 09 2016
next sibling parent reply qznc <qznc web.de> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
What is the actual reason for the mismatch? Does f lose precision as a float, while the 1.30 literal is a more precise double/real? Comparing float and double might be worth a warning. Does it encode the two literals differently? If so, why?
May 09 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 2:25 AM, qznc wrote:
 What is the actual reason for the mismatch?
floats cannot represent 1.30 exactly, and promoting it to a double gives a different result than 1.30 as a double.
May 09 2016
prev sibling next sibling parent Robert burner Schadek <rburners gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 So, should the compiler emit a warning for the former case?
I'm not for a compiler change. IMO a library called std.sanity_float with a equal and a notequal function would be better.
May 09 2016
prev sibling next sibling parent reply Jens Mueller via Digitalmars-d <digitalmars-d puremagic.com> writes:
Walter Bright via Digitalmars-d wrote:
 Don Clugston pointed out in his DConf 2016 talk that:
 
     float f = 1.30;
     assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a float. However,
 
     float f = 1.30;
     assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?
Since assert(f == 1.30f); passes I find the root cause lies in the implicit type conversion from float to double. Warning for those comparisons should be fine. Shouldn't mix them anyway. I wonder what's the difference between 1.30f and cast(float)1.30. Jens
May 09 2016
next sibling parent Chris <wendlec tcd.ie> writes:
On Monday, 9 May 2016 at 10:16:54 UTC, Jens Mueller wrote:
 Walter Bright via Digitalmars-d wrote:
 Don Clugston pointed out in his DConf 2016 talk that:
 
     float f = 1.30;
     assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a 
 float. However,
 
     float f = 1.30;
     assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?
Since assert(f == 1.30f); passes I find the root cause lies in the implicit type conversion from float to double. Warning for those comparisons should be fine. Shouldn't mix them anyway. I wonder what's the difference between 1.30f and cast(float)1.30. Jens
+1
May 09 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:
 passes I find the root cause lies in the implicit type conversion from
 float to double.
That isn't going to change.
 Warning for those comparisons should be fine. Shouldn't mix them anyway.
Too onerous.
 I wonder what's the difference between 1.30f and cast(float)1.30.
There isn't one.
May 09 2016
next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:
 On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:
 Warning for those comparisons should be fine. Shouldn't mix 
 them anyway.
Too onerous.
Surely not too onerous if we're only talking about == ? Mixing floating point types on either side of == seems like a pretty solidly bad idea.
May 09 2016
parent qznc <qznc web.de> writes:
On Monday, 9 May 2016 at 12:24:05 UTC, John Colvin wrote:
 On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:
 On 5/9/2016 3:16 AM, Jens Mueller via Digitalmars-d wrote:
 Warning for those comparisons should be fine. Shouldn't mix 
 them anyway.
Too onerous.
Surely not too onerous if we're only talking about == ? Mixing floating point types on either side of == seems like a pretty solidly bad idea.
Why only == ? The example also applies to opCmp. float f = 1.30; assert(f >= 1.30); assert(f <= 1.30);
May 09 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/9/16 7:26 AM, Walter Bright wrote:
 I wonder what's the difference between 1.30f and cast(float)1.30.
There isn't one.
I know this is a bit band-aid-ish, but if one is comparing literals to a float, why not treat the literal as the type being compared against? In other words, imply the 1.3f. This isn't integer-land where promotions do not change the outcome. What I see here is that double(1.3) cannot be represented as a float. So right there, the compiler can tell you, no, this is never going to be true. Something stinks when you can write an always-false expression as an if conditional by accident. -Steve
May 09 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 6:46 AM, Steven Schveighoffer wrote:
 I know this is a bit band-aid-ish, but if one is comparing literals to a float,
 why not treat the literal as the type being compared against? In other words,
 imply the 1.3f. This isn't integer-land where promotions do not change the
outcome.
Because it's yet another special case, and we know where those lead. For example, what if the 1.30 was the result of CTFE?
May 09 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/9/16 4:22 PM, Walter Bright wrote:
 On 5/9/2016 6:46 AM, Steven Schveighoffer wrote:
 I know this is a bit band-aid-ish, but if one is comparing literals to
 a float,
 why not treat the literal as the type being compared against? In other
 words,
 imply the 1.3f. This isn't integer-land where promotions do not change
 the outcome.
Because it's yet another special case, and we know where those lead. For example, what if the 1.30 was the result of CTFE?
This is true, it's a contrived example. -Steve
May 09 2016
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 9 May 2016 04:26:55 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 I wonder what's the difference between 1.30f and cast(float)1.30. =20
=20 There isn't one.
=20 Oh yes, there is! Don't you love floating-point... cast(float)1.30 rounds twice, first from a base-10 representation to a base-2 double value and then again to a float. 1.30f directly converts to float. In some cases this does not yield the same value as converting the base-10 literal directly to a float! Imagine this example (with mantissa bit count reduced for illustration): Original base-10 rational number converted to base-2: 111110|10000000|011010111011... =E2=86=96 =E2=86=96 float & double mantissa precision limits The 1st segment is the mantissa width of a float. The 2nd segment is the mantissa width of a double. The 3rd segment is the fraction used for rounding the base-10 literal to a double. Conversion to double rounds down, since fraction <0.5: 111110|10000000 (double mantissa) Proposed cast to float rounds down, too, because of round-to-even rule for 0.5: 111110 (float mantissa) But the conversion of base-10 directly to float, rounds UP since the fraction is then >0.5 111110|10000000_011010111011... 111111 It happens when the 29 bits difference between double and float mantissa are 100=E2=80=A6000 and the bit in front and after are 0. The error would occur to ~0.000000047% of numbers. --=20 Marco
May 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 4:06 PM, Marco Leise wrote:
 Am Mon, 9 May 2016 04:26:55 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:

 I wonder what's the difference between 1.30f and cast(float)1.30.
There isn't one.
Oh yes, there is! Don't you love floating-point... cast(float)1.30 rounds twice, first from a base-10 representation to a base-2 double value and then again to a float. 1.30f directly converts to float.
This is one reason why the compiler carries everything internally to 80 bit precision, even if they are typed as some other precision. It avoids the double rounding.
May 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.05.2016 21:25, Walter Bright wrote:
 On 5/12/2016 4:06 PM, Marco Leise wrote:
 Am Mon, 9 May 2016 04:26:55 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:

 I wonder what's the difference between 1.30f and cast(float)1.30.
There isn't one.
Oh yes, there is! Don't you love floating-point... cast(float)1.30 rounds twice, first from a base-10 representation to a base-2 double value and then again to a float. 1.30f directly converts to float.
This is one reason why the compiler carries everything internally to 80 bit precision, even if they are typed as some other precision. It avoids the double rounding.
IMO the compiler should never be allowed to use a precision different from the one specified.
May 13 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 12:48 PM, Timon Gehr wrote:
 IMO the compiler should never be allowed to use a precision different from the
 one specified.
I take it you've never been bitten by accumulated errors :-) Reduced precision is only useful for storage formats and increasing speed. If a less accurate result is desired, your algorithm is wrong.
May 13 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 13.05.2016 23:35, Walter Bright wrote:
 On 5/13/2016 12:48 PM, Timon Gehr wrote:
 IMO the compiler should never be allowed to use a precision different
 from the one specified.
I take it you've never been bitten by accumulated errors :-) ...
If that was the case it would be because I explicitly ask for high precision if I need it. If the compiler using or not using a higher precision magically fixes an actual issue with accumulated errors, that means the correctness of the code is dependent on something hidden, that you are not aware of, and that could break any time, for example at a time when you really don't have time to track it down.
 Reduced precision is only useful for storage formats and increasing
 speed.  If a less accurate result is desired, your algorithm is wrong.
Nonsense. That might be true for your use cases. Others might actually depend on IEE 754 semantics in non-trivial ways. Higher precision for temporaries does not imply higher accuracy for the overall computation. E.g., correctness of double-double arithmetic is crucially dependent on correct rounding semantics for double: https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic Also, it seems to me that for e.g. https://en.wikipedia.org/wiki/Kahan_summation_algorithm, the result can actually be made less precise by adding casts to higher precision and truncations back to lower precision at appropriate places in the code. And even if higher precision helps, what good is a "precision-boost" that e.g. disappears on 64-bit builds and then creates inconsistent results? Sometimes reproducibility/predictability is more important than maybe making fewer rounding errors sometimes. This includes reproducibility between CTFE and runtime. Just actually comply to the IEEE floating point standard when using their terminology. There are algorithms that are designed for it and that might stop working if the language does not comply. Then maybe add additional built-in types with a given storage size that additionally /guarantee/ a certain amount of additional scratch space when used for function-local computations.
May 13 2016
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.05.2016 02:49, Timon Gehr wrote:
 IEE
IEEE.
May 13 2016
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.05.2016 02:49, Timon Gehr wrote:
 result can actually be made less precise
less accurate. I need to go to sleep.
May 13 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 5:49 PM, Timon Gehr wrote:
 Nonsense. That might be true for your use cases. Others might actually depend
on
 IEE 754 semantics in non-trivial ways. Higher precision for temporaries does
not
 imply higher accuracy for the overall computation.
Of course it implies it. An anecdote: a colleague of mine was once doing a chained calculation. At every step, he rounded to 2 digits of precision after the decimal point, because 2 digits of precision was enough for anybody. I carried out the same calculation to the max precision of the calculator (10 digits). He simply could not understand why his result was off by a factor of 2, which was a couple hundred times his individual roundoff error.
 E.g., correctness of double-double arithmetic is crucially dependent on correct
 rounding semantics for double:
 https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic
Double-double has its own peculiar issues, and is not relevant to this discussion.
 Also, it seems to me that for e.g.
 https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
 the result can actually be made less precise by adding casts to higher
precision
 and truncations back to lower precision at appropriate places in the code.
I don't see any support for your claim there.
 And even if higher precision helps, what good is a "precision-boost" that e.g.
 disappears on 64-bit builds and then creates inconsistent results?
That's why I was thinking of putting in 128 bit floats for the compiler internals.
 Sometimes reproducibility/predictability is more important than maybe making
 fewer rounding errors sometimes. This includes reproducibility between CTFE and
 runtime.
A more accurate answer should never cause your algorithm to fail. It's like putting better parts in your car causing the car to fail.
 Just actually comply to the IEEE floating point standard when using their
 terminology. There are algorithms that are designed for it and that might stop
 working if the language does not comply.
Conjecture. I've written FP algorithms (from Cody+Waite, for example), and none of them degraded when using more precision. Consider that the 8087 has been operating at 80 bits precision by default for 30 years. I've NEVER heard of anyone getting actual bad results from this. They have complained about their test suites that tested for less accurate results broke. They have complained about the speed of x87. And Intel has been trying to get rid of the x87 forever. Sometimes I wonder if there's a disinformation campaign about more accuracy being bad, because it smacks of nonsense. BTW, I once asked Prof Kahan about this. He flat out told me that the only reason to downgrade precision was if storage was tight or you needed it to run faster. I am not making this up.
May 13 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
 BTW, I once asked Prof Kahan about this. He flat out told me 
 that the only reason to downgrade precision was if storage was 
 tight or you needed it to run faster. I am not making this up.
He should have been aware of reproducibility since people use fixed point to achieve it, if he wasn't then shame on him. In Java all compile time constants are done using strict settings and it provides a keyword «strictfp» to get strict behaviour for a particular class/function. In C++ template parameters cannot be floating point, you use std::ratio so you get exact rational number instead. This is to avoid inaccuracy problems in the type system. In interval-arithmetics you need to round up and down correctly on the bounds-computations to get correct results. (It is ok for the interval to be larger than the real result, but the opposite is a disaster). With reproducible arithmetics you can do advanced accurate static analysis of programs using floating point code. With reproducible arithmetics you can sync nodes in a cluster based on "time" alone, saving exchanges of data in simulations. There are lots of reasons to default to well defined floating point arithmetics.
May 13 2016
next sibling parent reply QAston <qaston gmail.com> writes:
On Saturday, 14 May 2016 at 05:46:38 UTC, Ola Fosheim Grøstad 
wrote:
 In Java all compile time constants are done using strict 
 settings and it provides a keyword «strictfp» to get strict 
 behaviour for a particular class/function.
In java everything used to be strictfp (and there was no keyword), but it was changed to do non-strict arithmetic by default after a backlash from numeric programmers.
May 14 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 09:11:49 UTC, QAston wrote:
 On Saturday, 14 May 2016 at 05:46:38 UTC, Ola Fosheim Grøstad 
 wrote:
 In Java all compile time constants are done using strict 
 settings and it provides a keyword «strictfp» to get strict 
 behaviour for a particular class/function.
In java everything used to be strictfp (and there was no keyword), but it was changed to do non-strict arithmetic by default after a backlash from numeric programmers.
Java had a healthy default, but switched in order to not look so bad in comparison to C on current day hardware. However, they retained the ability to get stricter floating point. At the end of the day there is literally no end to how far you can move down the line of implementation defined floating point. Take a look at the ARM instruction set, it makes x86 look high level. You can even choose how many iterations you want for complex instructions (i.e. choose the approximation level for faster execution). However, these days IEEE754-2008 is becoming available in hardware and therefore one is better off choosing the most well-defined semantics for the regular case. It means less optimization opportunities unless you specify relaxed semantics, but that isn't such a bad trade off as long as specifying relaxed semantics is easy.
May 14 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 10:46 PM, Ola Fosheim Grøstad wrote:
 On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
 BTW, I once asked Prof Kahan about this. He flat out told me that the only
 reason to downgrade precision was if storage was tight or you needed it to run
 faster. I am not making this up.
He should have been aware of reproducibility since people use fixed point to achieve it, if he wasn't then shame on him.
Kahan designed the x87 and wrote the IEEE 754 standard, so I'd do my homework before telling him he is wrong about basic floating point stuff.
 In Java all compile time constants are done using strict settings and it
 provides a keyword «strictfp» to get strict behaviour for a particular
 class/function.
What happened with Java was interesting. The original spec required double arithmetic to be done with double precision. This wound up failing all over the place on x86 machines, which (as I explained) does temporaries to 80 bits. Forcing the x87 to use doubles for intermediate values caused Java to run much slower, and Sun was forced to back off on that requirement.
May 14 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 18:58:35 UTC, Walter Bright wrote:
 Kahan designed the x87 and wrote the IEEE 754 standard, so I'd 
 do my homework before telling him he is wrong about basic 
 floating point stuff.
You don't have to tell me who Kahan is. I don't see the relevance. You are trying to appeal to authority. Stick to facts. :-)
May 14 2016
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
 An anecdote: a colleague of mine was once doing a chained 
 calculation. At every step, he rounded to 2 digits of precision 
 after the decimal point, because 2 digits of precision was 
 enough for anybody. I carried out the same calculation to the 
 max precision of the calculator (10 digits). He simply could 
 not understand why his result was off by a factor of 2, which 
 was a couple hundred times his individual roundoff error.
I'm sympathetic to this. Some of my work deals with statistics and you see people try to use formula that are faster but less accurate and it can really get you in to trouble. Var(X) = E(X^2) - E(X)^2 is only true for real numbers, not floating point arithmetic. It can also lead to weird results when dealing with matrix inverses. I like the idea of a float type that is effectively the largest precision on your machine (the D real type). However, I could be convinced by the argument that you should have to opt-in for this and that internal calculations should not implicitly use it. Mainly because I'm sympathetic to the people who would prefer speed to precision. Not everybody needs all the precision all the time.
May 13 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 10:52 PM, jmh530 wrote:
 I like the idea of a float type that is effectively the largest precision on
 your machine (the D real type). However, I could be convinced by the argument
 that you should have to opt-in for this and that internal calculations should
 not implicitly use it. Mainly because I'm sympathetic to the people who would
 prefer speed to precision. Not everybody needs all the precision all the time.
Speed matters on the generated target program, not in the compiler internal floating point calculations, simply because the compiler does very, very few of them.
May 14 2016
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
 Sometimes reproducibility/predictability is more important 
 than maybe making
 fewer rounding errors sometimes. This includes reproducibility 
 between CTFE and
 runtime.
A more accurate answer should never cause your algorithm to fail. It's like putting better parts in your car causing the car to fail.
This is all quite discouraging from a scientific programmers point of view. Precision is important, more precision is good, but reproducibility and predictability are critical. Tables of constants that change value if I put a `static` in front of them? Floating point code that produces different results after a compiler upgrade / with different non-fp-related switches? Ewwwww.
May 14 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/14/2016 3:16 AM, John Colvin wrote:
 This is all quite discouraging from a scientific programmers point of view.
 Precision is important, more precision is good, but reproducibility and
 predictability are critical.
I used to design and build digital electronics out of TTL chips. Over time, TTL chips got faster and faster. The rule was to design the circuit with a minimum signal propagation delay, but never a maximum. Therefore, putting in faster parts will never break the circuit. Engineering is full of things like this. It's sound engineering practice. I've never ever heard of a circuit requiring a resistor with 20% tolerance that would fail if a 10% tolerance one was put in, for another example.
 Tables of constants that change value if I put a `static` in front of them?

 Floating point code that produces different results after a compiler upgrade /
 with different non-fp-related switches?

 Ewwwww.
Floating point is not exact calculation. It just isn't. Designing an algorithm that relies on worse answers is absurd to my ears. Results should be tested to have a minimum number of correct bits in the answer, not a maximum number. This is, in fact, how std.math checks the result of the algorithms implemented in it, and how it should be done. This is not some weird crazy idea of mine, as I said, the x87 FPU in every x86 chip has been doing this for several decades.
May 14 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/14/2016 11:46 AM, Walter Bright wrote:
 I used to design and build digital electronics out of TTL chips. Over time, TTL
 chips got faster and faster. The rule was to design the circuit with a minimum
 signal propagation delay, but never a maximum. Therefore, putting in faster
 parts will never break the circuit.
Eh, I got the min and max backwards.
May 14 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 14 May 2016 at 20:38:54 UTC, Walter Bright wrote:
 On 5/14/2016 11:46 AM, Walter Bright wrote:
 I used to design and build digital electronics out of TTL 
 chips. Over time, TTL
 chips got faster and faster. The rule was to design the 
 circuit with a minimum
 signal propagation delay, but never a maximum. Therefore, 
 putting in faster
 parts will never break the circuit.
Eh, I got the min and max backwards.
Heh, I read that and thought, "wtf is he talking about, never a max?" :D Regarding floating-point, I'll go farther than you and say that if an algorithm depends on lower-precision floating-point to be accurate, it's a bad algorithm. Now, people can always make mistakes in their implementation and unwittingly depend on lower precision somehow, but that _should_ fail. None of this is controversial to me: you shouldn't be comparing floating-point numbers with anything other than approxEqual, increasing precision should never bother your algorithm, and a higher-precision, common soft-float for CTFE will help cross-compiling and you'll never notice the speed hit.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 19:38:10 UTC, Joakim wrote:
 Regarding floating-point, I'll go farther than you and say that 
 if an algorithm depends on lower-precision floating-point to be 
 accurate, it's a bad algorithm.
No. In system level programming a good algorithm is an effective and efficient algorithm on the hardware at hand. A good system level programming language give you control at the hardware level. Including the floating point unit, in the language itself. There are lots of algorithms that will break if you randomly switch precision of different expressions. Heck, nearly all of the SIMD optimizations on differentials that I use will break if one lane is computed with a different precision than the other lanes. I don't give a rats ass about increased precision. I WANT THE DAMN LANES TO BE IN THE SAME PHASE (or close to it)!! Phase-locking is much more important than value accuracy. Or to put it differently: It does not matter if all clocks are too slow, as long as they are running at the same speed. It is a lot worse if some clocks are too slow and others are too fast. That would lead to some serious noise in a time series. Of course, no hardware vendor is crazy enough to randomly switch precision on their ALU. Hardware vendors do understand that this _WILL_ lead disaster. Sadly many in the D community don't. Presumably because they don't actually try to write performant floating point code, they are not system level programmers. (Btw, many error correcting strategies break too if you randomly switch precision.)
 Now, people can always make mistakes in their implementation 
 and unwittingly depend on lower precision somehow, but that 
 _should_ fail.
People WILL make mistakes, but if you cannot control precision then you cannot: 1. create a reference implementation to compare with 2. unit test floating point code in a reliable way 3. test for convergence/divergence in feedback loops (which can have _disastrous_ results and could literally ruin your speakers/hearing in the case of audio).
 None of this is controversial to me: you shouldn't be comparing 
 floating-point numbers with anything other than approxEqual,
I don't agree. 1. Comparing typed constants for equality should be unproblematic. In D that is broken. 2. Testing for zero is a necessity when doing division. 3. Comparing for equality is the same as subtraction followed by testing for zero. So, the rule is: you shouldn't compare at all unless you know the error bounds, but that depends on WHY you are comparing. However, with constants/sentinels and some methods you do know... Also, with some input you do know that the algorithm WILL fail for certain values at a _GIVEN_ precision. Testing for equality for those values makes a lot of sense, until some a**hole decides to randomly "improve" precision where it was typed to something specific and known. Take this: f(x) = 1/(2-x) Should I not be able to test for the exact value "2" here? I don't see why "1.3" typed to a given precision should be different. You want to force me to a more than 3x more expensive test just to satisfy some useless FP semantic that does not provide any real world benefits whatsoever?
 increasing precision should never bother your algorithm, and a 
 higher-precision, common soft-float for CTFE will help 
 cross-compiling and you'll never notice the speed hit.
Randomly increasing precision is never a good idea. Yes, having different precision in different code paths can ruin the quality of both rendering, data analysis and break algorithms. Having infinite precision untyped real that may downgrade to say 64 bits mantissa is acceptable. Or in the case of Go, a 256 bit mantissa. That's a different story. Having single precision floats that randomly are turned into arbitrary precision floats is not acceptable. Not at all.
May 17 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 17 May 2016 at 14:59:45 UTC, Ola Fosheim Grøstad 
wrote:
 There are lots of algorithms that will break if you randomly 
 switch precision of different expressions.
There is nothing "random" about increasing precision till the end, it follows a well-defined rule.
 Heck, nearly all of the SIMD optimizations on differentials 
 that I use will break if one lane is computed with a different 
 precision than the other lanes. I don't give a rats ass about 
 increased precision. I WANT THE DAMN LANES TO BE IN THE SAME 
 PHASE (or close to it)!! Phase-locking is much more important 
 than value accuracy.
So you're planning on running phase-locking code partially in CTFE and runtime and it's somehow very sensitive to precision? If your "phase-locking" depends on producing bit-exact results with floating-point, you're doing it wrong.
 Now, people can always make mistakes in their implementation 
 and unwittingly depend on lower precision somehow, but that 
 _should_ fail.
People WILL make mistakes, but if you cannot control precision then you cannot: 1. create a reference implementation to compare with 2. unit test floating point code in a reliable way 3. test for convergence/divergence in feedback loops (which can have _disastrous_ results and could literally ruin your speakers/hearing in the case of audio).
If any of this depends on comparing bit-exact floating-point results, you're doing it wrong.
 None of this is controversial to me: you shouldn't be 
 comparing floating-point numbers with anything other than 
 approxEqual,
I don't agree. 1. Comparing typed constants for equality should be unproblematic. In D that is broken.
If the constant is calculated rather than a literal, you should be checking using approxEqual.
 2. Testing for zero is a necessity when doing division.
If the variable being tested is calculated, you should be using approxEqual.
 3. Comparing for equality is the same as subtraction followed 
 by testing for zero.
So what? You should be using approxEqual.
 So, the rule is: you shouldn't compare at all unless you know 
 the error bounds, but that depends on WHY you are comparing.
No, you should always use error bounds. Sometimes you can get away with checking bit-exact equality, say for constants that you defined yourself with no calculation, but it's never a good practice.
 However, with constants/sentinels and some methods you do 
 know... Also, with some input you do know that the algorithm 
 WILL fail for certain values at a _GIVEN_ precision. Testing 
 for equality for those values makes a lot of sense, until some 
 a**hole decides to randomly "improve" precision where it was 
 typed to something specific and known.

 Take this:

 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?
It would make more sense to figure out what the max value of f(x) is you're trying to avoid, say 1e6, and then check for approxEqual(x, 2, 2e-6). That would make much more sense than only avoiding 2, when an x that is arbitrarily close to 2 can also blow up f(x).
 I don't see why "1.3" typed to a given precision should be 
 different. You want to force me to a more than 3x more 
 expensive test just to satisfy some useless FP semantic that 
 does not provide any real world benefits whatsoever?
Oh, it's real world alright, you should be avoiding more than just 2 in your example above.
 increasing precision should never bother your algorithm, and a 
 higher-precision, common soft-float for CTFE will help 
 cross-compiling and you'll never notice the speed hit.
Randomly increasing precision is never a good idea. Yes, having different precision in different code paths can ruin the quality of both rendering, data analysis and break algorithms. Having infinite precision untyped real that may downgrade to say 64 bits mantissa is acceptable. Or in the case of Go, a 256 bit mantissa. That's a different story. Having single precision floats that randomly are turned into arbitrary precision floats is not acceptable. Not at all.
Simply repeating the word "random" over and over again does not make it so.
May 17 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.
Can you please quote that well-defined rule? It is indeed random, or arbitrary (which is the same thing): if(x<0){ // DMD choose 64 bit mantissa const float y = ... ... } else { // DMD choose 24 bit mantissa float y = ... ... } How is this not arbitrary? If x is the amplitude then a flaw like this can cause a DC offset to accumulate and you end up with reduced precision, not improved precision. A DC filter at the end does not help on this precision loss.
 So you're planning on running phase-locking code partially in 
 CTFE and runtime and it's somehow very sensitive to precision?  
 If your "phase-locking" depends on producing bit-exact results 
 with floating-point, you're doing it wrong.
I am not doing anything wrong. D is doing it wrong. If you add different deltas then you will get drift. So, no improved precision in calculating some deltas is not improving the accuracy. It makes it worse.
 If any of this depends on comparing bit-exact floating-point 
 results, you're doing it wrong.
It depends on the unit tests running with the exact same precision as the production code. Fast floating point code depends on the specifics of the hardware. A system level language should not introduce a different kind of bias that isn't present in the hardware! D is doing it wrong because it makes it is thereby forcing programmers to use algorithms that are 10-100x slower to get reliable results. That is _wrong_.
 If the constant is calculated rather than a literal, you should 
 be checking using approxEqual.
No. 1+2+3+4 is exact on all floating point units I know of.
 3. Comparing for equality is the same as subtraction followed 
 by testing for zero.
So what? You should be using approxEqual.
No.
 So, the rule is: you shouldn't compare at all unless you know 
 the error bounds, but that depends on WHY you are comparing.
No, you should always use error bounds. Sometimes you can get away with checking bit-exact equality, say for constants that you defined yourself with no calculation, but it's never a good practice.
It is if you know WHY you are doing a test. Btw, have you ever tried to prove error bounds for an iterative method? You actually think most people prove them to be correct? Or perhaps almost all of them just pick a number out of thin air which they think will work out and rely on testing their code? Well, the latter is no better than checking for exact equality. And I can assure you that the vast majority of programmers do not prove error bounds with the level of rigour it takes to get it correct. The reality is that it is common practice to write code that seems to work. But that does not make it correct. However, making it correct is way too time consuming and often not worth the trouble. So people rely on testing. Floating point code is no different. But with D semantics you cannot rely on testing. That's bad, because most people write incorrect code. Whether they are experts or not. (it is only matter of difference in frequency)
 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?
It would make more sense to figure out what the max value of f(x) is you're trying to avoid, say 1e6, and then check for approxEqual(x, 2, 2e-6). That would make much more sense than only avoiding 2, when an x that is arbitrarily close to 2 can also blow up f(x).
I am trying to avoid an exception, not a value.
 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.
Which number would that be?
 Simply repeating the word "random" over and over again does not 
 make it so.
That's right. It is DMD that makes it so, not my words. However, in order to reject what other say, you have to make an argument. And in this case we have: 1. A system level programming language that claims to excel at floating point. 2. Hardware platforms with specific behaviour. Unfortunately 2 is true, but not 1. D is not matching up to the minimum requirements for people wanting to write fast and reliable floating point code for a given platform. According to the D spec, the compiler could schedule typed single precision floating point calculations to two completely different floating point units (and yes, there are platforms that provide multiple incompatible floating point units with widely differing characteristics). That is random. And so is "float" behaving differently than "const float".
May 17 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.
Can you please quote that well-defined rule?
It appears to be "the compiler carries everything internally to 80 bit precision, even if they are typed as some other precision." http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com
 It is indeed random, or arbitrary (which is the same thing):
No, they're not the same thing: rules can be arbitrarily set yet consistent over time, whereas random usually means both arbitrary and inconsistent over time.
 if(x<0){
   // DMD choose 64 bit mantissa
   const float y = ...
   ...

 } else {
   // DMD choose 24 bit mantissa
   float y = ...
   ...
 }

 How is this not arbitrary?
I believe that means any calculation used to compute y at compile-time will be done in 80-bit or larger reals, then rounded to a const float for run-time, so your code comments would be wrong.
 If x is the amplitude then a flaw like this can cause a DC 
 offset to accumulate and you end up with reduced precision, not 
 improved precision. A DC filter at the end does not help on 
 this precision loss.
I don't understand why you're using const for one block and not the other, seems like a contrived example. If the precision of such constants matters so much, I'd be careful to use the same const float everywhere.
 So you're planning on running phase-locking code partially in 
 CTFE and runtime and it's somehow very sensitive to precision?
  If your "phase-locking" depends on producing bit-exact 
 results with floating-point, you're doing it wrong.
I am not doing anything wrong. D is doing it wrong. If you add different deltas then you will get drift. So, no improved precision in calculating some deltas is not improving the accuracy. It makes it worse.
If matching such small deltas matters so much, I wouldn't be using floating-point in the first place.
 If any of this depends on comparing bit-exact floating-point 
 results, you're doing it wrong.
It depends on the unit tests running with the exact same precision as the production code.
What makes you think they don't?
 Fast floating point code depends on the specifics of the 
 hardware. A system level language should not introduce a 
 different kind of bias that isn't present in the hardware!
He's doing this to take advantage of the hardware, not the opposite!
 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.
If programmers want to run their code 10-100x slower to get reliably inaccurate results, that is their problem.
 If the constant is calculated rather than a literal, you 
 should be checking using approxEqual.
No. 1+2+3+4 is exact on all floating point units I know of.
If you're so convinced it's exact for a few cases, then check exact equality there. For most calculation, you should be using approxEqual.
 Btw, have you ever tried to prove error bounds for an iterative 
 method?
 You actually think most people prove them to be correct?

 Or perhaps almost all of them just pick a number out of thin 
 air which they think will work out and rely on testing their 
 code?
No, I have never done so, and I'm well aware that most just pull the error bound they use out of thin air.
 Well, the latter is no better than checking for exact equality. 
 And I can assure you that the vast majority of programmers do 
 not prove error bounds with the level of rigour it takes to get 
 it correct.
Even an unproven error bound is better than "exact" equality, which technically is really assuming that the error bound is smaller than the highest precision you can specify the number. Since the real error bound is always larger than that, almost any error bound you pick will tend to be closer to the real error bound, or at least usually bigger and therefore more realistic, than checking for exact equality.
 The reality is that it is common practice to write code that 
 seems to work. But that does not make it correct. However, 
 making it correct is way too time consuming and often not worth 
 the trouble. So people rely on testing. Floating point code is 
 no different.

 But with D semantics you cannot rely on testing. That's bad, 
 because most people write incorrect code. Whether they are 
 experts or not. (it is only matter of difference in frequency)
You can still test with approxEqual, so I don't understand why you think that's not testing. std.math uses feqrel, approxEqual, and equalsDigit and other such lower-precision checks extensively in its tests.
 f(x) = 1/(2-x)

 Should I not be able to test for the exact value "2" here?
It would make more sense to figure out what the max value of f(x) is you're trying to avoid, say 1e6, and then check for approxEqual(x, 2, 2e-6). That would make much more sense than only avoiding 2, when an x that is arbitrarily close to 2 can also blow up f(x).
I am trying to avoid an exception, not a value.
That is the problem. In the real world, all such formulas are approximations that only apply over a certain range. If you were cranking that out by hand and some other calculation gave you an x really close to 2, say 2.0000035, you'd go back and check your math, as f(x) would blow up and give you unrealistically large numbers for the rest of your calculations. The computer doesn't know that, so it will just plug that x in and keep cranking, till you get nonsense data out the end, if you don't tell it to check that x isn't too close to 2 and not just 2. You have a wrong mental model that the math formulas are the "real world," and that the computer is mucking it up. The truth is that the computer, with its finite maximums and bounded precision, better models _the measurements we make to estimate the real world_ than any math ever written.
 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.
Which number would that be?
I told you, any numbers too close to 2.
 Simply repeating the word "random" over and over again does 
 not make it so.
That's right. It is DMD that makes it so, not my words. However, in order to reject what other say, you have to make an argument. And in this case we have: 1. A system level programming language that claims to excel at floating point. 2. Hardware platforms with specific behaviour. Unfortunately 2 is true, but not 1. D is not matching up to the minimum requirements for people wanting to write fast and reliable floating point code for a given platform.
On the contrary, it is done because 80-bit is faster and more precise, whereas your notion of reliable depends on an incorrect notion that repeated bit-exact results are better.
 According to the D spec, the compiler could schedule typed 
 single precision floating point calculations to two completely 
 different floating point units (and yes, there are platforms 
 that provide multiple incompatible floating point units with 
 widely differing characteristics).
You noted that you don't care that the C++ spec says similar things, so I don't see why you care so much about the D spec now. As for that scenario, nobody has suggested it.
 That is random.
It may be arbitrary, but it is not random unless it's inconsistently done.
 And so is "float" behaving differently than "const float".
I don't believe it does.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:
 On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
 wrote:
 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
 There is nothing "random" about increasing precision till the 
 end, it follows a well-defined rule.
Can you please quote that well-defined rule?
It appears to be "the compiler carries everything internally to 80 bit precision, even if they are typed as some other precision." http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com
"The compiler" means: implementation defined. That is the same as not being well-defined. :-)
 It is indeed random, or arbitrary (which is the same thing):
No, they're not the same thing: rules can be arbitrarily set yet consistent over time, whereas random usually means both arbitrary and inconsistent over time.
In this case it is the same thing then. I have no guarantee that my unit tests and production code will behave the same.
 I believe that means any calculation used to compute y at 
 compile-time will be done in 80-bit or larger reals, then 
 rounded to a const float for run-time, so your code comments 
 would be wrong.
No. The "const float y" will not be coerced to 32 bit, but the "float y" will be coerced to 32 bit. So you get two different y values. (On a specific compiler, i.e. DMD.)
 I don't understand why you're using const for one block and not 
 the other, seems like a contrived example.  If the precision of 
 such constants matters so much, I'd be careful to use the same 
 const float everywhere.
Now, that is a contrived defense for brittle language semantics! :-)
 If matching such small deltas matters so much, I wouldn't be 
 using floating-point in the first place.
Why not? The hardware gives the same delta. It only goes wrong if the compiler decides to "improve".
 It depends on the unit tests running with the exact same 
 precision as the production code.
What makes you think they don't?
Because the language says that I cannot rely on it and the compiler implementation proves that to be correct.
 Fast floating point code depends on the specifics of the 
 hardware. A system level language should not introduce a 
 different kind of bias that isn't present in the hardware!
He's doing this to take advantage of the hardware, not the opposite!
I don't understand what you mean. He is not taking advantage of the hardware?
 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.
If programmers want to run their code 10-100x slower to get reliably inaccurate results, that is their problem.
Huh? What I said is that D is doing it wrong because the "improvements" is forcing me to write code that is 10-100x slower to get the same level of reliability and required accuracy as I would get without the "improvements".
 If you're so convinced it's exact for a few cases, then check 
 exact equality there.  For most calculation, you should be 
 using approxEqual.
I am sorry, but this is not a normative rule at all. The rule is that you check for the bounds required. If it is exact, it just means the bounds are the same value (e.g. tight). It does not help to say that people should use "approxEqual", because it does not improve on correctness. Saying such things just means that non-expert programmers assume that guessing the bounds will be sufficient. Well, it isn't sufficient.
 Since the real error bound is always larger than that, almost 
 any error bound you pick will tend to be closer to the real 
 error bound, or at least usually bigger and therefore more 
 realistic, than checking for exact equality.
I disagree. It is much better to get extremely wrong results frequently and therefore detect the error in testing. What you are saying is that is better to get extremely wrong results infrequently which usually leads to error passing testing and enter production. In order to test well you also need to understand for input makes the algorithm unstable/fragile.
 You can still test with approxEqual, so I don't understand why 
 you think that's not testing.
It is not testing anything if the compiler can change the semantics when you use a different context.
 The computer doesn't know that, so it will just plug that x in 
 and keep cranking, till you get nonsense data out the end, if 
 you don't tell it to check that x isn't too close to 2 and not 
 just 2.
Huh? I am not getting nonsense data. I am getting what I am asking for, I only want to avoid dividing by zero because it will make the given hardware 100x slower than the test.
 You have a wrong mental model that the math formulas are the 
 "real world," and that the computer is mucking it up.
Nothing wrong with my mental model. My mental model is the hardware specification + the specifics of the programming platform. That is the _only_ model that matters. What D prevents me from getting is the specifics of the programming platform by making the specifics hidden.
 The truth is that the computer, with its finite maximums and 
 bounded precision, better models _the measurements we make to 
 estimate the real world_ than any math ever written.
I am not estimating anything. I am synthesising artificial worlds. My code is the model, the world is my code running at specific hardware. It is self contained. I don't want the compiler to change my model because that will generate the wrong world. ;-)
 Oh, it's real world alright, you should be avoiding more than 
 just 2 in your example above.
Which number would that be?
I told you, any numbers too close to 2.
All numbers close to 2 in the same precision will work out ok.
 On the contrary, it is done because 80-bit is faster and more 
 precise, whereas your notion of reliable depends on an 
 incorrect notion that repeated bit-exact results are better.
80 bit is much slower. 80 bit mul takes 3 micro ops, 64 bit takes 1. Without SIMD 64 bit is at least twice as fast. With SIMD multiply-add is maybe 10x faster in 64bit. And it is neither more precise or more accurate when you don't get consistent precision. In the real world you can get very good performance for the desired accuracy by using unstable algorithms by adding a stage that compensate for the instability. That does not mean that it is acceptable to have differences in the bias as that can lead to accumulating an offset that brings the result away from zero (thus a loss of precision).
 You noted that you don't care that the C++ spec says similar 
 things, so I don't see why you care so much about the D spec 
 now.
  As for that scenario, nobody has suggested it.
I care about what the C++ spec. I care about how the platform interprets the spec. I never rely on _ONLY_ the C++ spec for production code. You have said previously that you know the ARM platform. On Apple CPUs you have 3 different floating point units: 32 bit NEON, 64 bit NEON and 64 bit IEEE. It supports 1x64bit IEEE, 2x64bit NEON and 4x32 bit NEON. You have to know the language, the compiler and the hardware to make this work out.
 And so is "float" behaving differently than "const float".
I don't believe it does.
I have proven that it does, and posted it in this thread.
May 18 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:
 No. The "const float y" will not be coerced to 32 bit, but the 
 "float y" will be coerced to 32 bit. So you get two different y 
 values. (On a specific compiler, i.e. DMD.)
I'm not sure that the `const float` vs `float` is the difference per se. The difference is that in the examples you've given, the `const float` is being determined (and used) at compile time. But a `const float` won't _always_ be determined or used at compile time, depending on the context and manner in which the value is set. Let's be clear about the problem -- compile time vs. runtime, rather than `const` vs non-`const`.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 11:12:16 UTC, Joseph Rushton 
Wakeling wrote:
 I'm not sure that the `const float` vs `float` is the 
 difference per se.  The difference is that in the examples 
 you've given, the `const float` is being determined (and used) 
 at compile time.
They both have to be determined at compile time... as there is only a cast involved. The real issue is that the const float binding is treated textually.
 But a `const float` won't _always_ be determined or used at 
 compile time, depending on the context and manner in which the 
 value is set.
Which makes the problem worse, not better.
May 18 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 09:21:30 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 18 May 2016 at 07:21:30 UTC, Joakim wrote:
 On Wednesday, 18 May 2016 at 05:49:16 UTC, Ola Fosheim Grøstad 
 wrote:
 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
 There is nothing "random" about increasing precision till 
 the end, it follows a well-defined rule.
Can you please quote that well-defined rule?
It appears to be "the compiler carries everything internally to 80 bit precision, even if they are typed as some other precision." http://forum.dlang.org/post/nh59nt$1097$1 digitalmars.com
"The compiler" means: implementation defined. That is the same as not being well-defined. :-)
Welcome to the wonderful world of C++! :D More seriously, it is well-defined for that implementation, you did not raise the issue of the spec till now. In fact, you seemed not to care what the specs say.
 I don't understand why you're using const for one block and 
 not the other, seems like a contrived example.  If the 
 precision of such constants matters so much, I'd be careful to 
 use the same const float everywhere.
Now, that is a contrived defense for brittle language semantics! :-)
No, it has nothing to do with language semantics and everything to do with bad numerical programming.
 If matching such small deltas matters so much, I wouldn't be 
 using floating-point in the first place.
Why not? The hardware gives the same delta. It only goes wrong if the compiler decides to "improve".
Because floating-point is itself fuzzy, in so many different ways. You are depending on exactly repeatable results with a numerical type that wasn't meant for it.
 It depends on the unit tests running with the exact same 
 precision as the production code.
What makes you think they don't?
Because the language says that I cannot rely on it and the compiler implementation proves that to be correct.
You keep saying this: where did anyone mention unit tests not running with the same precision till you just brought it up out of nowhere? The only prior mention was that compile-time calculation of constants that are then checked for bit-exact equality in the tests might have problems, but that's certainly not all tests and I've repeatedly pointed out you should never be checking for bit-exact equality.
 D is doing it wrong because it makes it is thereby forcing 
 programmers to use algorithms that are 10-100x slower to get 
 reliable results.

 That is _wrong_.
If programmers want to run their code 10-100x slower to get reliably inaccurate results, that is their problem.
Huh?
The point is that what you consider reliable will be less accurate, sometimes much less.
 If you're so convinced it's exact for a few cases, then check 
 exact equality there.  For most calculation, you should be 
 using approxEqual.
I am sorry, but this is not a normative rule at all. The rule is that you check for the bounds required. If it is exact, it just means the bounds are the same value (e.g. tight). It does not help to say that people should use "approxEqual", because it does not improve on correctness. Saying such things just means that non-expert programmers assume that guessing the bounds will be sufficient. Well, it isn't sufficient.
The point is that there are _always_ bounds, so you can never check for the same value. Almost any guessed bounds will be better than incorrectly checking for the bit-exact value.
 Since the real error bound is always larger than that, almost 
 any error bound you pick will tend to be closer to the real 
 error bound, or at least usually bigger and therefore more 
 realistic, than checking for exact equality.
I disagree. It is much better to get extremely wrong results frequently and therefore detect the error in testing. What you are saying is that is better to get extremely wrong results infrequently which usually leads to error passing testing and enter production. In order to test well you also need to understand for input makes the algorithm unstable/fragile.
Nobody is talking about the general principle of how often you get wrong results or unit testing. We were talking about a very specific situation: how should compile-time constants be checked and variables compared to constants, compile-time or not, to avoid exceptional situations. My point is that both should always be thought about. In the latter case, ie your f(x) example, it has nothing to do with error bounds, but that your f(x) is not only invalid at 2, but in a range around 2. Now, both will lead to less "wrong results," but those are wrong results you _should_ be trying to avoid as early as possible.
 The computer doesn't know that, so it will just plug that x in 
 and keep cranking, till you get nonsense data out the end, if 
 you don't tell it to check that x isn't too close to 2 and not 
 just 2.
Huh? I am not getting nonsense data. I am getting what I am asking for, I only want to avoid dividing by zero because it will make the given hardware 100x slower than the test.
Zero is not the only number that screws up that calculation.
 You have a wrong mental model that the math formulas are the 
 "real world," and that the computer is mucking it up.
Nothing wrong with my mental model. My mental model is the hardware specification + the specifics of the programming platform. That is the _only_ model that matters. What D prevents me from getting is the specifics of the programming platform by making the specifics hidden.
Your mental model determines what you think is valid input to f(x) and what isn't, that has nothing to do with D. You want D to provide you a way to only check for 0.0, whereas my point is that there are many numbers in the neighborhood of 0.0 which will screw up your calculation, so really you should be using approxEqual.
 The truth is that the computer, with its finite maximums and 
 bounded precision, better models _the measurements we make to 
 estimate the real world_ than any math ever written.
I am not estimating anything. I am synthesising artificial worlds. My code is the model, the world is my code running at specific hardware. It is self contained. I don't want the compiler to change my model because that will generate the wrong world. ;-)
It isn't changing your model, you can always use a very small threshold in approxEqual. Yes, a few more values would be disallowed as input and output than if you were to compare exactly to 0.0, but your model is almost certainly undefined there too. If your point is that you're modeling artificial worlds that have nothing to do with reality, you can always change your threshold around 0.0 to be much smaller, and who cares if it can't go all the way to zero, it's all artificial, right? :) If you're modeling the real world, any function that blows up and gives you bad data, blows up over a range, never a single point, because that's how measurement works.
 Oh, it's real world alright, you should be avoiding more 
 than just 2 in your example above.
Which number would that be?
I told you, any numbers too close to 2.
All numbers close to 2 in the same precision will work out ok.
They will give you large numbers that can be represented in the computer, but do not work out to describe the real world, because such formulas are really invalid in a neighborhood of 2, not just at 2.
 On the contrary, it is done because 80-bit is faster and more 
 precise, whereas your notion of reliable depends on an 
 incorrect notion that repeated bit-exact results are better.
80 bit is much slower. 80 bit mul takes 3 micro ops, 64 bit takes 1. Without SIMD 64 bit is at least twice as fast. With SIMD multiply-add is maybe 10x faster in 64bit.
I have not measured this speed myself so I can't say.
 And it is neither more precise or more accurate when you don't 
 get consistent precision.

 In the real world you can get very good performance for the 
 desired accuracy by using unstable algorithms by adding a stage 
 that compensate for the instability. That does not mean that it 
 is acceptable to have differences in the bias as that can lead 
 to accumulating an offset that brings the result away from zero 
 (thus a loss of precision).
A lot of hand-waving about how more precision is worse, with no real example, which is what Walter keeps asking for.
 You noted that you don't care that the C++ spec says similar 
 things, so I don't see why you care so much about the D spec 
 now.
  As for that scenario, nobody has suggested it.
I care about what the C++ spec. I care about how the platform interprets the spec. I never rely on _ONLY_ the C++ spec for production code.
Then you must be perfectly comfortable with a D spec that says similar things. ;)
 You have said previously that you know the ARM platform. On 
 Apple CPUs you have 3 different floating point units: 32 bit 
 NEON, 64 bit NEON and 64 bit IEEE.

 It supports 1x64bit IEEE, 2x64bit NEON and 4x32 bit NEON.

 You have to know the language, the compiler and the hardware to 
 make this work out.
Sure, but nobody has suggested interchanging the three randomly.
 And so is "float" behaving differently than "const float".
I don't believe it does.
I have proven that it does, and posted it in this thread.
I don't think that example has much to do with what we're talking about. It appears to be some sort of constant folding in the assert that produces the different results, as Joe says, which goes away if you use approxEqual. If you look at the actual const float initially, it is very much a float, contrary to your assertions.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:
 Welcome to the wonderful world of C++! :D

 More seriously, it is well-defined for that implementation, you 
 did not raise the issue of the spec till now.  In fact, you 
 seemed not to care what the specs say.
Eh? All C/C++ compilers I have ever used coerce to single precision upon binding. I care about what the spec says, why it says it and how it is used when targetting the specific hardware. Or more generally, when writing libraries I target the language spec, when writing applications I target the platform (language + compiler + hardware).
 No, it has nothing to do with language semantics and everything 
 to do with bad numerical programming.
No. You can rant all you want about bad numerical programming. But by your definition all DSP programmers are bad at numerics. Which _obviously_ is not true. That is grasping for straws. There is NO excuse for preventing programmers from writing reliable performant code that solves the problem at hand to the accuracy that is required by the application.
 Because floating-point is itself fuzzy, in so many different 
 ways.  You are depending on exactly repeatable results with a 
 numerical type that wasn't meant for it.
Floating point is not fuzzy. It is basically a random sample on an interval of potential solutions with a range that increase with each computation. Unfortunately, you have to use interval arithmetics to get that interval, as in regular floating point you loose the information about the interval. It is an approximation. IEEE 754-2008 makes it possible to get that interval, btw. Fuzzy is different. Fuzzy means that the range itself is the value. It does not represent an approximation. ;-)
 You keep saying this: where did anyone mention unit tests not 
 running with the same precision till you just brought it up out 
 of nowhere?  The only prior mention was that compile-time 
 calculation of constants that are then checked for bit-exact 
 equality in the tests might have problems, but that's certainly 
 not all tests and I've repeatedly pointed out you should never 
 be checking for bit-exact equality.
I have stated time and time again that it is completely unacceptable if there is even a remote chance for anything in the unit test to evaluate at a higher precision than in the production code. That is not a unit test, it is a sham.
 The point is that what you consider reliable will be less 
 accurate, sometimes much less.
Define accurate. Accurate has no meaning without a desired outcome to reference. I care about 4 oscillators having the same phase. THAT MEANS: they all have to use the exact same constants. If not, they _will_ drift and phase cancellation _will_ occur. I care about adding and removing the same constant. If not, (more) DC offsets will build up. It is all about getting below the right tolerance threshold while staying real time. You can compensate by increasing the control rate (reduce the number of interpolated values).
 The point is that there are _always_ bounds, so you can never 
 check for the same value.  Almost any guessed bounds will be 
 better than incorrectly checking for the bit-exact value.
No. Guessed bounds are not better. I have never said that anyone should _incorrectly_ do anything. I have said that they should _correctly_ understand what they are doing and that guessing bounds just leads to a worse problem: Programs that intermittently fail in spectacular ways that are very difficult to debug. You cannot even compute the bounds if the compiler can use higher precision. IEEE754-2008 makes it possible to accurately compute bounds. Not supporting that is _very_ bad.
 should always be thought about.  In the latter case, ie your 
 f(x) example, it has nothing to do with error bounds, but that 
 your f(x) is not only invalid at 2, but in a range around 2.
It isn't. For what typed number besides 2 is it invalid?
 Zero is not the only number that screws up that calculation.
It is. Not only can it screw up the calculation. It can screw up the real time properties of the algorithm on a specific FPU. Which is even worse.
 f(x) and what isn't, that has nothing to do with D.  You want D 
 to provide you a way to only check for 0.0, whereas my point is 
 that there are many numbers in the neighborhood of 0.0 which 
 will screw up your calculation, so really you should be using 
 approxEqual.
What IEEE32 number besides 2 can cause problems?
 It isn't changing your model, you can always use a very small
But it is!
 If your point is that you're modeling artificial worlds that 
 have nothing to do with reality, you can always change your 
 threshold around 0.0 to be much smaller, and who cares if it 
 can't go all the way to zero, it's all artificial, right? :)
Why would I have a threshold around 2, when only 2 is causing problems at the hardware level?
 If you're modeling the real world, any function that blows up 
 and gives you bad data, blows up over a range, never a single 
 point, because that's how measurement works.
If I am analyzing real world data I _absolutely_ want all code paths to use the specified precision. I absolutely don't want to wonder whether some computations have a different pattern than others because of the compiler. Now, providing 64, 128 or 256 bits mantissa and software emulation is _perfectly_ sound. Computing 10 and 24 bits mantissas as if they are 256 bits without the programmer specifying it is bad. And yes, half-precision is only 10 bits.
 They will give you large numbers that can be represented in the 
 computer, but do not work out to describe the real world, 
 because such formulas are really invalid in a neighborhood of 
 2, not just at 2.
I don't understand what you mean. Even Inf as an outcome tells me something. Or in the case of simulation Inf might be completely valid input for the next stage.
 I have not measured this speed myself so I can't say.
www.agner.org It is of course even worse for 32 bit floats. Then we are at 10x-20x faster than 80bit.
 A lot of hand-waving about how more precision is worse, with no 
 real example, which is what Walter keeps asking for.
I have provided plenty of examples. You guys just don't want to listen. Because it is against the sects religious beliefs that D falls short of expectations both on integers and floating point. This keeps D at the hobby level as a language. It is a very deep-rooted cultural problem. But that is ok. Just don't complain about people saying that D is not fit for production. Because they have several good reasons to say that.
May 18 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:
 And yes, half-precision is only 10 bits.
Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)
May 18 2016
parent reply Matthias Bentrup <matthias.bentrup googlemail.com> writes:
On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
 wrote:
 And yes, half-precision is only 10 bits.
Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)
The mantissa is 10 bits, but it has 11 bit precision, just as the float type has a 23 bit mantissa and 24 bit precision. AFAIK the only float format that stores the always-one-bit of the mantissa is the x87 80 bit format.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 15:30:42 UTC, Matthias Bentrup wrote:
 On Wednesday, 18 May 2016 at 14:29:42 UTC, Ola Fosheim Grøstad 
 wrote:
 On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
 wrote:
 And yes, half-precision is only 10 bits.
Actually, it turns out that the mantissa is 11 bits. So it clearly plays louder than other floats. ;-)
The mantissa is 10 bits, but it has 11 bit precision, just as the float type has a 23 bit mantissa and 24 bit precision. AFAIK the only float format that stores the always-one-bit of the mantissa is the x87 80 bit format.
It turns out both 10bits and 11bits are "right" and that one should qualify it with «with/without the hidden bit», except that it shouldn't be called the «mantissa», but the «significand»: https://en.wikipedia.org/wiki/Significand Thanks, always fun with learning ;-)
May 18 2016
prev sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 12:27:38 UTC, Ola Fosheim Grøstad 
wrote:
 On Wednesday, 18 May 2016 at 11:16:44 UTC, Joakim wrote:
 Welcome to the wonderful world of C++! :D

 More seriously, it is well-defined for that implementation, 
 you did not raise the issue of the spec till now.  In fact, 
 you seemed not to care what the specs say.
Eh? All C/C++ compilers I have ever used coerce to single precision upon binding. I care about what the spec says, why it says it and how it is used when targetting the specific hardware. Or more generally, when writing libraries I target the language spec, when writing applications I target the platform (language + compiler + hardware).
I see, so the fact that both the C++ and D specs say the same thing doesn't matter, and the fact that D also has the const float in your example as single-precision at runtime, contrary to your claims, none of that matters.
 No, it has nothing to do with language semantics and 
 everything to do with bad numerical programming.
No. You can rant all you want about bad numerical programming. But by your definition all DSP programmers are bad at numerics. Which _obviously_ is not true. That is grasping for straws. There is NO excuse for preventing programmers from writing reliable performant code that solves the problem at hand to the accuracy that is required by the application.
No sane DSP programmer would write that like you did.
 Because floating-point is itself fuzzy, in so many different 
 ways.  You are depending on exactly repeatable results with a 
 numerical type that wasn't meant for it.
Floating point is not fuzzy. It is basically a random sample on an interval of potential solutions with a range that increase with each computation. Unfortunately, you have to use interval arithmetics to get that interval, as in regular floating point you loose the information about the interval. It is an approximation. IEEE 754-2008 makes it possible to get that interval, btw.
Or in the vernacular, fuzzy. :) Of course, this is true of all non-integer calculation.
 Fuzzy is different. Fuzzy means that the range itself is the 
 value. It does not represent an approximation. ;-)
It's all approximations.
 You keep saying this: where did anyone mention unit tests not 
 running with the same precision till you just brought it up 
 out of nowhere?  The only prior mention was that compile-time 
 calculation of constants that are then checked for bit-exact 
 equality in the tests might have problems, but that's 
 certainly not all tests and I've repeatedly pointed out you 
 should never be checking for bit-exact equality.
I have stated time and time again that it is completely unacceptable if there is even a remote chance for anything in the unit test to evaluate at a higher precision than in the production code. That is not a unit test, it is a sham.
Since the vast majority of tests will never use such compile-test constants, your opinion is not only wrong but irrelevant.
 The point is that what you consider reliable will be less 
 accurate, sometimes much less.
Define accurate. Accurate has no meaning without a desired outcome to reference. I care about 4 oscillators having the same phase. THAT MEANS: they all have to use the exact same constants. If not, they _will_ drift and phase cancellation _will_ occur. I care about adding and removing the same constant. If not, (more) DC offsets will build up. It is all about getting below the right tolerance threshold while staying real time. You can compensate by increasing the control rate (reduce the number of interpolated values).
Then don't use differently defined constants in different places and don't use floating point, simple.
 The point is that there are _always_ bounds, so you can never 
 check for the same value.  Almost any guessed bounds will be 
 better than incorrectly checking for the bit-exact value.
No. Guessed bounds are not better. I have never said that anyone should _incorrectly_ do anything. I have said that they should _correctly_ understand what they are doing and that guessing bounds just leads to a worse problem: Programs that intermittently fail in spectacular ways that are very difficult to debug. You cannot even compute the bounds if the compiler can use higher precision. IEEE754-2008 makes it possible to accurately compute bounds. Not supporting that is _very_ bad.
If you're comparing bit-exact equality, you're not using _any_ error bounds: that's the worst possible choice.
 should always be thought about.  In the latter case, ie your 
 f(x) example, it has nothing to do with error bounds, but that 
 your f(x) is not only invalid at 2, but in a range around 2.
It isn't. For what typed number besides 2 is it invalid?
 Zero is not the only number that screws up that calculation.
It is. Not only can it screw up the calculation. It can screw up the real time properties of the algorithm on a specific FPU. Which is even worse.
 f(x) and what isn't, that has nothing to do with D.  You want 
 D to provide you a way to only check for 0.0, whereas my point 
 is that there are many numbers in the neighborhood of 0.0 
 which will screw up your calculation, so really you should be 
 using approxEqual.
What IEEE32 number besides 2 can cause problems?
Sigh, any formula f(x) like the one you present is an approximation for the real world. The fact that it blows up at 2 means they couldn't measure any values there, so they just stuck that singularity in the formula. That means it's not merely undefined at 2, but that they couldn't take measurements _around_ 2, which is why any number close to 2 will give you unreasonably large output. Simply blindly plugging that formula in and claiming that it's only invalid at 2 and nowhere else means you don't really understand the math, nor the science that underlies it.
 It isn't changing your model, you can always use a very small
But it is!
Not in any meaningful way.
 If your point is that you're modeling artificial worlds that 
 have nothing to do with reality, you can always change your 
 threshold around 0.0 to be much smaller, and who cares if it 
 can't go all the way to zero, it's all artificial, right? :)
Why would I have a threshold around 2, when only 2 is causing problems at the hardware level?
Because it's not only the hardware that matters, the validity of your formula in the neighborhood of 2 also matters.
 If you're modeling the real world, any function that blows up 
 and gives you bad data, blows up over a range, never a single 
 point, because that's how measurement works.
If I am analyzing real world data I _absolutely_ want all code paths to use the specified precision. I absolutely don't want to wonder whether some computations have a different pattern than others because of the compiler. Now, providing 64, 128 or 256 bits mantissa and software emulation is _perfectly_ sound. Computing 10 and 24 bits mantissas as if they are 256 bits without the programmer specifying it is bad. And yes, half-precision is only 10 bits.
None of this has anything to do with the point you responded to.
 They will give you large numbers that can be represented in 
 the computer, but do not work out to describe the real world, 
 because such formulas are really invalid in a neighborhood of 
 2, not just at 2.
I don't understand what you mean. Even Inf as an outcome tells me something. Or in the case of simulation Inf might be completely valid input for the next stage.
Then why are you checking for it? This entire example was predicated on your desire to avoid Inf at precisely 2. My point is that it's not only Inf that has to be avoided, other output can be finite and still worthless, because the formula doesn't really apply there. So you should really be using approxEqual, as anywhere else with floating point math, and your example is wrong.
 A lot of hand-waving about how more precision is worse, with 
 no real example, which is what Walter keeps asking for.
I have provided plenty of examples.
No, you only provided one, the one about const float, that had little relevance. The rest was a lot of hand-waving about vague scenarios, with no actual example.
 You guys just don't want to listen. Because it is against the 
 sects religious beliefs that D falls short of expectations both 
 on integers and floating point.

 This keeps D at the hobby level as a language. It is a very 
 deep-rooted cultural problem.

 But that is ok. Just don't complain about people saying that D 
 is not fit for production. Because they have several good 
 reasons to say that.
Funny, considering this entire thread has had many making countervailing claims about the choices Walter has made for D.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 15:42:56 UTC, Joakim wrote:
 I see, so the fact that both the C++ and D specs say the same 
 thing doesn't matter, and the fact that D also has the const 
 float in your example as single-precision at runtime, contrary 
 to your claims, none of that matters.
D doesn't even have a spec, so how can they possibly say the same thing? However, quoting the Wikipedia page on IEEE floats: «The IEEE 754-1985 allowed many variations in implementations (such as the encoding of some values and the detection of certain exceptions). IEEE 754-2008 has strengthened up many of these, but a few variations still remain (especially for binary formats). The reproducibility clause recommends that language standards should provide a means to write reproducible programs (i.e., programs that will produce the same result in all implementations of a language), and describes what needs to be done to achieve reproducible results.» That's the map people who care about floating point follow.
 No sane DSP programmer would write that like you did.
What kind of insult is that? I've read lots of DSP code written by others. I know what kind of programming it entails. In fact, what I have described here are techniques picked up from state-of-the-art DSP code written by top-the-of-line DSP programmers.
 Since the vast majority of tests will never use such 
 compile-test constants, your opinion is not only wrong but 
 irrelevant.
Oh... Not only am I wrong, but my opinion is irrelevant. Well, with this attitude D will remain irrelevant as well. For good reasons.
 Then don't use differently defined constants in different places
I don't, and I didn't. DMD did it.
May 18 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 18 May 2016 at 07:49, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 18 May 2016 at 03:01:14 UTC, Joakim wrote:
 There is nothing "random" about increasing precision till the end, it
 follows a well-defined rule.
Can you please quote that well-defined rule? It is indeed random, or arbitrary (which is the same thing): if(x<0){ // DMD choose 64 bit mantissa const float y = ... ... } else { // DMD choose 24 bit mantissa float y = ... ... } How is this not arbitrary?
Can you back that up statistically? Try running this same operation 600 million times plot a graph for the result from each run for it so we can get an idea of just how random or arbitrary it really is. If you get the same result back each time, maybe it isn't as arbitrary or random as you would have some believe.
May 18 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:
 Can you back that up statistically?  Try running this same 
 operation 600 million times plot a graph for the result from 
 each run for it so we can get an idea of just how random or 
 arbitrary it really is.
Huh? This isn't about statistics. It is about math. The magnitude of the difference depends on what you do with the constant. It can be exponentially boosted.
May 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 3:46 AM, Ola Fosheim Grøstad wrote:
 On Wednesday, 18 May 2016 at 09:13:35 UTC, Iain Buclaw wrote:
 Can you back that up statistically?  Try running this same operation 600
 million times plot a graph for the result from each run for it so we can get
 an idea of just how random or arbitrary it really is.
Huh? This isn't about statistics.
It is when you say 'random'. D's fp math is not random, it is completely deterministic, and I've told you so before. You've been pushing the 'random' notion here, and other people have picked it up and inferred they'd get random results every time they ran a D compiled program. Please use correct meanings of words.
May 18 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 14 May 2016 at 18:46:50 UTC, Walter Bright wrote:
 On 5/14/2016 3:16 AM, John Colvin wrote:
 This is all quite discouraging from a scientific programmers 
 point of view.
 Precision is important, more precision is good, but 
 reproducibility and
 predictability are critical.
I used to design and build digital electronics out of TTL chips. Over time, TTL chips got faster and faster. The rule was to design the circuit with a minimum signal propagation delay, but never a maximum. Therefore, putting in faster parts will never break the circuit. Engineering is full of things like this. It's sound engineering practice. I've never ever heard of a circuit requiring a resistor with 20% tolerance that would fail if a 10% tolerance one was put in, for another example.
Should scientific software be written to not break if the floating-point precision is enhanced, and to allow greater precision to be used when the hardware supports it? Sure. However, that's not the same as saying that the choice of precision should be in the hands of the hardware, rather than the person building + running the program. I for one would not like to have to spend time working out why my program was producing different results, just because I (say) switched from a machine supporting maximum 80-bit float to one supporting 128-bit.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 6:49 AM, Joseph Rushton Wakeling wrote:
 However, that's not the same as saying that the choice of precision should be
in
 the hands of the hardware, rather than the person building + running the
 program.
 I for one would not like to have to spend time working out why my
 program was producing different results, just because I (say) switched from a
 machine supporting maximum 80-bit float to one supporting 128-bit.
If you wrote it "to not break if the floating-point precision is enhanced, and to allow greater precision to be used when the hardware supports it" then what's the problem? Can you provide an example of a legitimate algorithm that produces degraded results if the precision is increased?
May 15 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 15 May 2016 at 18:30:20 UTC, Walter Bright wrote:
 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?
I've just been observing this argument (over and over again) and I think the two sides are talking about different things. Walter, you keep saying degraded results. The scientific folks keep saying *consistent* results. Think about a key goal in scientific experiments: to measure changes across repeated experiments, to reproduce and confirm or falsify results. They want to keep as much equal as they can. I suppose people figure if they use the same compiler, same build options, same source code and feed the same data into it, they expect to get the *same* results. It is a deterministic machine, right? You might argue they should add "same hardware" to that list, but apparently it isn't that easy, or I doubt people would be talking about this.
May 15 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:41:57 UTC, Adam D. Ruppe wrote:
 I suppose people figure if they use the same compiler, same 
 build options, same source code and feed the same data into it, 
 they expect to get the *same* results. It is a deterministic 
 machine, right?
In this case it is worse, you get this: float f = some_pure_function(); const float cf = some_pure_function(); assert(cf==f); // SHUTDOWN!!! But Walter doesn't think this is an issue, because correctness is just a high school feature.
May 15 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:59:51 UTC, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 18:41:57 UTC, Adam D. Ruppe wrote:
 I suppose people figure if they use the same compiler, same 
 build options, same source code and feed the same data into 
 it, they expect to get the *same* results. It is a 
 deterministic machine, right?
In this case it is worse, you get this: float f = some_pure_function(); const float cf = some_pure_function(); assert(cf==f); // SHUTDOWN!!!
That was too quick, but you get the idea: assert((cf==N) == (f==N)); //SHUTDOWN!!
May 15 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:41 AM, Adam D. Ruppe wrote:
 Walter, you keep saying degraded results.

 The scientific folks keep saying *consistent* results.
Programs will not produce inconsistent results. You run the same program, you get the same results. Scientists know that all measurements contain error. The idea is to identify what the maximum error is. I've never heard of a scientific experiment that required a minimum error.
 You might argue they should add "same hardware" to that list, but apparently it
 isn't that easy, or I doubt people would be talking about this.
You're arguing for floating point portability, which is something else again. Java tried that (discussed earlier) and it was a near complete failure.
May 15 2016
prev sibling next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 15 May 2016 at 18:30:20 UTC, Walter Bright wrote:
 If you wrote it "to not break if the floating-point precision 
 is enhanced, and to allow greater precision to be used when the 
 hardware supports it" then what's the problem?

 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?
No, but I think that's not really relevant to the point I was making. Results don't have to be degraded to be inconsistent.
May 15 2016
prev sibling parent reply poliklosio <poliklosio happypizza.com> writes:
 Can you provide an example of a legitimate algorithm that 
 produces degraded results if the precision is increased?
The real problem here is the butterfly effect (the chaos theory thing). Imagine programming a multiplayer game. Ideally you only need to synchronize user events, like key presses etc. Other computation can be duplicated on all machines participating in a session. Now imagine that some logic other than display (e.g. player-bullet collision detection) is using floating point. If those computations are not reproducible, a higher precision on one player's machine can lead to huge inconsistencies in game states between the machines (e.g. my character is dead on your machine but alive on mine)! If the game developer cannot achieve reproducibility or it takes too much work, the workarounds can be wery costly. He can, for example, convert implementation to soft float or increase amount of synchronization over the network. Also I think Adam is making a very good point about generl reproducibility here. If a researcher gets a little bit different results, he has to investigate why, because he needs to rule out all the serious mistakes that could be the cause of the difference. If he finds out that the source was an innocuous refactoring of some D code, he will be rightly frustrated that D has caused so much unnecessary churn. I think the same problem can occur in mission-critical software which undergoes strict certification.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 1:49 PM, poliklosio wrote:
 Can you provide an example of a legitimate algorithm that produces degraded
 results if the precision is increased?
The real problem here is the butterfly effect (the chaos theory thing). Imagine programming a multiplayer game. Ideally you only need to synchronize user events, like key presses etc. Other computation can be duplicated on all machines participating in a session. Now imagine that some logic other than display (e.g. player-bullet collision detection) is using floating point. If those computations are not reproducible, a higher precision on one player's machine can lead to huge inconsistencies in game states between the machines (e.g. my character is dead on your machine but alive on mine)! If the game developer cannot achieve reproducibility or it takes too much work, the workarounds can be wery costly. He can, for example, convert implementation to soft float or increase amount of synchronization over the network.
If you use the same program binary, you *will* get the same results.
 Also I think Adam is making a very good point about generl reproducibility
here.
 If a researcher gets a little bit different results, he has to investigate why,
 because he needs to rule out all the serious mistakes that could be the cause
of
 the difference. If he finds out that the source was an innocuous refactoring of
 some D code, he will be rightly frustrated that D has caused so much
unnecessary
 churn.

 I think the same problem can occur in mission-critical software which undergoes
 strict certification.
Frankly, I think you are setting unreasonable expectations. Today, if you take a standard compliant C program, and compile it with different switch settings, or run it on a machine with a different CPU, you can very well get different answers. If you reorganize the code expressions, you can very well get different answers.
May 15 2016
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 08:05, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 1:49 PM, poliklosio wrote:

 Also I think Adam is making a very good point about generl reproducibility
 here.
 If a researcher gets a little bit different results, he has to investigate
 why,
 because he needs to rule out all the serious mistakes that could be the
 cause of
 the difference. If he finds out that the source was an innocuous
 refactoring of
 some D code, he will be rightly frustrated that D has caused so much
 unnecessary
 churn.

 I think the same problem can occur in mission-critical software which
 undergoes
 strict certification.
Frankly, I think you are setting unreasonable expectations. Today, if you take a standard compliant C program, and compile it with different switch settings, or run it on a machine with a different CPU, you can very well get different answers. If you reorganize the code expressions, you can very well get different answers.
The argument you used against me earlier was that it was unacceptable for some C code pasted in D to behave differently than in C... but here you've just destroyed your own argument by noting that C behaves differently than itself. The rest of us would never get away with this ;) For reference:
 I think it's the only reasonable solution.
 It may be, but it is unusual and therefore surprising behavior.
 What is the problem with this behaviour I suggest?
 Code will do one thing in C, and the same code will do something unexpectedly
different in D.
So let's reopen the discussion from my first post that you dismissed? If the situation is that C compilers produce no predictable behaviour (as you claim above, and I agree), what is the harm to applying a behaviour that actually works?
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 6:59 PM, Manu via Digitalmars-d wrote:
 The argument you used against me earlier was that it was unacceptable
 for some C code pasted in D to behave differently than in C... but
 here you've just destroyed your own argument by noting that C behaves
 differently than itself.
D nails down some C behavior that is specified as "implementation defined". This is not being incompatible.
May 15 2016
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 14 May 2016 at 01:26:18 UTC, Walter Bright wrote:
 On 5/13/2016 5:49 PM, Timon Gehr wrote:
 And even if higher precision helps, what good is a 
 "precision-boost" that e.g.
 disappears on 64-bit builds and then creates inconsistent 
 results?
That's why I was thinking of putting in 128 bit floats for the compiler internals.
I almost wonder if the cent/ucent will be avaliable soon as types rather than reserved keywords. Anyways my two cents. I remember having issues doing a direct compare on an identical value when it was immutable which proved false, and this made no sense and still doesn't. Regardless I've tried to exhaustively do the compares and found some unexpected results. float f = 1.30; const float cf = 1.30; immutable If = 1.30; writeln(f == 1.30); //false writeln(cf == 1.30); writeln(If == 1.30); writeln(f == 1.30f); writeln(cf == 1.30f); writeln(If == 1.30f); writeln(f == cf); writeln(f == If); //false writeln(If == cf); The real reason it's false is because you're comparing against an immutable (constant), or my results lead me to believe that; Otherwise why could cf and If work?
May 14 2016
next sibling parent reply Binarydepth <binarydepth gmail.com> writes:
On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:
     writeln(f == 1.30f);
     writeln(cf == 1.30f);
     writeln(If == 1.30f);
I guess this returned True ?
May 14 2016
parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Saturday, 14 May 2016 at 20:54:44 UTC, Binarydepth wrote:
 On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:
     writeln(f == 1.30f);
     writeln(cf == 1.30f);
     writeln(If == 1.30f);
I guess this returned True ?
I only noted the ones that returned false, since those are the interesting cases.
May 14 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:
  The real reason it's false is because you're comparing against 
 an immutable (constant), or my results lead me to believe that; 
 Otherwise why could cf and If work?
The "immutable" is of type real... Unfortunately the "cf" is also computing as real whereas "f" is implicitly converted to real from float. It makes comparing floating point types for equality implementation defined in D, i.e. unreliable. The assignment to "cf" ought to have brought cf to 32 bit float. So the "false" results are the correct ones, and the "true" are the unexpected ones where you compare different types. import numpy f=numpy.float32(1.30) r=numpy.float64(1.30) print r==f # false
May 14 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 06:49:19 UTC, Ola Fosheim Grøstad wrote:
 On Saturday, 14 May 2016 at 20:41:12 UTC, Era Scarecrow wrote:
  The real reason it's false is because you're comparing 
 against an immutable (constant), or my results lead me to 
 believe that; Otherwise why could cf and If work?
The "immutable" is of type real... Unfortunately the "cf" is also computing as real whereas "f" is implicitly converted to real from float. It makes comparing floating point types for equality implementation defined in D, I.e. unreliable.
But the const and immutable still have to fit within the confines of the size provided, so the fact they were calculated as real doesn't matter when they are stored as floats. If all the results are inverted, I don't see how that works... Although it does make sense that a float is being promoted up, and the lost precision vs the real is the problem... to which point it should issue a warning for different types. Does this mean that instead for compares with a float/constant should be demoted down a level if it can? Although I'm sure using the f will force it to be correct. But then why does f==If fail? It doesn't answer that critical question, since they should be identical.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 07:10:49 UTC, Era Scarecrow wrote:
  But the const and immutable still have to fit within the 
 confines of the size provided, so the fact they were calculated 
 as real doesn't matter when they are stored as floats. If all 
 the results are inverted, I don't see how that works...
The problem is that they aren't stored... It is implementation defined... What is really bad about this is that you have: writeln(f==cf); // true writeln(f==1.30); // false writeln(cf==1.30); // true hence the following fallacy: ((f==cf) && (cf==1.30)) implies (f != 1.30) That is _very_ bad.
 Although it does make sense that a float is being promoted up, 
 and the lost precision vs the real is the problem...
The problem is that D "optimizes" out casts to floats so that "cast(real)cast(float)somereal" is replaced with "somereal": writeln(1.30==cast(real)cast(float)1.30); // true
 But then why does f==If fail? It doesn't answer that critical 
 question, since they should be identical.
Because D does not cancel out "cast(real)cast(float)1.30)" for "f", but does cancel it out for cf. Or basically: it uses textual substitution (macro expansion/inlining) for "cf", but not for "f". ?
May 15 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:
 That is _very_ bad.
Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) in floating point math? FP math is simply not the math you learned in high school. You'll have nothing but grief with it if you insist it must be the same.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 18:35:15 UTC, Walter Bright wrote:
 On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:
 That is _very_ bad.
Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) in floating point math? FP math is simply not the math you learned in high school. You'll have nothing but grief with it if you insist it must be the same.
Err... these kind of problems only applies to D. Please leave your high school experiences out of it. :-)
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:45 AM, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 18:35:15 UTC, Walter Bright wrote:
 On 5/15/2016 1:33 AM, Ola Fosheim Grøstad wrote:
 That is _very_ bad.
Oh, rubbish. Did you know that (x+y)+z is not equal to x+(y+z) in floating point math? FP math is simply not the math you learned in high school. You'll have nothing but grief with it if you insist it must be the same.
Err... these kind of problems only applies to D.
Nope. They occur with every floating point implementation in every programming language. FP math does not adhere to associative identities. http://www.walkingrandomly.com/?p=5380 http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html Ironically, the identity is more likely to hold with D's extended precision for intermediate values than with other languages.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 21:01:14 UTC, Walter Bright wrote:
 Err... these kind of problems only applies to D.
Nope. They occur with every floating point implementation in every programming language. FP math does not adhere to associative identities.
No. ONLY D give different results for the same pure function call because you bind the result to a "const float" rather than a "float". Yes. Algorithms can break because of it.
 Ironically, the identity is more likely to hold with D's 
 extended precision for intermediate values than with other 
 languages.
No, it is not more likely to hold with D's hazard game. I don't know of a single language that doesn't heed a request to truncate/round the mantissa if it provides the means to do it. I care about algorithms working they way I designed them to work and what I have tested them for. If I request rounding to a 24 bit mantissa then I _expect_ the rounding to take place. And yes, it can break algorithms if you don't.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 2:36 PM, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 21:01:14 UTC, Walter Bright wrote:
 Err... these kind of problems only applies to D.
Nope. They occur with every floating point implementation in every programming language. FP math does not adhere to associative identities.
No. ONLY D give different results for the same pure function call because you bind the result to a "const float" rather than a "float".
It's a fact that with floating point arithmetic, (x+y)+z is not equal to x+(y+x), on every programming language, whether or not the result is bound to a const float or a float.
 Yes. Algorithms can break because of it.
So far, nobody has posted a legitimate one (i.e. not contrived).
 Ironically, the identity is more likely to hold with D's extended precision
 for intermediate values than with other languages.
No, it is not more likely to hold with D's hazard game. I don't know of a single language that doesn't heed a request to truncate/round the mantissa if it provides the means to do it.
Standard C provides no such hooks for constant folding at compile time. Neither does Standard C++. In fact I know of no language that does. Perhaps you can provide a link?
 I care about algorithms working they way I designed them to work and what I
have
 tested them for. If I request rounding to a 24 bit mantissa then I _expect_ the
 rounding to take place.
Example, please, of how you 'request' rounding/truncation.
 And yes, it can break algorithms if you don't.
Example, please. ---- Something you should know from the C++ Standard: "The values of the floating operands and the results of floating expressions may be represented in greater precision and range than that required by the type; the types are not changed thereby" -- 5.0.11 D is clearly making FP arithmetic *more* predictable, not less. Since you care about FP results, I seriously suggest studying http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html and what the C++ Standard actually says.
May 15 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 22:34:24 UTC, Walter Bright wrote:
 So far, nobody has posted a legitimate one (i.e. not contrived).
Oh, so comparing the exact same calculation, using the exact same binary executable function, is not a legitimate algorithm. It is the most trivial thing to do and rather common. It should be _very_ convincing. But hey, here is another one: const real x = f(); assert(0<=x && x<1); x += 1; const float f32 = cast(float)(x); const real residue = x - cast(real)f32; // ZERO!!!! output(dither(f32, residue)); // DITHERING IS FUBAR!!!
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:14 PM, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 22:34:24 UTC, Walter Bright wrote:
 So far, nobody has posted a legitimate one (i.e. not contrived).
Oh, so comparing the exact same calculation, using the exact same binary executable function, is not a legitimate algorithm.
What algorithm relies on such? And as numerous links posted here show, it doesn't work reliably on just about any C/C++ compiler, either. If you're writing C++ code that relies on such, you're in for some tragic late night debugging sessions as Manu relates.
 It is the most trivial
 thing to do and rather common. It should be _very_ convincing.
Post a link to some algorithm that does that.
 But hey, here is another one:

 const real x = f();
 assert(0<=x && x<1);
 x += 1;

 const float f32 = cast(float)(x);
 const real residue = x - cast(real)f32; // ZERO!!!!

 output(dither(f32, residue)); // DITHERING IS FUBAR!!!
Why would anyone dither based on the difference in precision of floats and doubles? Dithering is usually done based on different pixel densities. I'm sure you can (and did) write code contrived to show a difference. I asked for something different, though.
May 16 2016
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 16 May 2016 00:52:41 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 5/15/2016 11:14 PM, Ola Fosheim Gr=C3=B8stad wrote:
 But hey, here is another one:

 const real x =3D f();
 assert(0<=3Dx && x<1);
 x +=3D 1;

 const float f32 =3D cast(float)(x);
 const real residue =3D x - cast(real)f32; // ZERO!!!!

 output(dither(f32, residue)); // DITHERING IS FUBAR!!! =20
=20 Why would anyone dither based on the difference in precision of floats an=
d=20
 doubles? Dithering is usually done based on different pixel densities.
To be fair, several image editors have various dithering algorithms that seek to reduce the apparent banding when converting an image from a higher to a lower bit depth. 3dfx also performed dithering in hardware when going from internal 24-bit RGB calculations to 16-bit output. Ola's example could be some X-ray imaging format. Although it is still a vague example, this "cast doesn't do anything to floating point constants" semantic surprised me too, I have to say. I don't want to dive too deep into the matter though. --=20 Marco
May 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 09:00:49 UTC, Marco Leise wrote:
 Ola's example could be some X-ray imaging format. Although
 it is still a vague example,
I don't think quantization is a particularly vague example! I think that is rather common. Doing something like floor(x*(1<<24))*(1.0/(1<<24)) to quantize to 24 bit is just silly and inefficient. Error diffusion is useful in scenarios where you want to reduce the accumulation-error or avoid banding. It can be used in audio in many other settings. Anyway, irregular higher precision is almost always worse than regular lower precision when computing time series, comparing results or doing unit testing.
May 16 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 00:34, Walter Bright wrote:
 Yes. Algorithms can break because of it.
So far, nobody has posted a legitimate one (i.e. not contrived).
My examples were not contrived. Manu's even less so. What you call "illegitimate" are really just legitimate examples that you dismiss because they do not match your own specific experience. I think that's a little disrespectful.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 1:43 AM, Timon Gehr wrote:
 My examples were not contrived.
I don't see where you posted examples in this thread. You did link to two wikipedia articles, which seemed to be more conjecture than actual examples.
 Manu's even less so.
Manu has said he's had trouble with it. I believe him. But I haven't seen his code to see just what he was doing.
 What you call "illegitimate" are really just legitimate examples that you
 dismiss because they do not match your own specific experience.
Of course, legitimate is a matter of opinion. Can code be written to rely on lower precision? Of course. Is it portable to any standard conforming C/C++ compiler? Nope. Can algorithms be coded to not require reduced precision? I'm pretty sure they can be, and should be. The links I've posted here make it clear that there is no "obvious" right way for compilers to implement FP. If you're going to write sensitive FP algorithms, it only makes sense to take the arbitrariness of FP implementations into account. My proposal for D means less is implementation defined than C/C++, the semantics fall within the "implementation defined" part of the C/C++ standards, and the implementation is completely IEEE compliant. I don't believe it is a service to programmers to have the panopoly of FP behavior switches sported by typical compilers that apparently very few people understand. Worse, writing code that is invisibly, subtly, and critically dependent on specific compiler switches and versions is not something I can recommend as a best practice.
 I think that's a little disrespectful.
Perhaps, but disrespecting code is not an ad homenim argument.
May 16 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 09:56:07 UTC, Walter Bright wrote:
 What you call "illegitimate" are really just legitimate 
 examples that you
 dismiss because they do not match your own specific experience.
Of course, legitimate is a matter of opinion. Can code be written to rely on lower precision? Of course. Is it portable to any standard conforming C/C++ compiler? Nope. Can algorithms be coded to not require reduced precision? I'm pretty sure they can be, and should be.
If I've understood people's arguments right, the point of concern is that there are use cases where the programmer wants to be able to guarantee _a specific precision of their choice_. That strikes me as a legitimate use-case that it would be worth trying to support.
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:26 AM, Joseph Rushton Wakeling wrote:
 If I've understood people's arguments right, the point of concern is that there
 are use cases where the programmer wants to be able to guarantee _a specific
 precision of their choice_.

 That strikes me as a legitimate use-case that it would be worth trying to
support.
Yes, and I've proposed roundToFloat() and roundToDouble() intrinsics in this thread and the last two or three times this came up. I also strongly feel that use of those intrinsics is a red flag that there's a problem with the algorithm. At least the use of them will make it obvious in the code that there's a bug :-) in the algorithm. Floats are chosen for speed and storage. Insisting that lower accuracy is desired as a default consequence just baffles me. It's like saying people choose sugary snacks because they want dental cavities.
May 16 2016
prev sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 08:33:44 UTC, Ola Fosheim Grøstad wrote:
 Because D does not cancel out "cast(real)cast(float)1.30)" for 
 "f", but does cancel it out for cf.
Hmmm... I noticed my code example fails to specify float for the immutable, fixing that only the first line with f == 1.30 fail while all others succeed (for some reason?), which is good news. Does that mean the error that I had noted a few years ago actually was fixed?
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 20:34:19 UTC, Era Scarecrow wrote:
  I noticed my code example fails to specify float for the 
 immutable, fixing that only the first line with f == 1.30 fail 
 while all others succeed (for some reason?), which is good news.
Well, it isn't actually good news. Both "const float" and "immutable float" should fail as you have requested coercion to 32 bit floats which ought to change the value of "1.30", but unfortunately D does not heed the coercion. The net result is that adding const/immutable to a type can change the semantics of the program entirely at the whim of the compiler implementor. In comparison C++: float f = 1.30; const float c = 1.30; constexpr float i = 1.30; std::cout << (f==1.30) << std::endl; // false std::cout << (c==1.30) << std::endl; // false std::cout << (i==1.30) << std::endl; // false std::cout << (1.30==(float)1.30) << std::endl; // false
May 15 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 21:06:22 UTC, Ola Fosheim Grøstad wrote:
   std::cout << (f==1.30) << std::endl;  // false
   std::cout << (c==1.30) << std::endl;  // false
   std::cout << (i==1.30) << std::endl;  // false
   std::cout << (1.30==(float)1.30) << std::endl;  // false
If we want equality then we should compare to the representation for a 32 bit float: std::cout << (f == 1.2999999523162841796875) << std::endl; // true
May 15 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can change the
semantics
 of the program entirely at the whim of the compiler implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard. My proposal removes the "whim" by requiring 128 bit precision for CTFE.
May 15 2016
next sibling parent Seb <seb wilzba.ch> writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can 
 change the semantics
 of the program entirely at the whim of the compiler 
 implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard.
I am watching this conversation for quite a while now and it's interesting to see how we went from the problem float f = 1.30; assert(f == 1.30); // NO assert(f == cast(float)1.30); //OK to this (endless) discussion.
 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.
Walter: I think this is a great idea and increased precision is always better! As you pointed out, yes the only reason to decrease precision if a trade-off for speed. Please keep doing/pushing this!
May 15 2016
prev sibling next sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.
Is there an option to use a reproducible fraction type that doesn't have the issues floating point has? Consider for most things perhaps this format would work better? struct Ftype { //Ftype a placeholder name long whole; long fraction; long divider; } Now 1.30 can perfectly be represented as Ftype(1,3,10) (or 13/10) and would never fail, represent incredibly small and incredibly large things, just not nearly to the degree floating/doubles could for scientific calculations, while at the same time avoiding the imprecision (or requirement) of the FPU entirely.
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 5:08 PM, Era Scarecrow wrote:
  Is there an option to use a reproducible fraction type that doesn't have the
 issues floating point has?
D has excellent facilities for creating your own types with the semantics you wish.
May 15 2016
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can 
 change the semantics
 of the program entirely at the whim of the compiler 
 implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard.
This is a crazy attitude to take. C++ provides means to detect that IEEE floats are being used in the standard library. C/C++ supports non-standard floating point because some platforms only provide non-standard floating point. They don't do it because it is _desirable_ in general. You might as well say that you are not required to drive on the right side on the road, because you occasionally have to drive on the left. So therefore it is ok to always drive on left.
 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.
No, D's take on floating point is FUBAR. const float value = 1.30; float copy = value; assert(value*0.5 == copy*0.5); // FAILS! => shutdown
May 15 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 08:00, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can change the
 semantics
 of the program entirely at the whim of the compiler implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard.
This is a crazy attitude to take. C++ provides means to detect that IEEE floats are being used in the standard library. C/C++ supports non-standard floating point because some platforms only provide non-standard floating point. They don't do it because it is _desirable_ in general. You might as well say that you are not required to drive on the right side on the road, because you occasionally have to drive on the left. So therefore it is ok to always drive on left.
 My proposal removes the "whim" by requiring 128 bit precision for CTFE.
No, D's take on floating point is FUBAR. const float value = 1.30; float copy = value; assert(value*0.5 == copy*0.5); // FAILS! => shutdown
This says more about promoting float operations to double than anything else, and has nothing to do with CTFE.
May 15 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:
 This says more about promoting float operations to double than 
 anything else, and has nothing to do with CTFE.
No, promoting to double is ok. NOT coercing the value to 32 bits representation when it is requested is what is FUBAR and that is the same issue as CTFE. But worse. This alone is a good reason to avoid D in production. Debugging signal processing/3D code is hard enough as it is. Not having a reliable way to cast to float32 representation is really really really bad. Random precision is not a good thing, in any way. 1. It does not improve accuracy in a predictable manner. If anything it adds noise. 2. It makes harnessing library code by extensive testing near impossible. 3. It makes debugging of complex system level floating point code very hard. This ought to be obvious.
May 16 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:
 This says more about promoting float operations to double than anything
 else, and has nothing to do with CTFE.
No, promoting to double is ok.
Your own example: const float value = 1.30; float copy = value; assert(value*0.5 == copy*0.5); Versus how you'd expect it to work. const float value = 1.30; float copy = value; assert(cast(float)(value*0.5) == cast(float)(copy*0.5)); // Compare as float, not double.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 07:54:32 UTC, Iain Buclaw wrote:
 On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 No, promoting to double is ok.
Your own example: const float value = 1.30; float copy = value; assert(value*0.5 == copy*0.5); Versus how you'd expect it to work. const float value = 1.30; float copy = value; assert(cast(float)(value*0.5) == cast(float)(copy*0.5)); // Compare as float, not double.
Promoting to double is sound. That is not the issue. Just replace the assert with a function like "check(real x, real y)". The programmer shouldn't have to look it up. Using "check(real x, real y)" should _not_ give a different result than using "check(float x, float y)". If you have implemented the former, you shouldn't have to implement the latter to get reasonable results. Binding to "const float" or "float" should not change the outcome. The problem is not the promotion, the problem is that you don't get the coercion to 32 bit floats where you requested it. This violates the principle of "least surprise".
May 16 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 10:25, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 07:54:32 UTC, Iain Buclaw wrote:
 On 16 May 2016 at 09:22, Ola Fosheim Grøstad via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 06:34:04 UTC, Iain Buclaw wrote:

 No, promoting to double is ok.
Your own example: const float value = 1.30; float copy = value; assert(value*0.5 == copy*0.5); Versus how you'd expect it to work. const float value = 1.30; float copy = value; assert(cast(float)(value*0.5) == cast(float)(copy*0.5)); // Compare as float, not double.
Promoting to double is sound. That is not the issue. Just replace the assert with a function like "check(real x, real y)". The programmer shouldn't have to look it up. Using "check(real x, real y)" should _not_ give a different result than using "check(float x, float y)". If you have implemented the former, you shouldn't have to implement the latter to get reasonable results. Binding to "const float" or "float" should not change the outcome. The problem is not the promotion, the problem is that you don't get the coercion to 32 bit floats where you requested it. This violates the principle of "least surprise".
But you *didn't* request coercion to 32 bit floats. Otherwise you would have used 1.30f.
May 16 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:
 But you *didn't* request coercion to 32 bit floats.  Otherwise 
 you would have used 1.30f.
const float f = 1.3f; float c = f; assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
May 16 2016
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:
 But you *didn't* request coercion to 32 bit floats.  Otherwise you would
 have used 1.30f.
const float f = 1.3f; float c = f; assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
Your still using doubles. Are you intentionally missing the point?
May 16 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:
 On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:
 But you *didn't* request coercion to 32 bit floats.  
 Otherwise you would have used 1.30f.
const float f = 1.3f; float c = f; assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
Your still using doubles. Are you intentionally missing the point?
I think you're missing what Ola means when he's talking about 32-bit floats. Of course when you multiply a float by a double (here, 1.0) you promote it to a double; but you'd expect the result to reflect the data available in the 32-bit float. Whereas in Ola's example, the fact that `const float f` is known at compile-time means that the apparent 32-bit precision is in fact entirely ignored when doing the * 1.0 calculation. In other words, Ola's expecting float * double == float * double but is getting something more like float * double == double * double or maybe even float * double == real * real ... because of the way FP constants are treated at compile time.
May 16 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:
 Your still using doubles.  Are you intentionally missing the 
 point?
You'll never know, he is Poe's law in action. At the end, it doesn't matter if he a moron or doing it intentionally, the result is the same.
May 16 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 7:28 AM, deadalnix wrote:
 On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:
 Your still using doubles.  Are you intentionally missing the point?
You'll never know, he is Poe's law in action. At the end, it doesn't matter if he a moron or doing it intentionally, the result is the same.
To all: please do not engage trolls. The best way to keep their incompetent platitudes away is to get them bored. -- Andrei
May 16 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 09:54:51 UTC, Iain Buclaw wrote:
 On 16 May 2016 at 10:52, Ola Fosheim Grøstad via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:
 But you *didn't* request coercion to 32 bit floats.  
 Otherwise you would have used 1.30f.
const float f = 1.3f; float c = f; assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
Your still using doubles. Are you intentionally missing the point?
What is your point? My point is that no other language I have ever used has overruled me requesting a coercion to single precision floats. And yes, binding it to a single precision float does qualify as a coercion on all platforms I have ever used. I should not have to implement a function with float parameters if I have a working function with real parameters, just to get reasonable behaviour: void assert_equality(real x, real y){ assert(x==y);} void main(){ const float f = 1.3f; float c = f; assert_equality(f,c); // Fails! } Stuff like this makes the codebase brittle. Not being able to control precision in unit tests make unit tests potentially succeed when they should fail. That makes testing floating point code for correctness virtually impossible in D. I don't use 32 bit float scalars to save space. I use it to get higher performance and in order to be able to turn the code into pure SIMD code at a later stage. So I require SIMD like semantics for float scalars. Anything less is unacceptable. If I want high precision at compile time, then I use rational numbers like std::ratio in C++ which gives me _exact_ values. If I want something more advanced then I use Maxima and literally copy in the results. C++17 is getting hex literals for floating point for a reason: accurate bit level representation.
May 16 2016
parent reply Max Samukha <maxsamukha gmail.com> writes:
On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad wrote:

 C++17 is getting hex literals for floating point for a reason: 
 accurate bit level representation.
D has had hex FP literals for ages.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 7:47 AM, Max Samukha wrote:
 On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad wrote:

 C++17 is getting hex literals for floating point for a reason: accurate bit
 level representation.
D has had hex FP literals for ages.
Since the first version, if I recall correctly. Of course, C++ had the idea first! (Actually, the idea came from the NCEG (Numerical C Extensions Group) work in the early 90's, which was largely abandoned. C++ has been very slow to adapt to the needs of numerics programmers.)
May 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 17 May 2016 at 20:50:06 UTC, Walter Bright wrote:
 On 5/16/2016 7:47 AM, Max Samukha wrote:
 On Monday, 16 May 2016 at 14:21:34 UTC, Ola Fosheim Grøstad 
 wrote:

 C++17 is getting hex literals for floating point for a 
 reason: accurate bit
 level representation.
D has had hex FP literals for ages.
Since the first version, if I recall correctly. Of course, C++ had the idea first! (Actually, the idea came from the NCEG (Numerical C Extensions Group) work in the early 90's, which was largely abandoned. C++ has been very slow to adapt to the needs of numerics programmers.)
The argument is not that C++ did anything first. The argument is that bit level precision for float is becoming the norm. Both in standards, languages and hardware. And that this makes it easier to write faster and more accurate code. Please note that I don't complain about D providing 80 bit floats. Just don't apply it when I request 32 bit floats. Some file formats have fields that are in the 80 bit intel format, so having access to it is a good thing.
May 18 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 08:52:16 UTC, Ola Fosheim Grøstad wrote:
 On Monday, 16 May 2016 at 08:47:03 UTC, Iain Buclaw wrote:
 But you *didn't* request coercion to 32 bit floats.  Otherwise 
 you would have used 1.30f.
const float f = 1.3f; float c = f; assert(c*1.0 == f*1.0); // Fails! SHUTDOWN!
IIRC, there are circumstances where you can get an equality failure even for non-compile-time calculations that superficially look identical: e.g. if one of the sides of the comparison is still stored in one of the 80-bit registers used for calculation. I ran into this when doing some work in std.complex ... :-( Reworking Ola's example: import std.stdio : writefln, writeln; const float constVar = 1.3f; float var = constVar; writefln("%.64g", constVar); writefln("%.64g", var); writefln("%.64g", constVar * 1.0); writefln("%.64g", var * 1.0); writefln("%.64g", 1.30 * 1.0); writefln("%.64g", 1.30f * 1.0); ... produces: 1.2999999523162841796875 1.2999999523162841796875 1.3000000000000000444089209850062616169452667236328125 1.2999999523162841796875 1.3000000000000000444089209850062616169452667236328125 1.3000000000000000444089209850062616169452667236328125 ... which is unintuitive, to say the least; the `f` qualifier on the manifest constant being assigned to `const float constVar` is being completely ignored for the purposes of the multiplication by 1.0. In other words, the specified precision of floating-point values is being ignored for calculations performed at compile-time, regardless of how much you "qualify" what you want.
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:14 AM, Joseph Rushton Wakeling wrote:
 1.2999999523162841796875
 1.3000000000000000444089209850062616169452667236328125
Note the increase in correctness of the result by 10 digits.
 ... which is unintuitive, to say the least;
It isn't any less intuitive than: f + f + 1.3f being calculated in 64 or 80 bit precision, which is commonplace, or for that matter: ubyte b = 200; ubyte c = 100; writeln(b + c); giving an answer of 300 (instead of 44), which every C/C++/D compiler does.
May 16 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 10:57:00 UTC, Walter Bright wrote:
 On 5/16/2016 3:14 AM, Joseph Rushton Wakeling wrote:
 1.2999999523162841796875
 1.3000000000000000444089209850062616169452667236328125
Note the increase in correctness of the result by 10 digits.
As Adam mentioned, you keep saying "correctness" or "accuracy", when people are consistently talking to you about "consistency" ... :-) I can always request more precision if I need or want it. Getting different results for a superficially identical float * double calculation, because one was performed at compile time and another at runtime, is an inconsistency that it might be nicer to avoid.
 ... which is unintuitive, to say the least;
It isn't any less intuitive than: f + f + 1.3f being calculated in 64 or 80 bit precision
It is less intuitive. If someFloat + 1.3f is calculated in 64 or 80 bit precision at runtime, it's still constrained by the fact that someFloat only provides 32 bits of floating-point input to the calculation. If someFloat + 1.3f is calculated instead at compile time, the reasonable assumption is that someFloat _still_ only brings 32 bits of floating-point input. But as we've seen above, it doesn't.
 or for that matter:

    ubyte b = 200;
    ubyte c = 100;
    writeln(b + c);

 giving an answer of 300 (instead of 44), which every C/C++/D 
 compiler does.
The latter result, at least (AIUI) is consistent depending on whether the calculation is done at compile time or runtime.
May 16 2016
next sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 16 May 2016 at 11:18:45 UTC, Joseph Rushton Wakeling 
wrote:
 As Adam mentioned, you keep saying "correctness" or "accuracy"
... meant to say '"correctness" or "precision"'.
May 16 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 4:18 AM, Joseph Rushton Wakeling wrote:
 you keep saying "correctness" or "accuracy", when people are
 consistently talking to you about "consistency" ... :-)
I see consistently wrong answers as being as virtuous as getting tires for my car that are not round :-)
 I can always request more precision if I need or want it.  Getting different
 results for a superficially identical float * double calculation, because one
 was performed at compile time and another at runtime, is an inconsistency that
 it might be nicer to avoid.
As the links I posted explain in great detail, such a design produces other problems. Secondly, as I think this thread amply demonstrates, few programmers are particularly aware of the issues of cumulative roundoff error. It's very easy to miss it and just assume the result of the calculation is precise to 7 digits, when it might be off by factors of 2.
 The latter result, at least (AIUI) is consistent depending on whether the
 calculation is done at compile time or runtime.
True, but I was talking about intuitiveness. Integral promotion rules are not intuitive, they are a surprise to every C/C++/D newbie. Except for me, since I came from a PDP-11 assembler background, and that's how the PDP-11 CPU worked, and C's semantics are based on the 11.
May 16 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:00 PM, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can change the
semantics
 of the program entirely at the whim of the compiler implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard.
This is a crazy attitude to take. C++ provides means to detect that IEEE floats are being used in the standard library.
IEEE floats do not specify precision of intermediate results. A C/C++ compiler can be fully IEEE compliant and yet legitimately have increased precision for intermediate results. I posted several links here pointing out this behavior in VC++ and g++. If your C++ numerics code didn't have a problem with it, it's likely you wrote the code in such a way that more accurate answers were not wrong. FP behavior has complex trade-offs with speed, accuracy, compatibility, and size. There are no easy, obvious answers.
May 16 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 08:10:02 UTC, Walter Bright wrote:
 IEEE floats do not specify precision of intermediate results. A 
 C/C++ compiler can be fully IEEE compliant and yet legitimately 
 have increased precision for intermediate results.
IEEE 754-2008 provide language designers with features that enables predictable bit-accurate computations using floats for ordinary floating point operations. Only a subset of functions are implementation defined. This also has the advantage that you can do bit-level optimizations... including proving asserts to hold at compile time and "assume assert" optimizations that you are fond of ;-). But all the C/C++ compilers I have used support reliable coercion to 32 bit floats.
 I posted several links here pointing out this behavior in VC++ 
 and g++. If your C++ numerics code didn't have a problem with 
 it, it's likely you wrote the code in such a way that more 
 accurate answers were not wrong.
I use clang++ only for production, and I don't really care how I wrote my code. What I do know is that in performance optimized code for 32 bit floats and simd I do rely upon unit-testing with guaranteed 32 bit floats. I absolutely do not want unit tests to execute with higher precision. I want it to break if 32 bit floats fails. If I cannot be sure of this I risk having libraries that enter infinite loops in production code. Keep in mind that even "simple" algorithms can get complex when you write for high performance. E.g. sound processing. So having predictable outcome is very much desirable. Such code also don't gain much from compiler optimizations...
 FP behavior has complex trade-offs with speed, accuracy, 
 compatibility, and size. There are no easy, obvious answers.
Well, but randomly increasing precision is always a bad idea when you deal with related computations, like time series. It is better to have consistent noise/bias than random noise/bias. Also, in the case of audio processing it is not unheard of to exploit the 24 bit mantissa and the specifics of the IEEE 32 bit floating point format. Now, I don't object to having a "real" type that works the way you want. What I object to is having float and double act that way. Or rather, not having strict ieee32 and ieee64 types.
May 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 4:10 AM, Walter Bright wrote:
 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.
That's a fair statement. My understanding is also that 80-bit math is on the wrong side of the tradeoff simply because it's disproportionately slow (again I cite http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All modern ALUs I looked at have 32- and 64-bit FP units only. I'm trying to figure why. -- Andrei
May 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:33 AM, Andrei Alexandrescu wrote:
 On 5/16/16 4:10 AM, Walter Bright wrote:
 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.
That's a fair statement. My understanding is also that 80-bit math is on the wrong side of the tradeoff simply because it's disproportionately slow (again I cite http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All modern ALUs I looked at have 32- and 64-bit FP units only. I'm trying to figure why. --
I think it is slow because no effort has been put into speeding it up. All the effort went into SIMD. The x87 FPU is a library module that is just plopped onto the chip for compatibility. The x87 register stack also sux because it's hard to generate good code for. That didn't help.
May 16 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 7:53 AM, Walter Bright wrote:
 On 5/16/2016 3:33 AM, Andrei Alexandrescu wrote:
 On 5/16/16 4:10 AM, Walter Bright wrote:
 FP behavior has complex trade-offs with speed, accuracy, compatibility,
 and size. There are no easy, obvious answers.
That's a fair statement. My understanding is also that 80-bit math is on the wrong side of the tradeoff simply because it's disproportionately slow (again I cite http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/). All modern ALUs I looked at have 32- and 64-bit FP units only. I'm trying to figure why. --
I think it is slow because no effort has been put into speeding it up. All the effort went into SIMD. The x87 FPU is a library module that is just plopped onto the chip for compatibility.
That makes sense.
 The x87 register stack also sux because it's hard to generate good code
 for. That didn't help.
It may indeed be a bummer but it's the way it is. So we should act accordingly. -- Andrei
May 16 2016
prev sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 16 May 2016 at 10:33:58 UTC, Andrei Alexandrescu wrote:
 My understanding is also that 80-bit math is on the wrong side 
 of the tradeoff simply because it's disproportionately slow.
Speed in theory shouldn't be that big of a problem. As I recall the FPU *was* a separate processor; Sending the instructions took like 3 cycles. Following that you could do other stuff before returning for the result(s), but that assumes you aren't *only* doing FPU work. The issue would then come up when you are waiting for the result after the fact (and that's only for really slow operations, most operations are very fast, but my knowledge/experience is probably more than a decade out of date). Today I'm sure the FPU is built directly into the CPU. Not to mention there's a whole slew of instructions for the x86 that haven't improved much because they aren't used at all and instead are there for historical reasons or for compatibility. jcxz, xlat, rep, and probably another dozen I am not sure just off the top of my head. Perhaps more processors and in general should move to a RISC style instructions.
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 8:15 PM, Era Scarecrow wrote:
  Speed in theory shouldn't be that big of a problem. As I recall the FPU *was*
a
 separate processor; Sending the instructions took like 3 cycles. Following that
 you could do other stuff before returning for the result(s), but that assumes
 you aren't *only* doing FPU work. The issue would then come up when you are
 waiting for the result after the fact (and that's only for really slow
 operations, most operations are very fast, but my knowledge/experience is
 probably more than a decade out of date).
With some clever programming, you could execute other instructions in parallel with the x87.
May 17 2016
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 18:10, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 11:00 PM, Ola Fosheim Grøstad wrote:
 On Sunday, 15 May 2016 at 22:49:27 UTC, Walter Bright wrote:
 On 5/15/2016 2:06 PM, Ola Fosheim Grøstad wrote:
 The net result is that adding const/immutable to a type can change the
 semantics
 of the program entirely at the whim of the compiler implementor.
C++ Standard allows the same increased precision, at the whim of the compiler implementor, as quoted to you earlier. What your particular C++ compiler does is not relevant, as its behavior is not required by the Standard.
This is a crazy attitude to take. C++ provides means to detect that IEEE floats are being used in the standard library.
IEEE floats do not specify precision of intermediate results. A C/C++ compiler can be fully IEEE compliant and yet legitimately have increased precision for intermediate results. I posted several links here pointing out this behavior in VC++ and g++. If your C++ numerics code didn't have a problem with it, it's likely you wrote the code in such a way that more accurate answers were not wrong. FP behavior has complex trade-offs with speed, accuracy, compatibility, and size. There are no easy, obvious answers.
I think I might have misunderstood Iain's initial post on the matter. I understand the links you've posted, but the point I was objecting to was the moment an assignment is made, truncation must occur. If you do: float f() { float x = some_float_expression; float y = some_expression_involving_x; return some_expression_involving_y; } If I run that in CTFE, will assignment to x, y, and the return statement all truncate intermediate results to float at those moments? If the answer is yes, I apologise for the confusion.
May 16 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sunday, May 15, 2016 15:49:27 Walter Bright via Digitalmars-d wrote:
 My proposal removes the "whim" by requiring 128 bit precision for CTFE.
Based on some of the recent discussions, it sounds like having soft floating point in the compiler would also help with cross-compilation. So, completely aside from the precision chosen, it sounds like having a soft floating point implementation in CTFE would definitely help - though maybe I misunderstood. My understanding of the floating point stuff is pretty bad, unfortunately. And Don's talk and this discussion don't exactly make me feel any better about it - since of course what I'd really like is math math and not magician's math, but that's not what we get, and it will never be what we get. I really should study up on FP in detail in the near future. My typical approach has been to avoid it as much as possible and then use the highest precision possible when I'm stuck using it in an attempt to minimize the error that creeps in. - Jonathan M Davis
May 16 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Monday, 16 May 2016 at 18:44:57 UTC, Jonathan M Davis wrote:
 On Sunday, May 15, 2016 15:49:27 Walter Bright via 
 Digitalmars-d wrote:
 My proposal removes the "whim" by requiring 128 bit precision 
 for CTFE.
Based on some of the recent discussions, it sounds like having soft floating point in the compiler would also help with cross-compilation. So, completely aside from the precision chosen, it sounds like having a soft floating point implementation in CTFE would definitely help - though maybe I misunderstood. My understanding of the floating point stuff is pretty bad, unfortunately.
My understanding of floating point is bad too. I understand fixed floating point (a number of bits is considered the fraction) but as I recall while trying to break down and answer questions while referring to as much of the information as I could, the power/significand portion I got stuck when looking at the raw bits and proper answers failed to come out. As for soft floating point, I'd hopefully see an option to control how much precision you can raise it to (so it would probably be a template struct), as well as making it a library we can use. Same with fixed integers which we could then incorporate cent and ucent until such a time that the types are commonly avaliable. (Afterall 128bit computers will come sooner or later, maybe in 20 years? I doubt the memory model would need to move up from 64bit though) Although for implementation it could use BCD math instead (which is easier to print off as decimal); Just specify the number of digits for precision (40+), how many digits it can shift larger/smaller; The floating point type would be related to the 8-bit computers of old but on steroids! Actually basic floating point like that wouldn't be too hard to implement over a weekend.
May 16 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 17 May 2016 04:12:09 +0000
schrieb Era Scarecrow <rtcvb32 yahoo.com>:

 I understand fixed floating point
Quote of the day :D -- Marco
May 17 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
I had written and sent this message three days ago, but it seemingly 
never showed up on the newsgroup. I'm sorry if it seemed that I didn't 
explain myself, I was operating under the assumption that this message 
had been made available to you.


On 14.05.2016 03:26, Walter Bright wrote:
 On 5/13/2016 5:49 PM, Timon Gehr wrote:
 Nonsense. That might be true for your use cases. Others might actually
 depend on
 IEE 754 semantics in non-trivial ways. Higher precision for
 temporaries does not
 imply higher accuracy for the overall computation.
Of course it implies it. ...
No, see below.
 An anecdote: a colleague of mine was once doing a chained calculation.
 At every step, he rounded to 2 digits of precision after the decimal
 point, because 2 digits of precision was enough for anybody. I carried
 out the same calculation to the max precision of the calculator (10
 digits). He simply could not understand why his result was off by a
 factor of 2, which was a couple hundred times his individual roundoff
 error.
 ...
Now assume that colleague of your was doing that chained calculation, and his calculator magically added the additional digits behind his back (it can do this by caching the last full-precision value for each number prefix). He wouldn't even notice that his rounding strategy does not work. Sometime later he might then use a calculator that does not do the magical enhancing.
 E.g., correctness of double-double arithmetic is crucially dependent
 on correct
 rounding semantics for double:
 
https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic

 Double-double has its own peculiar issues, and is not relevant to this
 discussion.
 ...
It is relevant to this discussion insofar that it can occur in algorithms that use double-precision floating-point arithmetic. It illustrates a potential issue with implicit enhancement of precision. For double-double, there are two values of type double that together represent a higher-precision value. (One of them has a shifted exponent, such that their mantissa bits do not overlap.) You have mantissas like: |------double1------| |------double2--------| Now assume that the compiler instead uses extended precision, what you get is something we might call extended-extended of the form: |---------extended1---------| |---------extended2-----------| Now those values are written back into 64-bit double storage, now observe which part of the double-double mantissa is lost: |---------extended1---xxxxxx| |---------extended2-----xxxxxx| | v |------double1------| |------double2--------| The middle part of the mantissa is thrown away, and we are left with single double-precision plus some noise. Implicitly using extended precision for some parts of the computation approximately cuts the number of accurate mantissa bits in half. I don't want to have to deal with this. Just give me what I ask for.
 Also, it seems to me that for e.g.
 https://en.wikipedia.org/wiki/Kahan_summation_algorithm,
 the result can actually be made less precise by adding casts to higher
 precision
 and truncations back to lower precision at appropriate places in the
 code.
I don't see any support for your claim there. ....
It's using the same trick that double-double does. The above reasoning should apply.
 And even if higher precision helps, what good is a "precision-boost"
 that e.g.
 disappears on 64-bit builds and then creates inconsistent results?
That's why I was thinking of putting in 128 bit floats for the compiler internals. ...
Runtime should do the same as CTFE. Are you suggesting we use 128-bit soft-floats at run time for all float types?
 Sometimes reproducibility/predictability is more important than maybe
 making
 fewer rounding errors sometimes. This includes reproducibility between
 CTFE and
 runtime.
A more accurate answer should never cause your algorithm to fail.
It's not more accurate, just more precise, and it is only for some temporary computations, and you don't necessarily know which. The way the new roundoff errors propagate is chaotic, and might not be what the code anticipated.
 It's like putting better parts in your car causing the car to fail.
 ...
It's like (possibly repeatedly) interchanging "better" parts and "worse" parts while the engine is still running. Anyway, it should be obvious that this kind of reasoning by analogy does not lead anywhere.
 Just actually comply to the IEEE floating point standard when using 
their
 terminology. There are algorithms that are designed for it and that
 might stop
 working if the language does not comply.
Conjecture.
I have given a concrete example.
 I've written FP algorithms (from Cody+Waite, for example),
 and none of them degraded when using more precision.
 ...
For the entire computation or some random temporaries?
 Consider that the 8087 has been operating at 80 bits precision by
 default for 30 years. I've NEVER heard of anyone getting actual bad
 results from this.
Fine, so you haven't.
 They have complained about their test suites that
 tested for less accurate results broke.
What happened is that the test suites broke.
 They have complained about the
 speed of x87. And Intel has been trying to get rid of the x87 forever.
It's nice to have 80-bit precision. I just want to explicitly ask for it.
 Sometimes I wonder if there's a disinformation campaign about more
 accuracy being bad, because it smacks of nonsense.

 BTW, I once asked Prof Kahan about this. He flat out told me that the
 only reason to downgrade precision was if storage was tight or you
 needed it to run faster. I am not making this up.
Obviously, but I think his comment was about enhancing precision for the entire computation front-to-back, not just some parts of it. I can do that on my own. I don't need the compiler to second-guess me.
May 18 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 14.05.2016 02:49, Timon Gehr wrote:
 On 13.05.2016 23:35, Walter Bright wrote:
 On 5/13/2016 12:48 PM, Timon Gehr wrote:
 IMO the compiler should never be allowed to use a precision different
 from the one specified.
I take it you've never been bitten by accumulated errors :-) ...
If that was the case it would be because I explicitly ask for high precision if I need it. If the compiler using or not using a higher precision magically fixes an actual issue with accumulated errors, that means the correctness of the code is dependent on something hidden, that you are not aware of, and that could break any time, for example at a time when you really don't have time to track it down.
 Reduced precision is only useful for storage formats and increasing
 speed.  If a less accurate result is desired, your algorithm is wrong.
Nonsense. That might be true for your use cases. Others might actually depend on IEE 754 semantics in non-trivial ways. Higher precision for temporaries does not imply higher accuracy for the overall computation. E.g., correctness of double-double arithmetic is crucially dependent on correct rounding semantics for double: https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format#Double-double_arithmetic
We finally have someone on D.learn who is running into this exact problem: http://forum.dlang.org/post/vimvfarzqkcmbvtnznrf forum.dlang.org
Jun 28
prev sibling parent Binarydepth <binarydepth gmail.com> writes:
On Monday, 9 May 2016 at 11:26:55 UTC, Walter Bright wrote:
 I wonder what's the difference between 1.30f and 
 cast(float)1.30.
There isn't one.
I always saw type casting happening run-time and "literals" (pardon if that's not the correct term) to happen during compilation. I know that there's no difference in this topic since it is a comparison of values.
May 14 2016
prev sibling next sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
Yes, I think it is a good idea, just like emitting a warning for mismatched signed/unsigned comparison.
May 09 2016
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 4:16 AM, ZombineDev wrote:
 just like emitting a warning for mismatched signed/unsigned comparison.
Not at all the same, such are not always false.
May 09 2016
prev sibling parent reply Ecstatic Coder <ecstatic.coder gmail.com> writes:
On Monday, 9 May 2016 at 11:16:53 UTC, ZombineDev wrote:
 On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
Yes, I think it is a good idea, just like emitting a warning for mismatched signed/unsigned comparison.
I agree for the float but not for the long/ulong comparisons. I often do code like "x < array.length" where x needs to be a long to be able to handle negative values. I want my code to compile without warning, and therefore I'm against requiring "x < array.length.to!long()" to remove that warning.
Jun 29
parent reply ag0aep6g <anonymous example.com> writes:
On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:
 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.
`x < array.length` and `x < array.length.to!long` have different results when x is negative. That's why a warning/error is in order.
Jun 29
next sibling parent Ecstatic Coder <ecstatic.coder gmail.com> writes:
On Thursday, 29 June 2017 at 19:12:24 UTC, ag0aep6g wrote:
 On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:
 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.
`x < array.length` and `x < array.length.to!long` have different results when x is negative. That's why a warning/error is in order.
I often have array indices that go up and down (x++/x--) depending on the logic. I find convenient to be able to test them (x >= 0, x < a.length) without having to manage the fact that the array stores its length as an unsigned integer to double its potential size, since anyway I never use arrays with 2^63 items.
Jun 29
prev sibling parent reply Ecstatic Coder <ecstatic.coder gmail.com> writes:
On Thursday, 29 June 2017 at 19:12:24 UTC, ag0aep6g wrote:
 On Thursday, 29 June 2017 at 18:03:39 UTC, Ecstatic Coder wrote:
 I often do code like "x < array.length" where x needs to be a 
 long to be able to handle negative values.

 I want my code to compile without warning, and therefore I'm 
 against requiring "x < array.length.to!long()" to remove that 
 warning.
`x < array.length` and `x < array.length.to!long` have different results when x is negative. That's why a warning/error is in order.
I can perfectly understand that others may want to check their code in a "strict" mode. So actually I'm not against a warning for signed/unsigned implicit conversions. I'm just against putting it on by default, so that the current behavior is kept, because I don't see where is the language improvement in having to put these ugly manual conversion everywhere just because the string/array length was made unsigned. I always use signed integers for string/array indices because unsigned indices become dangerous as soon as your algorithm starts to decrement it... And as I said, I compile my D code with the 64-bit compiler, and 2^63 characters/items is much more than needed for my personal use cases. So until the day I will have a computer which can store a string of 16 million terabytes, at the moment I prefer to use long values for indices, instead of ulong, because negative indices can already occur right now with my current algorithms.
Jun 29
parent reply ag0aep6g <anonymous example.com> writes:
On 06/30/2017 07:38 AM, Ecstatic Coder wrote:
 I'm just against putting it on by default, so that the current behavior 
 is kept, because I don't see where is the language improvement in having 
 to put these ugly manual conversion everywhere just because the 
 string/array length was made unsigned.
So what you really want is: signed array lengths. You only have a use for the sloppy conversion, because D doesn't have signed array lengths.
 I always use signed integers for string/array indices because unsigned 
 indices become dangerous as soon as your algorithm starts to decrement 
 it...
I guess they're "dangerous" because you get ulong.max instead of (a signed) -1? But that's exactly what happens with the implicit conversion, too. At some point you have to think of that. Either when adding/subtracting or when comparing signed with unsigned. With a warning/error on the implicit conversion, you'd get a nice reminder. By the way, std.experimental.checkedint may interesting: ---- import std.experimental.checkedint: checked, ProperCompare, Saturate; /* If negative values should not occur, throw an error on wrap around: */ auto x = checked(size_t(0)); --x; /* error */ /* Or stop at 0: */ auto y = checked!Saturate(size_t(0)); --y; /* does nothing */ assert(y == 0); /* passes */ /* If negative values are valid, you can use proper comparisons without sloppy implicit conversions or verbose manual ones: */ auto z = checked!ProperCompare(int(-1)); auto array = [1, 2, 3]; assert(z < array.length); /* passes */ ----
Jun 30
parent Ecstatic Coder <ecstatic.coder gmail.com> writes:
 So what you really want is: signed array lengths. You only have 
 a use for the sloppy conversion, because D doesn't have signed 
 array lengths.
Yes and no. Signed safety is better, but the current behavior works too. Honestly I don't care if the bit maniacs want 8 million terabytes more for their strings, as long as I can safely ignore that implementation detail.
Jun 30
prev sibling next sibling parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 So, should the compiler emit a warning for the former case?
Yes, please. I would prefer, at least, a warning flag for that.
May 09 2016
prev sibling next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 So, should the compiler emit a warning for the former case?
Would that include comparison of variables only aswell? float f = 1.3; double d = 1.3; assert(f == d); // compiler warning
May 09 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 4:38 AM, Nordlöw wrote:
 Would that include comparison of variables only aswell?

     float f = 1.3;
     double d = 1.3;
     assert(f == d); // compiler warning
No.
May 09 2016
next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:
 On 5/9/2016 4:38 AM, Nordlöw wrote:
 Would that include comparison of variables only aswell?

     float f = 1.3;
     double d = 1.3;
     assert(f == d); // compiler warning
No.
Just get rid of the problem : remove == and != from floats.
May 09 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 9 May 2016 at 12:29:14 UTC, Temtaime wrote:
 Just get rid of the problem : remove == and != from floats.
Diagnostics Suggestion: Issue a warning including a direction to approxEqual() https://dlang.org/phobos/std_math.html#.approxEqual
May 09 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:
 On 5/9/2016 4:38 AM, Nordlöw wrote:
 Would that include comparison of variables only aswell?
No.
Why?
May 09 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 09 May 2016 15:56:21 +0000
schrieb Nordl=C3=B6w <per.nordlow gmail.com>:

 On Monday, 9 May 2016 at 12:28:04 UTC, Walter Bright wrote:
 On 5/9/2016 4:38 AM, Nordl=C3=B6w wrote: =20
 Would that include comparison of variables only aswell? =20
No. =20
=20 Why?
Because the float would be converted to double without loss of precision just like uint =3D=3D ulong is a valid comparison which can yield 'true' in 2^32 cases. float =3D=3D 1.30 on the other hand is false in any case. You'd have to warn on _every_ comparison with a widening conversion to be consistent! --=20 Marco
May 09 2016
prev sibling next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
I'd assume in the first case that the float is being promoted to double for the comparison. Is there already a warning for loss of precision? We treat warnings as errors in our C++ code, so C4244 triggers all the time in MSVC with integer operations. I just tested that float initialisation in MSVC, initialising a float with a double triggers C4305. So my preference is "Yes please". https://msdn.microsoft.com/en-us/library/th7a07tz.aspx https://msdn.microsoft.com/en-us/library/0as1ke3f.aspx
May 09 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 5:21 AM, Ethan Watson wrote:
 I'd assume in the first case that the float is being promoted to double for the
 comparison. Is there already a warning for loss of precision?
Promoting to double does not lose precision.
 We treat warnings
 as errors in our C++ code, so C4244 triggers all the time in MSVC with integer
 operations. I just tested that float initialisation in MSVC, initialising a
 float with a double triggers C4305.
That's going quite a bit further.
May 09 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Monday, 9 May 2016 at 12:30:13 UTC, Walter Bright wrote:
 Promoting to double does not lose precision.
Yes, badly worded on my part, I was getting at the original assignment from double to float.
May 09 2016
prev sibling next sibling parent reply Xinok <xinok live.com> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
(1) Yes, emit a warning for this case. (2) Generalize it to all variables, like Nordlöw suggested. (3) Generalize it to all comparisons as well, including < and > . (4) While we're at it, let's also emit a warning when comparing signed and unsigned types. (5) Dare I say it... warn against implicit conversions of double to float. (6) The same applies to "real" as well. All of these scenarios are capable of producing "incorrect" results, are a source of discrete bugs (often corner cases that we failed to consider and test), and can be hard to detect. It's about time we stopped being stubborn and flagged these things as warnings. Even if they require a special compiler flag and are disabled by default, that's better than nothing.
May 09 2016
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:
 (1) Yes, emit a warning for this case.
 (2) Generalize it to all variables, like Nordlöw suggested.
 (3) Generalize it to all comparisons as well, including < and > 
 .
 (4) While we're at it, let's also emit a warning when comparing 
 signed and unsigned types.
 (5) Dare I say it... warn against implicit conversions of 
 double to float.
 (6) The same applies to "real" as well.

 All of these scenarios are capable of producing "incorrect" 
 results, are a source of discrete bugs (often corner cases that 
 we failed to consider and test), and can be hard to detect. 
 It's about time we stopped being stubborn and flagged these 
 things as warnings. Even if they require a special compiler 
 flag and are disabled by default, that's better than nothing.
(1) is good, because the code in question is always wrong. (2) is a logical extension, in those cases where constant folding and VRP can prove that the code is always wrong. (3) Makes no sense though; inequalities with mixed floating-point types are perfectly safe. (Well, as safe as any floating-point code can be, anyway.) (4) is already planned; it's just taking *a lot* longer than anticipated to actually implement it: https://issues.dlang.org/show_bug.cgi?id=259 https://github.com/dlang/dmd/pull/1913 https://github.com/dlang/dmd/pull/5229
May 09 2016
next sibling parent reply Xinok <xinok live.com> writes:
On Monday, 9 May 2016 at 18:51:58 UTC, tsbockman wrote:
 On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:
 ...
 (3) Generalize it to all comparisons as well, including < and 
 ...
(3) Makes no sense though; inequalities with mixed floating-point types are perfectly safe. (Well, as safe as any floating-point code can be, anyway.) ...
Not necessarily. Reusing 1.3 from the original case, the following assertion passes: float f = 1.3; assert(f < 1.3); And considering we also have <= and >=, we may as well check all of the comparison operators. It's a complex issue because there isn't necessarily right or wrong behavior, only things that *might* be wrong. But if we want to truly detect all possible cases of incorrect behavior, then we have to be exhaustive in our checks.
May 09 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Monday, 9 May 2016 at 19:15:20 UTC, Xinok wrote:
 It's a complex issue because there isn't necessarily right or 
 wrong behavior, only things that *might* be wrong. But if we 
 want to truly detect all possible cases of incorrect behavior, 
 then we have to be exhaustive in our checks.
We absolutely, emphatically, DO NOT want to "detect all possible cases of incorrect behavior". Given the limited information available to the compiler and the intractability of the Halting Problem, the correct algorithm for doing so is this: foreach(line; sourceCode) warning("This line could not be proven correct."); Only warnings with a reasonably high signal-to-noise ratio should be implemented.
 Not necessarily. Reusing 1.3 from the original case, the 
 following assertion passes:

     float f = 1.3;
     assert(f < 1.3);

 And considering we also have <= and >=, we may as well check 
 all of the comparison operators.
Because of the inevitability of rounding errors, FP code generally cannot be allowed to depend upon precise equality comparisons to any value other than zero (even implicit ones as in your example). Warning about this makes sense: float f = 1.3; assert(f == 1.3); Because the `==` makes it clear that the programmer's intent was to test for equality, and that will fail unexpectedly. On the other hand, with `<`, `>`, `<=`, and `>=`, the compiler should generally assume that they are being used correctly, in a way that is tolerant of small rounding errors. Doing otherwise would cause tons of false positives in competently written FP code. Educating programmers who've never studied how to write correct FP code is too complex of a task to implement via compiler warnings. The warnings should be limited to cases that are either obviously wrong, or where the warning is likely to be a net positive even for FP experts.
May 09 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 12:39 PM, tsbockman wrote:
 Educating programmers who've never studied how to write correct FP code is too
 complex of a task to implement via compiler warnings. The warnings should be
 limited to cases that are either obviously wrong, or where the warning is
likely
 to be a net positive even for FP experts.
I've seen a lot of proposals which try to hide the reality of how FP works. The cure is worse than the disease. The same goes for hiding signed/unsigned, and the autodecode mistake of pretending that code units aren't there.
May 09 2016
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 9 May 2016 at 20:20:00 UTC, Walter Bright wrote:
 On 5/9/2016 12:39 PM, tsbockman wrote:
 Educating programmers who've never studied how to write 
 correct FP code is too
 complex of a task to implement via compiler warnings. The 
 warnings should be
 limited to cases that are either obviously wrong, or where the 
 warning is likely
 to be a net positive even for FP experts.
I've seen a lot of proposals which try to hide the reality of how FP works. The cure is worse than the disease. The same goes for hiding signed/unsigned, and the autodecode mistake of pretending that code units aren't there.
I completely agree that complexity that cannot be properly hidden should not be hidden. The underlying mechanisms of floating point is complexity that we shouldn't paper over. However, the peculiarities of language conventions w.r.t. floating point expressions doesn't quite fit that category.
May 09 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.05.2016 22:20, Walter Bright wrote:
 On 5/9/2016 12:39 PM, tsbockman wrote:
 Educating programmers who've never studied how to write correct FP
 code is too
 complex of a task to implement via compiler warnings. The warnings
 should be
 limited to cases that are either obviously wrong, or where the warning
 is likely
 to be a net positive even for FP experts.
I've seen a lot of proposals which try to hide the reality of how FP works. The cure is worse than the disease. The same goes for hiding signed/unsigned, and the autodecode mistake of pretending that code units aren't there.
I feel the same way about automated enhancement of precision for intermediate computations behind the back of the programmer. In the best case, you are pretending that the algorithm has better numerical properties than it actually has, in the worst case you are destroying the accuracy of the result.
May 17 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 05/17/2016 02:13 PM, Timon Gehr wrote:
 On 09.05.2016 22:20, Walter Bright wrote:
 On 5/9/2016 12:39 PM, tsbockman wrote:
 Educating programmers who've never studied how to write correct FP
 code is too
 complex of a task to implement via compiler warnings. The warnings
 should be
 limited to cases that are either obviously wrong, or where the warning
 is likely
 to be a net positive even for FP experts.
I've seen a lot of proposals which try to hide the reality of how FP works. The cure is worse than the disease. The same goes for hiding signed/unsigned, and the autodecode mistake of pretending that code units aren't there.
I feel the same way about automated enhancement of precision for intermediate computations behind the back of the programmer. In the best case, you are pretending that the algorithm has better numerical properties than it actually has, in the worst case you are destroying the accuracy of the result.
That's an interesting assessment, thanks. -- Andrei
May 17 2016
prev sibling parent Observer <here nowhere.org> writes:
On Monday, 9 May 2016 at 19:39:52 UTC, tsbockman wrote:
 Educating programmers who've never studied how to write correct 
 FP code is too complex of a task to implement via compiler 
 warnings. The warnings should be limited to cases that are 
 either obviously wrong, or where the warning is likely to be a 
 net positive even for FP experts.
Any warning message for this type of problem should mention the "What Every Computer Scientist Should Know About Floating-Point Arithmetic" paper (and perhaps give a standard public URL such as https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html at which the paper can be easily accessed).
May 10 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 11:51 AM, tsbockman wrote:
 On Monday, 9 May 2016 at 18:37:10 UTC, Xinok wrote:
 (4) While we're at it, let's also emit a warning when comparing signed and
 unsigned types.
(4) is already planned; it's just taking *a lot* longer than anticipated to actually implement it: https://issues.dlang.org/show_bug.cgi?id=259 https://github.com/dlang/dmd/pull/1913 https://github.com/dlang/dmd/pull/5229
I oppose this change. You'd be better off not having unsigned types at all than this mess, which was Java's choice. But then there are more problems created.
May 09 2016
next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:
 On 5/9/2016 11:51 AM, tsbockman wrote:
 (4) is already planned; it's just taking *a lot* longer than 
 anticipated to
 actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229
I oppose this change. You'd be better off not having unsigned types at all than this mess, which was Java's choice. But then there are more problems created.
What mess? The actual fix for issue 259 is simple, elegant, and shouldn't require much code in the wild to be changed. The difficulties and delays have all been associated with the necessary improvements to VRP and constant folding, which are worthwhile in their own right, since they help the compiler generate faster code.
May 09 2016
prev sibling next sibling parent tsbockman <thomas.bockman gmail.com> writes:
On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:
 (4) is already planned; it's just taking *a lot* longer than 
 anticipated to
 actually implement it:
     https://issues.dlang.org/show_bug.cgi?id=259
     https://github.com/dlang/dmd/pull/1913
     https://github.com/dlang/dmd/pull/5229
I oppose this change. You'd be better off not having unsigned types at all than this mess, which was Java's choice. But then there are more problems created.
One other thing - according to the bug report discussion, the proposed solution was pre-approved both by Andrei Alexandrescu and by *YOU*. Proposing a solution, letting various people work on implementing it for three years, and then suddenly announcing that you "oppose this change" and calling the solution a "mess" with no explanation is a fantastic way to destroy all motivation for outside contributors.
May 09 2016
prev sibling parent Kagamin <spam here.lot> writes:
On Monday, 9 May 2016 at 20:16:59 UTC, Walter Bright wrote:
 I oppose this change. You'd be better off not having unsigned 
 types at all than this mess, which was Java's choice.
The language forces usage of unsigned types. Though in my experience it's relatively easy to fight back including interfacing with C that uses unsigned types exclusively.
 But then there are more problems created.
I've seen no problem from using signed types so far. The last prejudice left is usage of ubyte[] for buffers. How often one looks into individual bytes in some abstract buffer?
May 11 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 11:37 AM, Xinok wrote:
 All of these scenarios are capable of producing "incorrect" results, are a
 source of discrete bugs (often corner cases that we failed to consider and
 test), and can be hard to detect. It's about time we stopped being stubborn and
 flagged these things as warnings. Even if they require a special compiler flag
 and are disabled by default, that's better than nothing.
I've used a B+D language that does as you suggest (Wirth Pascal). It was highly unpleasant to use, as the code became littered with casts. Casts introduce their own set of bugs.
May 09 2016
parent reply Xinok <xinok live.com> writes:
On Monday, 9 May 2016 at 20:14:36 UTC, Walter Bright wrote:
 On 5/9/2016 11:37 AM, Xinok wrote:
 All of these scenarios are capable of producing "incorrect" 
 results, are a
 source of discrete bugs (often corner cases that we failed to 
 consider and
 test), and can be hard to detect. It's about time we stopped 
 being stubborn and
 flagged these things as warnings. Even if they require a 
 special compiler flag
 and are disabled by default, that's better than nothing.
I've used a B+D language that does as you suggest (Wirth Pascal). It was highly unpleasant to use, as the code became littered with casts. Casts introduce their own set of bugs.
Maybe it's a bad idea to enable these warnings by default but what's wrong with providing a compiler flag to perform these checks anyways? For example, GCC has a compiler flag to yield warnings for signed+unsigned comparisons but it's not even enabled with the -Wall flag, only by specifying -Wextra or -Wsign-compare.
May 09 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 8:00 PM, Xinok wrote:
 Maybe it's a bad idea to enable these warnings by default but what's wrong with
 providing a compiler flag to perform these checks anyways? For example, GCC has
 a compiler flag to yield warnings for signed+unsigned comparisons but it's not
 even enabled with the -Wall flag, only by specifying -Wextra or -Wsign-compare.
Warnings balkanize the language into endless dialects.
May 10 2016
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, May 09, 2016 02:10:19 Walter Bright via Digitalmars-d wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

      float f = 1.30;
      assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

      float f = 1.30;
      assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
It does seem like having implicit conversions with floating point numbers is problematic in general, though warning about it or making it illegal could very well be too annoying to be worth it. But at bare minimum, warning about literals not matching the type that they're being compared against when there _is_ a literal that would be of the same type is probably worth warning about - and that could apply to more than just floating point values. But figuring out when implicit conversions are genuinely useful and should be allowed and when they're more trouble than they're worth is surprisingly hard to get right. :( - Jonathan M Davis
May 09 2016
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 9 May 2016 02:10:19 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 Don Clugston pointed out in his DConf 2016 talk that:
 
      float f = 1.30;
      assert(f == 1.30);
 
 will always be false since 1.30 is not representable as a float. However,
 
      float f = 1.30;
      assert(f == cast(float)1.30);
 
 will be true.
 
 So, should the compiler emit a warning for the former case?
I'd say yes, but exclude the case where it can be statically verified, that the comparison can yield true, because the constant can be losslessly converted to the type of 'f'. By example, don't warn for these: f == 1.0, f == -0.5, f == 3.625, f == 2UL^^60 But do warn for: f == 1.30, f == 2UL^^60+1 As an extension of the existing "comparison is always false/true" check it could read "Comparison is always false: literal 1.30 is not representable as 'float'". There is a whole bunch in this warning category: byte b; if (b == 1000) {} "Comparison is always false: literal 1000 is not representable as 'byte'" -- Marco
May 09 2016
next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10 May 2016 at 06:25, Marco Leise via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Am Mon, 9 May 2016 02:10:19 -0700
 schrieb Walter Bright <newshound2 digitalmars.com>:

 Don Clugston pointed out in his DConf 2016 talk that:

      float f = 1.30;
      assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

      float f = 1.30;
      assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
I'd say yes, but exclude the case where it can be statically verified, that the comparison can yield true, because the constant can be losslessly converted to the type of 'f'. By example, don't warn for these: f == 1.0, f == -0.5, f == 3.625, f == 2UL^^60 But do warn for: f == 1.30, f == 2UL^^60+1 As an extension of the existing "comparison is always false/true" check it could read "Comparison is always false: literal 1.30 is not representable as 'float'". There is a whole bunch in this warning category: byte b; if (b == 1000) {} "Comparison is always false: literal 1000 is not representable as 'byte'" -- Marco
This.
May 10 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/9/2016 1:25 PM, Marco Leise wrote:
 There is a whole bunch in this warning category:
   byte b;
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"
You're right, we may be opening a can of worms with this.
May 10 2016
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-05-10 23:44, Walter Bright wrote:
 On 5/9/2016 1:25 PM, Marco Leise wrote:
 There is a whole bunch in this warning category:
   byte b;
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"
You're right, we may be opening a can of worms with this.
Scala gives a warning/error (don't remember which) for "isInstanceOf" where it can prove at compile time it will never be true. That has helped me a couple of times. -- /Jacob Carlborg
May 10 2016
prev sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 10 May 2016 at 21:44:45 UTC, Walter Bright wrote:
   if (b == 1000) {}
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"
What's wrong with having this warning?
 You're right, we may be opening a can of worms with this.
May 12 2016
parent =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 12 May 2016 at 09:22:02 UTC, Nordlöw wrote:
 "Comparison is always false: literal 1000 is not representable
 as 'byte'"
What's wrong with having this warning?
This is, IMO, just a more informative diagnostic than: "Statement at `foo()` is not reachable": in the following code: if (b == 1000) { foo(); }
May 12 2016
prev sibling next sibling parent Joe Duarte <jose.duarte asu.edu> writes:
On Monday, 9 May 2016 at 09:10:19 UTC, Walter Bright wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a 
 float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
I think it really depends on what the warning actually says. I think people have different expectations for what that warning would be. When you say 1.30 is not representable as a float, when is the "not representable" enforced? Because it looks like the programmer just represented it in the assignment of the literal – but that's not where the warning would be right? I mean I assume so because people need nonrational literals all the time, and this is the only way they can do it, which means it's a hole in the type system right? There should be a decimal type to cover all these cases, like some databases have. Would the warning say that you can't compare 1.30 to a float because 1.30 is not representable as a float? Or would it say that f was rounded upon assignment and is no longer 1.30? Short of a decimal type, I think it would be nice to have a "float equality" operator that covered this whole class of cases, where floats that started their lives as nonrational literals and floats that have been rounded with loss of precision can be treated as equal if they're within something like .0000001% of each other (well a percentage that can actually be represented as a float...) Basically equality that covers the known mutational properties of fp arithmetic. There's no way to do this right now without ranges right? I know that ~ is for concat. I saw ~= is an operator. What does that do? The Unicode ≈ would be nice for this. I assume IEEE 754 or ISO 10967 don't cover this? I was just reading the latter (zip here: http://standards.iso.org/ittf/PubliclyAvailableStandards/c051317_ISO_IEC_10967-1_2012.zip)
May 09 2016
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9 May 2016 at 19:10, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
Perhaps float comparison should *always* be done at the lower precision? There's no meaningful way to perform a float/double comparison where the float is promoted, whereas demoting the double for the comparison will almost certainly yield the expected result.
May 10 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 10 May 2016 at 07:28:21 UTC, Manu wrote:
 Perhaps float comparison should *always* be done at the lower 
 precision? There's no meaningful way to perform a float/double 
 comparison where the float is promoted, whereas demoting the 
 double for the comparison will almost certainly yield the 
 expected result.
Assuming that's what you want, it's reasonably straightforward to use feqrel(someFloat, someDouble) >= float.mant_dig ... to compare to the level of precision that matters to you. That's probably a better option than adjusting `==` to always prefer a lower level of precision (because it's arguably accurate to say that 1.3f != 1.3).
May 16 2016
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 10 May 2016 at 17:28, Manu <turkeyman gmail.com> wrote:
 On 9 May 2016 at 19:10, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Don Clugston pointed out in his DConf 2016 talk that:

     float f = 1.30;
     assert(f == 1.30);

 will always be false since 1.30 is not representable as a float. However,

     float f = 1.30;
     assert(f == cast(float)1.30);

 will be true.

 So, should the compiler emit a warning for the former case?
Perhaps float comparison should *always* be done at the lower precision? There's no meaningful way to perform a float/double comparison where the float is promoted, whereas demoting the double for the comparison will almost certainly yield the expected result.
Think of it like this; a float doesn't represent a precise point (it's an approximation by definition), so see the float as representing the interval from the absolute value it stores, and that + 1 mantissa bit. If you see float's that way, then the natural way to compare them is to demote to the lowest common precision, and it wouldn't be considered erroneous, or even warning-worthy; just documented behaviour.
May 10 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/10/2016 12:31 AM, Manu via Digitalmars-d wrote:
 Think of it like this; a float doesn't represent a precise point (it's
 an approximation by definition), so see the float as representing the
 interval from the absolute value it stores, and that + 1 mantissa bit.
 If you see float's that way, then the natural way to compare them is
 to demote to the lowest common precision, and it wouldn't be
 considered erroneous, or even warning-worthy; just documented
 behaviour.
Floating point behavior is so commonplace, I am wary of inventing new, unusual semantics for it.
May 10 2016
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 May 2016 at 07:47, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/10/2016 12:31 AM, Manu via Digitalmars-d wrote:
 Think of it like this; a float doesn't represent a precise point (it's
 an approximation by definition), so see the float as representing the
 interval from the absolute value it stores, and that + 1 mantissa bit.
 If you see float's that way, then the natural way to compare them is
 to demote to the lowest common precision, and it wouldn't be
 considered erroneous, or even warning-worthy; just documented
 behaviour.
Floating point behavior is so commonplace, I am wary of inventing new, unusual semantics for it.
Is it unusual to demote to the lower common precision? I think it's the only reasonable solution. It's never reasonable to promote a float, since it has already suffered precision loss. It can't meaningfully be compared against anything higher precision than itself. What is the problem with this behaviour I suggest? The reason I'm wary about emitting a warning is because people will encounter the warning *all the time*, and for a user who doesn't have comprehensive understanding of floating point (and probably many that do), the natural/intuitive thing to do would be to place an explicit cast of the lower precision value to the higher precision type, which is __exactly the wrong thing to do__. I don't think the warning improves the problem, it likely just causes people to emit the same incorrect code explicitly. Honestly, who would naturally respond to such a warning by demoting the higher precision type? I don't know that guy, other than those of us who have just watched Don's talk.
May 11 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:
 Floating point behavior is so commonplace, I am wary of inventing new,
 unusual semantics for it.
Is it unusual to demote to the lower common precision?
Yes.
 I think it's the only reasonable solution.
It may be, but it is unusual and therefore surprising behavior.
 What is the problem with this behaviour I suggest?
Code will do one thing in C, and the same code will do something unexpectedly different in D.
 The reason I'm wary about emitting a warning is because people will
 encounter the warning *all the time*, and for a user who doesn't have
 comprehensive understanding of floating point (and probably many that
 do), the natural/intuitive thing to do would be to place an explicit
 cast of the lower precision value to the higher precision type, which
 is __exactly the wrong thing to do__.
 I don't think the warning improves the problem, it likely just causes
 people to emit the same incorrect code explicitly.
The warning is intended for people who understand, as then they will figure out what they actually wanted and implement that. People who randomly and without comprehension insert casts hoping to make the compiler shut up cannot be helped.
May 12 2016
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 12 May 2016 at 17:32, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:

 The reason I'm wary about emitting a warning is because people will
 encounter the warning *all the time*, and for a user who doesn't have
 comprehensive understanding of floating point (and probably many that
 do), the natural/intuitive thing to do would be to place an explicit
 cast of the lower precision value to the higher precision type, which
 is __exactly the wrong thing to do__.
 I don't think the warning improves the problem, it likely just causes
 people to emit the same incorrect code explicitly.
The warning is intended for people who understand, as then they will figure out what they actually wanted and implement that. People who randomly and without comprehension insert casts hoping to make the compiler shut up cannot be helped.
But they can easily be helped by implementing behaviour that makes sense. If you're set on a warning, at least make the warning recommend down-casting the higher precision term to the lower precision?
May 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 3:30 AM, Manu via Digitalmars-d wrote:
 But they can easily be helped by implementing behaviour that makes sense.
One thing that's immutably true about computer floating point is that it does not make intuitive sense if your intuition is based on mathematics. It's a hopeless cause trying to bash it into 'making sense'. It's something one has to simply take the time and learn. This reminds me of all the discussions around trying to hide the fact that D strings are UTF-8 code units. The ultimate outcome of trying to make it "make sense" was the utter disaster of autodecoding.
 If you're set on a warning, at least make the warning recommend
 down-casting the higher precision term to the lower precision?
Yes, of course. I believe error messages should suggest corrective action.
May 12 2016
next sibling parent reply Guillaume Chatelet <chatelet.guillaume gmail.com> writes:
On Thursday, 12 May 2016 at 15:55:52 UTC, Walter Bright wrote:
 This reminds me of all the discussions around trying to hide 
 the fact that D strings are UTF-8 code units. The ultimate 
 outcome of trying to make it "make sense" was the utter 
 disaster of autodecoding.
Well maybe it was a disaster because the problem was only half solved. It looks like Perl 6 got it right: https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/
May 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 9:15 AM, Guillaume Chatelet wrote:
 Well maybe it was a disaster because the problem was only half solved.
 It looks like Perl 6 got it right:
 https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/
Perl isn't a systems programming language. A systems language requires access to code units, invalid encodings, etc. Nor is Perl efficient. There are a lot of major efficiency gains by not autodecoding.
May 12 2016
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, May 12, 2016 at 09:20:05AM -0700, Walter Bright via Digitalmars-d wrote:
[...]
 There are a lot of major efficiency gains by not autodecoding.
Any chance of killing autodecoding in Phobos in the foreseeable future? It's one of the things that I really liked about D in the beginning, but over time, I've come to realize more and more that it was a mistake. It's one of those things that only becomes clear in retrospect. T -- Ph.D. = Permanent head Damage
May 12 2016
prev sibling parent reply Guillaume Chatelet <chatelet.guillaume gmail.com> writes:
On Thursday, 12 May 2016 at 16:20:05 UTC, Walter Bright wrote:
 On 5/12/2016 9:15 AM, Guillaume Chatelet wrote:
 Well maybe it was a disaster because the problem was only half 
 solved.
 It looks like Perl 6 got it right:
 https://perl6advent.wordpress.com/2015/12/07/day-7-unicode-perl-6-and-you/
Perl isn't a systems programming language. A systems language requires access to code units, invalid encodings, etc. Nor is Perl efficient. There are a lot of major efficiency gains by not autodecoding.
[Sorry for the OT] I never claimed Perl was a systems programming language nor that it was efficient, just that their design looks more mature than ours. Also I think you missed this part of the article: "Of course, that’s all just for the default Str type. If you don’t want to work at a grapheme level, then you have several other string types to choose from: If you’re interested in working within a particular normalization, there’s the self-explanatory types of NFC, NFD, NFKC, and NFKD. If you just want to work with codepoints and not bother with normalization, there’s the Uni string type (which may be most appropriate in cases where you don’t want the NFC normalization that comes with normal Str, and keep text as-is). And if you want to work at the binary level, well, there’s always the Blob family of types :)." We basically have "Uni" in D, no normalized nor grapheme level.
May 12 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 9:36 AM, Guillaume Chatelet wrote:
 just that their design looks more mature than ours.
I don't think that can be inferred from a brief article. If you want to access D strings by various means, there's .byChar, .byWchar, .byDchar, .byCodeunit, etc. foreach can also pick off characters by various schemes.
May 12 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/12/16 6:55 PM, Walter Bright wrote:
 This reminds me of all the discussions around trying to hide the fact
 that D strings are UTF-8 code units. The ultimate outcome of trying to
 make it "make sense" was the utter disaster of autodecoding.
I am as unclear about the problems of autodecoding as I am about the necessity to remove curl. Whenever I ask I hear some arguments that work well emotionally but are scant on reason and engineering. Maybe it's time to rehash them? I just did so about curl, no solid argument seemed to come together. I'd be curious of a crisp list of grievances about autodecoding. -- Andrei
May 12 2016
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 05/12/2016 06:29 PM, Andrei Alexandrescu wrote:
 I'd be curious of a crisp list of grievances about
 autodecoding. -- Andrei
It emits code points (dchar) which is an awkward middle point between code units (char/wchar) and graphemes. Without any auto-decoding at all, every array T[] would be a random-access range of Ts as well. `.front` would be the same as `[0]`, `.length` would be the same as `.walkLength`, etc. That would make things less confusing for newbies, and more experienced programmers wouldn't accidentally mix the two abstraction levels. Of course, you'd have to be aware that a (w)char is not a character as perceived by humans, but a code unit. But auto-decoding to code points only shifts that problem: You have to be aware that a dchar is not a character either. Multiple dchars may form one visible character, one grapheme. For example, "\u00E4" and "a\u0308" encode the same grapheme: "ä". If char[], wchar[], dchar[] (and qualified variants) were ranges of graphemes, things would make the most sense for people who are not aware of delicate details of Unicode. You wouldn't accidentally cut code points or graphemes in half, `.walkLength` makes intuitive sense, etc. You could still accidentally use `.length` or `[0]`, though. So it still has some pitfalls.
May 12 2016
parent ag0aep6g <anonymous example.com> writes:
On 05/12/2016 07:12 PM, ag0aep6g wrote:
[...]

tl;dr:

This is surprising to newbies, and a possible source of bugs for 
experienced programmers:

----
writeln("ä".length); /* "2" */
writeln("ä"[0]); /* question mark in a diamond */
----

When people understand the above, they might still not understand this:

----
writeln("length of 'a\u0308': ", "a\u0308".walkLength); /* "length of 
'ä': 2" */
writeln("a\u0308".front); /* "a" */
----
May 12 2016
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/12/16 12:29 PM, Andrei Alexandrescu wrote:
 On 5/12/16 6:55 PM, Walter Bright wrote:
 This reminds me of all the discussions around trying to hide the fact
 that D strings are UTF-8 code units. The ultimate outcome of trying to
 make it "make sense" was the utter disaster of autodecoding.
I am as unclear about the problems of autodecoding as I am about the necessity to remove curl. Whenever I ask I hear some arguments that work well emotionally but are scant on reason and engineering. Maybe it's time to rehash them? I just did so about curl, no solid argument seemed to come together. I'd be curious of a crisp list of grievances about autodecoding. -- Andrei
Autodecoding, IMO, is not the problem. The problem is hijacking an array to mean something other than an array. I ran into this the other day. In iopipe, I treat char[] buffers as actual buffers of code units. This makes sense, as I'm reading/writing text to buffers, and care very little about decoding. I wanted to test my system's ability to handle random-access ranges, and I'm using isRandomAccessRange || isNarrowString to get around this. As soon as I do chain(a, b) where a and b are narrow strings, this doesn't work, and I can't get it back. See my exception in the unit test: https://github.com/schveiguy/iopipe/blob/master/source/iopipe/traits.d#L91 If you want to avoid auto-decoding, you have to be very cautious about using Phobos features. -Steve
May 12 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 9:29 AM, Andrei Alexandrescu wrote:
 I am as unclear about the problems of autodecoding
Started a new thread on that.
May 12 2016
prev sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 12 May 2016 08:55:52 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 5/12/2016 3:30 AM, Manu via Digitalmars-d wrote:
 If you're set on a warning, at least make the warning recommend
 down-casting the higher precision term to the lower precision?  
Yes, of course. I believe error messages should suggest corrective action.
Because of afore mentioned difference between cast(float)1.30 and 1.30f, the correct action for the original case is to suffix the literal with 'f'. That gives you the correct number to compare to. -- Marco
May 12 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/12/16 3:32 AM, Walter Bright wrote:
 On 5/11/2016 2:24 AM, Manu via Digitalmars-d wrote:
 What is the problem with this behaviour I suggest?
Code will do one thing in C, and the same code will do something unexpectedly different in D.
Not taking one side or another on this, but due to D doing everything with reals, this is already the case. -Steve
May 12 2016
next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Thursday, 12 May 2016 at 13:03:58 UTC, Steven Schveighoffer 
wrote:
 Not taking one side or another on this, but due to D doing 
 everything with reals, this is already the case.
Mmmm. I don't want to open up another can of worms right now, but our x64 C++ code only emits SSE instructions at compile time (or AVX on the Xbox One). The only thing that attempts to use reals in our codebase is our D code.
May 12 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/12/16 10:05 AM, Ethan Watson wrote:
 On Thursday, 12 May 2016 at 13:03:58 UTC, Steven Schveighoffer wrote:
 Not taking one side or another on this, but due to D doing everything
 with reals, this is already the case.
Mmmm. I don't want to open up another can of worms right now, but our x64 C++ code only emits SSE instructions at compile time (or AVX on the Xbox One). The only thing that attempts to use reals in our codebase is our D code.
There was a question on the forums a while back about equivalent C++ code that didn't work in D. The answer turned out to be, you had to shoehorn everything into doubles in order to get the same answer. -Steve
May 12 2016
parent Ethan Watson <gooberman gmail.com> writes:
On Thursday, 12 May 2016 at 14:29:01 UTC, Steven Schveighoffer 
wrote:
 There was a question on the forums a while back about 
 equivalent C++ code that didn't work in D. The answer turned 
 out to be, you had to shoehorn everything into doubles in order 
 to get the same answer.
I can certainly see that being the case, especially when dealing with SSE-based code. floats and doubles in XMM registers don't get calculated at 80-bit precision internally, their storage size dictates their calculation precision. Which has led MSVC to promoting floats to doubles for CRT functions when it thinks it can get away with it (and one instance where the compiler forgot to convert back to float afterwards and thus the lower 32 bits of a double were being treated as a float...) It's fun comparing assembly too. There's one particular function we have here that collapsed to something like 20-30 lines of SSE-based code (half after I hand optimised it with branchless SSE intrinsics and without a call to fmod). The same function in D resulted in a significantly larger amount of x87 code. I don't miss x87 at all. But this is getting OT.
May 12 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 6:03 AM, Steven Schveighoffer wrote:
 Not taking one side or another on this, but due to D doing everything with
 reals, this is already the case.
Actually, C allows that behavior if I understood the spec correctly. D just makes things more explicit.
May 12 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/12/16 12:00 PM, Walter Bright wrote:
 On 5/12/2016 6:03 AM, Steven Schveighoffer wrote:
 Not taking one side or another on this, but due to D doing everything
 with
 reals, this is already the case.
Actually, C allows that behavior if I understood the spec correctly. D just makes things more explicit.
This is the thread I was thinking about: https://forum.dlang.org/post/ugxmeiqsbitxxzoyakko forum.dlang.org Essentially, copy-pasted code from C results in different behavior in D because of the different floating point decisions made by the compilers. Your quote was: "[The problem is] Code will do one thing in C, and the same code will do something unexpectedly different in D." My response is that it already happens, so it may not be a convincing argument. I don't know the requirements or allowances of the C spec. All I know is the testable reality of the behavior on the same platform. -Steve
May 12 2016
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Thu, 12 May 2016 09:03:58 -0400
schrieb Steven Schveighoffer <schveiguy yahoo.com>:

 Not taking one side or another on this, but due to D doing everything 
 with reals, this is already the case.
 
 -Steve
As far as I have understood the situation: - FPU instructions are inaccurate - FPU is typically set to highest precision (80-bit) to give accurate float/double results - SSE instructions yield accurately rounded results - Need for 80 bits greatly reduced with SSE - FPU deprectated on 64-bit x86 - Results of FPU and SSE math differ - Compiler writers discordant GCC: compiler switch for SSE or FPU, SSE default DMD: FPU only - Unless CTFE uses soft-float implementation, depending on compiler and flags used to compile a D compiler, resulting executable produces different CTFE floating-point results -- Marco
May 12 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 4:32 PM, Marco Leise wrote:
 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results
I've actually been thinking of writing a 128 bit float emulator, and then using that in the compiler internals to do all FP computation with. It's not a panacea, as it won't change how things work in the back ends, nor will it change what happens at runtime, but it means the front end will give portable, consistent results.
May 12 2016
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:
 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.

 It's not a panacea, as it won't change how things work in the 
 back ends, nor will it change what happens at runtime, but it 
 means the front end will give portable, consistent results.
And be 20x slower than hardware floats. Is it really worth it?
May 12 2016
next sibling parent xenon325 <anm programmer.net> writes:
On Friday, 13 May 2016 at 03:18:05 UTC, Jack Stouffer wrote:
 On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:
 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.
 [...]
And be 20x slower than hardware floats. Is it really worth it?
Emulator is meant for computation during compilation only, so CTFE results are consistent across different compilers and compiler host hardware (IIUC). -- Alexander
May 12 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 8:18 PM, Jack Stouffer wrote:
 And be 20x slower than hardware floats. Is it really worth it?
I seriously doubt the slowdown would be measurable, as the number of float ops the compiler performs is insignificant.
May 13 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 13 May 2016 at 03:18:05 UTC, Jack Stouffer wrote:
 On Friday, 13 May 2016 at 01:03:57 UTC, Walter Bright wrote:
 I've actually been thinking of writing a 128 bit float 
 emulator, and then using that in the compiler internals to do 
 all FP computation with.

 It's not a panacea, as it won't change how things work in the 
 back ends, nor will it change what happens at runtime, but it 
 means the front end will give portable, consistent results.
And be 20x slower than hardware floats. Is it really worth it?
Yes. What is the proportion of CTFE floating point operation right know in the total compile time ? I'd be highly supersized if it was anything close to 1% .
May 15 2016
parent Max Samukha <maxsamukha gmail.com> writes:
On Sunday, 15 May 2016 at 12:11:58 UTC, deadalnix wrote:

 Yes. What is the proportion of CTFE floating point operation 
 right know in the total compile time ? I'd be highly supersized 
 if it was anything close to 1% .
That was said about CTFE in general before people started to use it and discovered it is unusable because of performance issues (vibe.d is a prominent example).
May 15 2016
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/12/2016 4:32 PM, Marco Leise wrote:
 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results
I've actually been thinking of writing a 128 bit float emulator, and then using that in the compiler internals to do all FP computation with.
No. Do not. I've worked on systems where the compiler and the runtime don't share floating point precisions before, and it was a nightmare. One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit iirc?) float precision, code compiled for it used 32bits in the compiler... to make it worse, the CPU also ran 32bits. The result was, literals/constants, or float data fed from the CPU didn't match data calculated by the vector unit at runtime (ie, runtime computation of the same calculation that may have occurred at compile time to produce some constant didn't match). The result was severe cracking and visible/shimmering seams between triangles as sub-pixel alignment broke down. We struggled with this for years. It was practically impossible to solve, and mostly involved workarounds. I really just want D to use double throughout, like all the cpu's that run code today. This 80bit real thing (only on x86 cpu's though!) is a never ending pain.
 It's not a panacea, as it won't change how things work in the back ends, nor
 will it change what happens at runtime, but it means the front end will give
 portable, consistent results.
This sounds like designing specifically for my problem from above, where the frontend is always different than the backend/runtime. Please have the frontend behave such that it operates on the precise datatype expressed by the type... the backend probably does this too, and runtime certainly does; they all match.
May 12 2016
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 13 May 2016 at 05:12:14 UTC, Manu wrote:
 No. Do not.
 I've worked on systems where the compiler and the runtime don't 
 share
 floating point precisions before, and it was a nightmare.
Use reproducible cross platform IEEE754-2008 and use exact rational numbers. All other representations are just painful. Nothing wrong with supporting 16, 32, 64 and 128 bit, but stick to the reproducible standard. If people want "non-reproducible fast math", then they should specify it.
May 13 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/12/2016 10:12 PM, Manu via Digitalmars-d wrote:
 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.
 One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit
 iirc?) float precision, code compiled for it used 32bits in the
 compiler... to make it worse, the CPU also ran 32bits. The result was,
 literals/constants, or float data fed from the CPU didn't match data
 calculated by the vector unit at runtime (ie, runtime computation of
 the same calculation that may have occurred at compile time to produce
 some constant didn't match). The result was severe cracking and
 visible/shimmering seams between triangles as sub-pixel alignment
 broke down.
 We struggled with this for years. It was practically impossible to
 solve, and mostly involved workarounds.
I understand there are some cases where this is needed, I've proposed intrinsics for that.
 I really just want D to use double throughout, like all the cpu's that
 run code today. This 80bit real thing (only on x86 cpu's though!) is a
 never ending pain.
It's 128 bits on other CPUs.
 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.
Except this never happens anyway.
May 13 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 13 May 2016 at 18:16:29 UTC, Walter Bright wrote:
 Please have the frontend behave such that it operates on the 
 precise
 datatype expressed by the type... the backend probably does 
 this too,
 and runtime certainly does; they all match.
Except this never happens anyway.
It should in C++ with the right strict-settings, which makes the compiler use reproducible floating point operations. AFAIK it should work out even in modern JavaScript.
May 13 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:
 It should in C++ with the right strict-settings,
Consider what the C++ Standard says, not what the endless switches to tweak the compiler do.
May 13 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Friday, 13 May 2016 at 21:36:52 UTC, Walter Bright wrote:
 On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:
 It should in C++ with the right strict-settings,
Consider what the C++ Standard says, not what the endless switches to tweak the compiler do.
The C++ standard cannot even require IEEE754. Nobody relies only on what the C++ standard says in real projects. They rely on what the chosen compiler(s) on concrete platform(s) do.
May 13 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/13/2016 2:42 PM, Ola Fosheim Grøstad wrote:
 On Friday, 13 May 2016 at 21:36:52 UTC, Walter Bright wrote:
 On 5/13/2016 1:57 PM, Ola Fosheim Grøstad wrote:
 It should in C++ with the right strict-settings,
Consider what the C++ Standard says, not what the endless switches to tweak the compiler do.
The C++ standard cannot even require IEEE754. Nobody relies only on what the C++ standard says in real projects. They rely on what the chosen compiler(s) on concrete platform(s) do.
Nevertheless, C++ is what the Standard says it is. If Brand X compiler does something else, you should call it "Brand X C++".
May 13 2016
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 14 May 2016 at 04:16, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/12/2016 10:12 PM, Manu via Digitalmars-d wrote:
 No. Do not.
 I've worked on systems where the compiler and the runtime don't share
 floating point precisions before, and it was a nightmare.
 One anecdote, the PS2 had a vector coprocessor; it ran reduced (24bit
 iirc?) float precision, code compiled for it used 32bits in the
 compiler... to make it worse, the CPU also ran 32bits. The result was,
 literals/constants, or float data fed from the CPU didn't match data
 calculated by the vector unit at runtime (ie, runtime computation of
 the same calculation that may have occurred at compile time to produce
 some constant didn't match). The result was severe cracking and
 visible/shimmering seams between triangles as sub-pixel alignment
 broke down.
 We struggled with this for years. It was practically impossible to
 solve, and mostly involved workarounds.
I understand there are some cases where this is needed, I've proposed intrinsics for that.
Intrinsics for... what? Making the compiler use the type specified at compile time? Is it true that that's not happening already? I really don't want to use an intrinsic to have float behave like a float at CTFE... nobody will EVER do that.
 I really just want D to use double throughout, like all the cpu's that
 run code today. This 80bit real thing (only on x86 cpu's though!) is a
 never ending pain.
It's 128 bits on other CPUs.
What?
 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.
Except this never happens anyway.
Huh? I'm sorry, I didn't follow those points.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 7:04 PM, Manu via Digitalmars-d wrote:
 I understand there are some cases where this is needed, I've proposed
 intrinsics for that.
Intrinsics for... what?
float roundToFloat(float f);
 I really don't want to use an intrinsic to have float behave like a
 float at CTFE... nobody will EVER do that.
Floats aren't required to have float precision by the C or C++ Standards. I quoted it for Ola :-)
 It's 128 bits on other CPUs.
What?
Some CPUs have 128 bit floats.
 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.
Except this never happens anyway.
Huh? I'm sorry, I didn't follow those points.
The belief that compile time and runtime are exactly the same floating point in C/C++ is false.
May 15 2016
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 12:56, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 7:04 PM, Manu via Digitalmars-d wrote:
 I understand there are some cases where this is needed, I've proposed
 intrinsics for that.
Intrinsics for... what?
float roundToFloat(float f);
 I really don't want to use an intrinsic to have float behave like a
 float at CTFE... nobody will EVER do that.
Floats aren't required to have float precision by the C or C++ Standards. I quoted it for Ola :-)
 It's 128 bits on other CPUs.
What?
Some CPUs have 128 bit floats.
Yes, but you don't accidentally use 128bit floats, you type: extended x = 1.3; x + y; If that code were to CTFE, I expect the CTFE to use extended precision. My point is, CTFE should surely follow the types and language semantics as if it were runtime generated code... It's not reasonable that CTFE has higher precision applied than the same code at runtime. CTFE must give the exact same result as runtime execution of the function.
 This sounds like designing specifically for my problem from above,
 where the frontend is always different than the backend/runtime.
 Please have the frontend behave such that it operates on the precise
 datatype expressed by the type... the backend probably does this too,
 and runtime certainly does; they all match.
Except this never happens anyway.
Huh? I'm sorry, I didn't follow those points.
The belief that compile time and runtime are exactly the same floating point in C/C++ is false.
I'm not interested in C/C++, I gave some anecdotes where it's gone wrong for me too, but regardless; generally, they do match, and I can't think of a single modern example where that's not true. If you *select* fast-math, then you may generate code that doesn't match, but that's a deliberate selection. If I want 'real' math (in CTFE or otherwise), I will type "real". It is completely unreasonable to reinterpret the type that the user specified. CTFE should execute code the same way runtime would execute the code (without -ffast-math, and conformant ieee hardware). This is not a big ask. Incidentally, I made the mistake of mentioning this thread (due to my astonishment that CTFE ignores float types) out loud to my colleagues... and they actually started yelling violently out loud. One of them has now been on a 10 minute angry rant with elevated tone and waving his arms around about how he's been shafted by this sort behaviour so many times before. I wish I recorded it, I'd love to have submit it as evidence.
May 15 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 9:02 PM, Manu via Digitalmars-d wrote:
 Yes, but you don't accidentally use 128bit floats, you type:

 extended x = 1.3;
 x + y;
The C/C++ standards allow constant folding at 128 bits, despite floats being 32 bits.
 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.
It hasn't for decades on x86 machines, and the world hasn't collapsed, in fact, few people ever even notice. Not many people prefer less accurate answers. The initial Java spec worked as you desired, and they were pretty much forced to back off of it.
 The belief that compile time and runtime are exactly the same floating point
 in C/C++ is false.
I'm not interested in C/C++, I gave some anecdotes where it's gone wrong for me too, but regardless; generally, they do match, and I can't think of a single modern example where that's not true. If you *select* fast-math, then you may generate code that doesn't match, but that's a deliberate selection.
They won't match on any code that uses the x87. The standard doesn't require float math to use float instructions, they can (and do) use double instructions for temporaries.
 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)
Float types are not selected because they are less accurate, they are selected because they are smaller and faster.
 out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.
I'm interested to hear how he was "shafted" by this. This apparently also contradicts the claim that other languages do as you ask.
May 15 2016
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 14:26, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 9:02 PM, Manu via Digitalmars-d wrote:
 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.
It hasn't for decades on x86 machines, and the world hasn't collapsed, in fact, few people ever even notice. Not many people prefer less accurate answers.
That doesn't mean it's not wrong. Don noticed, he gave a lecture on floating-point gotchas. I'm still firmly engaged in trying to use D professionally... should I ever successfully pass that barrier, then it's just a matter of time until such a piece of code that tend to challenge these things is written. There are not enough uses of D that we can offer anecdotes, but we can offer anecdotes from our decades with C++ that we would desperately like to avoid in the future. If we don't care about doing what's right, then we just accept that float remains a highly expert/special-knowledge-centric field, and talks like the one Don gave should be taken as thought-provoking, however of no practical relevance for the language, since it's 'fine for most people, most of the time', and we get on with other things. This thread is evidence that people would like to do the best we can. 1.3f != 1.3 is not accurate, it's wrong.
 The initial Java spec worked as you desired, and they were pretty much
 forced to back off of it.
Ok, why's that?
 The belief that compile time and runtime are exactly the same floating
 point
 in C/C++ is false.
I'm not interested in C/C++, I gave some anecdotes where it's gone wrong for me too, but regardless; generally, they do match, and I can't think of a single modern example where that's not true. If you *select* fast-math, then you may generate code that doesn't match, but that's a deliberate selection.
They won't match on any code that uses the x87. The standard doesn't require float math to use float instructions, they can (and do) use double instructions for temporaries.
If it does, then it is careful to make sure the precision expectations are maintained. If you don't '-ffast-math', the FPU code produces a IEEE conformant result on reasonable compilers. We depend on this.
 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)
Float types are not selected because they are less accurate, they are selected because they are smaller and faster.
They are selected because they are smaller and faster with the understood trade-off that they are less accurate. They are certainly selected with the _intent_ that they are less accurate. It's not reasonable that a CTFE function may produce a radically different result than the same function at runtime.
 out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.
I'm interested to hear how he was "shafted" by this. This apparently also contradicts the claim that other languages do as you ask.
I've explained prior the cases where this has happened are most often invoked by the hardware having a reduced runtime precision than the compiler. The only cases I know of where this has happened due to the compiler internally is CodeWarrior; an old/dead C++ compiler that always sucked and caused us headaches of all kinds. The point is, the CTFE behaviour is essentially identical to our classic case where the hardware runs a different precision than the compiler, and that's built into the language! It's not just an anomaly expressed by one particular awkward platform we're required to support.
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 10:13 PM, Manu via Digitalmars-d wrote:
 1.3f != 1.3 is not accurate, it's wrong.
I'm sorry, there is no way to make FP behave like mathematics. It's its own beast, with its own rules.
 The initial Java spec worked as you desired, and they were pretty much
 forced to back off of it.
Ok, why's that?
Because forcing the x87 to work at reduced precision caused a 2x slowdown or something like that, making Java uncompetitive (i.e. unusable) for numerics work.
 They won't match on any code that uses the x87. The standard doesn't require
 float math to use float instructions, they can (and do) use double
 instructions for temporaries.
If it does, then it is careful to make sure the precision expectations are maintained.
Have you tested this?
 If you don't '-ffast-math', the FPU code produces a
 IEEE conformant result on reasonable compilers.
Googling 'fp:fast' yields: "Creates the fastest code in most cases by relaxing the rules for optimizing floating-point operations. This enables the compiler to optimize floating-point code for speed at the expense of accuracy and correctness. When /fp:fast is specified, the compiler may not round correctly at assignment statements, typecasts, or function calls, and may not perform rounding of intermediate expressions. The compiler may reorder operations or perform algebraic transforms—for example, by following associative and distributive rules—without regard to the effect on finite precision results. The compiler may change operations and operands to single-precision instead of following the C++ type promotion rules. Floating-point-specific contraction optimizations are always enabled (fp_contract is ON). Floating-point exceptions and FPU environment access are disabled (/fp:except- is implied and fenv_access is OFF)." This doesn't line up with what you said it does?
 We depend on this.
I googled 'fp:precise', which is the VC++ default, and found this: "Using /fp:precise when fenv_access is ON disables optimizations such as compile-time evaluations of floating-point expressions." How about that? No CTFE! Is that really what you wanted? :-)
 They are certainly selected with the _intent_ that they are less accurate.
This astonishes me. What algorithm requires less accuracy?
 It's not reasonable that a CTFE function may produce a radically
 different result than the same function at runtime.
Yeah, since the less accurate version can suffer from a phenomenon called "total loss of precision" where accumulated roundoff errors make the value utter garbage. When is this desirable?
 I'm interested to hear how he was "shafted" by this. This apparently also
 contradicts the claim that other languages do as you ask.
I've explained prior the cases where this has happened are most often invoked by the hardware having a reduced runtime precision than the compiler. The only cases I know of where this has happened due to the compiler internally is CodeWarrior; an old/dead C++ compiler that always sucked and caused us headaches of all kinds. The point is, the CTFE behaviour is essentially identical to our classic case where the hardware runs a different precision than the compiler, and that's built into the language! It's not just an anomaly expressed by one particular awkward platform we're required to support.
You mentioned there was a "shimmer" effect. With the extremely limited ability of C++ compilers to fold constants, I'm left wondering how your code suffered from this, and why you would calculate the same value at both compile and run time.
May 15 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 06:26, Walter Bright wrote:
 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types)
Float types are not selected because they are less accurate,
(AFAIK, accuracy is a property of a value given some additional context. Types have precision.)
 they are selected because they are smaller and faster.
Right. Hence, the 80-bit CTFE results have to be converted to the final precision at some point in order to commence the runtime computation. This means that additional rounding happens, which was not present in the original program. The additional total roundoff error this introduces can exceed the roundoff error you would have suffered by using the lower precision in the first place, sometimes completely defeating precision-enhancing improvements to an algorithm. This might be counter-intuitive, but this is floating point. The precision should just stay the specified one during the entire computation (even if part of it is evaluated at compile-time). The claim here is /not/ that lower precision throughout delivers more accurate results. The additional rounding is the problem. There are other reasons why I think that this kind of implementation-defined behaviour is a terribly bad idea, eg.: - it breaks common assumptions about code, especially how it behaves under seemingly innocuous refactorings, or with a different set of compiler flags. - it breaks reproducibility, which is sometimes more important that being close to the infinite precision result (which you cannot guarantee with any finite floating point type anyway). (E.g. in a game, it is enough if the result seems plausible, but it should be the same for everyone. For some scientific experiments, the ideal case is to have 100% reproducibility of the computation, even if it is horribly wrong, such that other scientists can easily uncover and diagnose the problem, for example.)
May 17 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 17 May 2016 at 18:08:47 UTC, Timon Gehr wrote:
 Right. Hence, the 80-bit CTFE results have to be converted to 
 the final precision at some point in order to commence the 
 runtime computation. This means that additional rounding 
 happens, which was not present in the original program. The 
 additional total roundoff error this introduces can exceed the 
 roundoff error you would have suffered by using the lower 
 precision in the first place, sometimes completely defeating 
 precision-enhancing improvements to an algorithm.
WAT ? Is that really possible ?
May 17 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 17.05.2016 21:31, deadalnix wrote:
 On Tuesday, 17 May 2016 at 18:08:47 UTC, Timon Gehr wrote:
 Right. Hence, the 80-bit CTFE results have to be converted to the
 final precision at some point in order to commence the runtime
 computation. This means that additional rounding happens, which was
 not present in the original program. The additional total roundoff
 error this introduces can exceed the roundoff error you would have
 suffered by using the lower precision in the first place, sometimes
 completely defeating precision-enhancing improvements to an algorithm.
WAT ? Is that really possible ?
Yes, I'm sorry, but this can and does happen. Consider http://forum.dlang.org/post/nhi7m4$css$1 digitalmars.com You can build similar examples involving only CTFE. Refer to http://forum.dlang.org/post/nhi9gh$fa4$1 digitalmars.com for an explanation of one case where this can happen. (I had actually written that post three days ago, and assumed that it had been posted to the newsgroup, but something went wrong, apparently.)
May 18 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/17/2016 11:08 AM, Timon Gehr wrote:
 Right. Hence, the 80-bit CTFE results have to be converted to the final
 precision at some point in order to commence the runtime computation. This
means
 that additional rounding happens, which was not present in the original
program.
 The additional total roundoff error this introduces can exceed the roundoff
 error you would have suffered by using the lower precision in the first place,
 sometimes completely defeating precision-enhancing improvements to an
algorithm.
I'd like to see an example of double rounding "completely defeating" an algorithm, and why an unusual case of producing a slightly worse answer trumps the usual case of producing better answers.
 There are other reasons why I think that this kind of implementation-defined
 behaviour is a terribly bad idea, eg.:

 - it breaks common assumptions about code, especially how it behaves under
 seemingly innocuous refactorings, or with a different set of compiler flags.
As pointed out, this already happens with just about every language. It happens with all C/C++ compilers I'm aware of. It happens as the default behavior of the x86. And as pointed out, refactoring (x+y)+z to x+(y+z) often produces different results, and surprises a lot of people.
 - it breaks reproducibility, which is sometimes more important that being close
 to the infinite precision result (which you cannot guarantee with any finite
 floating point type anyway).
   (E.g. in a game, it is enough if the result seems plausible, but it should be
 the same for everyone. For some scientific experiments, the ideal case is to
 have 100% reproducibility of the computation, even if it is horribly wrong,
such
 that other scientists can easily uncover and diagnose the problem, for
example.)
Nobody is proposing a D feature that does not produce reproducible results with the same program on the same inputs. This complaint is a strawman, as I've pointed out multiple times. In fact, the results would be MORE portable than with C/C++, because the FP behavior is completely implementation defined, and compilers take advantage of that.
May 17 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, May 17, 2016 at 02:07:21PM -0700, Walter Bright via Digitalmars-d wrote:
 On 5/17/2016 11:08 AM, Timon Gehr wrote:
[...]
- it breaks reproducibility, which is sometimes more important that
being close to the infinite precision result (which you cannot
guarantee with any finite floating point type anyway).  (E.g. in a
game, it is enough if the result seems plausible, but it should be
the same for everyone. For some scientific experiments, the ideal
case is to have 100% reproducibility of the computation, even if it
is horribly wrong, such that other scientists can easily uncover and
diagnose the problem, for example.)
Nobody is proposing a D feature that does not produce reproducible results with the same program on the same inputs. This complaint is a strawman, as I've pointed out multiple times.
Wasn't Manu's original complaint that, given a particular piece of FP code that uses floats, evaluating that code at compile-time may produce different results than evaluating it at runtime, because (as you're proposing) the compiler will use higher precision than specified for intermediate results? Of course, the compile-time answer is arguably "more correct" because it has less roundoff error, but the point here is not how accurate that answer is, but that it *doesn't match the runtime results*. This mismatch, from what I understand, is what causes the graphical glitches that Manu was referring to. According to your prescription, then, the runtime code should be "fixed" to use higher precision, so that it will also produce the same, "more correct" answer. But unfortunately, that's not workable because of the performance implications. At the end of the day, nobody cares whether a game draws a polygon with the most precise coordinates, what people *do* care is that there's a mismatch between the "more correct" and "less correct" rendering of the polygon (produced, respectively, from CTFE and from runtime) that causes a visually noticeable glitch. It *looks* wrong, no matter how much you may argue that it's "more correct". You are probably right scientifically, but in a game, people are concerned about what they see, not whether polygon coordinates have less roundoff error at CTFE vs. at runtime. T -- We are in class, we are supposed to be learning, we have a teacher... Is it too much that I expect him to teach me??? -- RL
May 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/17/2016 5:22 PM, H. S. Teoh via Digitalmars-d wrote:
 Wasn't Manu's original complaint that, given a particular piece of FP
 code that uses floats, evaluating that code at compile-time may produce
 different results than evaluating it at runtime, because (as you're
 proposing) the compiler will use higher precision than specified for
 intermediate results?  Of course, the compile-time answer is arguably
 "more correct" because it has less roundoff error, but the point here is
 not how accurate that answer is, but that it *doesn't match the runtime
 results*. This mismatch, from what I understand, is what causes the
 graphical glitches that Manu was referring to.
Except this happens with C/C++ too, and in fact Manu wasn't using D.
 According to your prescription, then, the runtime code should be "fixed"
 to use higher precision, so that it will also produce the same, "more
 correct" answer.  But unfortunately, that's not workable because of the
 performance implications. At the end of the day, nobody cares whether a
 game draws a polygon with the most precise coordinates, what people *do*
 care is that there's a mismatch between the "more correct" and "less
 correct" rendering of the polygon (produced, respectively, from CTFE and
 from runtime) that causes a visually noticeable glitch. It *looks*
 wrong, no matter how much you may argue that it's "more correct". You
 are probably right scientifically, but in a game, people are concerned
 about what they see, not whether polygon coordinates have less roundoff
 error at CTFE vs. at runtime.
That wasn't my prescription. My prescription was either changing the algorithm so it was not sensitive to exact bits-in-last-place, or to use roundToFloat() and roundToDouble() functions.
May 17 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 18 May 2016 at 05:40:57 UTC, Walter Bright wrote:
 That wasn't my prescription. My prescription was either 
 changing the algorithm so it was not sensitive to exact 
 bits-in-last-place, or to use roundToFloat() and 
 roundToDouble() functions.
With Manu's example, that would have been a good old fashioned matrix multiply to transform a polygon vertex from local space to screen space, with whatever other values were required for the render effect. The problem there being that the hardware itself only calculated 24 bits of precision while dealing with 32 bit values. Such a solution was not an option. Gaming hardware has gotten a lot less cheap and nasty. But Manu brought it up because it is conceptually the same problem as 32/64 bit run time values vs 80 bit compile time values. Every solution offered here either comes down to "rewrite your code" or "increase code complexity", neither of which is often an option (changing the code in Manu's example required a seven+ hour compile time each iteration of the code; and being a very hot piece of code, it needed to be as simple as possible to maintain speed). Unlike the hardware, game programming has not gotten less cheap nor nasty. We will cheat our way to the fastest performing code using whatever trick we can find that doesn't cause the whole thing to come crashing down. The standard way for creating float values at compile time is to calculate them manually at the correct precision and put a #define in with that value. Being unable to specify/override compile time precision means that the solution is to declare enums in the exact same manner, and might result in more maintenance work if someone decides they want to switch from float to double etc. for their value.
May 17 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/17/2016 11:15 PM, Ethan Watson wrote:
 With Manu's example, that would have been a good old fashioned matrix multiply
 to transform a polygon vertex from local space to screen space, with whatever
 other values were required for the render effect. The problem there being that
 the hardware itself only calculated 24 bits of precision while dealing with 32
 bit values. Such a solution was not an option.
I don't understand the 24 vs 32 bit value thing. There is no 32 bit mantissa floating point type. Floats have 24 bit mantissas, doubles 52.
 Gaming hardware has gotten a lot less cheap and nasty. But Manu brought it up
 because it is conceptually the same problem as 32/64 bit run time values vs 80
 bit compile time values. Every solution offered here either comes down to
 "rewrite your code" or "increase code complexity", neither of which is often an
 option (changing the code in Manu's example required a seven+ hour compile time
 each iteration of the code; and being a very hot piece of code, it needed to be
 as simple as possible to maintain speed). Unlike the hardware, game programming
 has not gotten less cheap nor nasty. We will cheat our way to the fastest
 performing code using whatever trick we can find that doesn't cause the whole
 thing to come crashing down. The standard way for creating float values at
 compile time is to calculate them manually at the correct precision and put a
 #define in with that value. Being unable to specify/override compile time
 precision means that the solution is to declare enums in the exact same manner,
 and might result in more maintenance work if someone decides they want to
switch
 from float to double etc. for their value.
I do not understand why the compile time version cannot use roundToFloat() in places where it matters. And if the hardware was using a different precision than float/double, which appears to have been the case, the code would have to be written to account for that anyway. In any case, the problem Manu was having was with C++. The precision of calculations is implementation defined in C++, and does vary all over the place depending on compiler brands, compiler versions, compiler switches, and exactly how the code is laid out. There can also be differences in how the FP hardware works on the compiler host machine and the target machine. My proposal would make the behavior more consistent than C++, not less. Lastly, it is hard to make suggestions on how to deal with the problem without seeing the actual offending code. There may very well be something else going on, or some simple adjustment that can be made. One way that *will* make the results exactly the same as on the target hardware is to actually run the code on the target hardware, save the results to a file, and incorporate that file in the build.
May 17 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 18 May 2016 at 06:57:58 UTC, Walter Bright wrote:
 I don't understand the 24 vs 32 bit value thing. There is no 32 
 bit mantissa floating point type. Floats have 24 bit mantissas, 
 doubles 52.
Not in the standards, no. But older gaming hardware was never known to be standards-conformant. As it turns out, the original hardware manuals can be found on the internet. https://www.dropbox.com/s/rsgx6xmpkf2zzz8/VU_Users_Manual.pdf Relevant info copied from page 27: Calculation * A 24-bit calculation including hidden bits is performed, and the result is truncated. The rounding-off operation in IEEE 754 is performed in the 0 direction, so the values for the least significant bit may vary.
 In any case, the problem Manu was having was with C++.
VU code was all assembly, I don't believe there was a C/C++ compiler for it.
 My proposal would make the behavior more consistent than C++, 
 not less.
This is why I ask for a compiler option to make it consistent with the C++ floating point architecture I select. Making it better than C++ is great for use cases where you're not inter-opping with C++ extensively. Although I do believe saying C++ is just clouding things here. Language doesn't matter, it's the runtime code using a different floating point instruction set/precision to compile time code that's the problem. See the SSE vs x87 comparisons posted in this thread for a concrete example. Same C++ code, different instruction sets and precisions. Regardless. Making extra build steps with pre-calculated values or whatever is of course a workable solution, but it also raises the barrier of entry. You can't just, for example, select a compiler option in your IDE and have it just work. You have to go out of your way to make it work the way you want it to. And if there's one thing you can count on, it's end users being lazy.
May 18 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 12:56 AM, Ethan Watson wrote:
 In any case, the problem Manu was having was with C++.
VU code was all assembly, I don't believe there was a C/C++ compiler for it.
The constant folding part was where, then?
 This is why I ask for a compiler option to make it consistent with the C++
 floating point architecture I select. Making it better than C++ is great for
use
 cases where you're not inter-opping with C++ extensively.
Trying to make D behave exactly like various C++ compilers do, with all their semi-documented behavior and semi-documented switches that affect constant folding behavior, is a hopeless task. I doubt various C++ compilers are this compatible, even if they follow the same ABI. You're also asking for a mode where the compiler for one machine is supposed to behave like hand-coded assembler for another machine with a different instruction set. I doubt any compiler is going to deliver that, unless you create a customized compiler specifically for that.
 Although I do believe saying C++ is just clouding things here. Language doesn't
 matter, it's the runtime code using a different floating point instruction
 set/precision to compile time code that's the problem. See the SSE vs x87
 comparisons posted in this thread for a concrete example. Same C++ code,
 different instruction sets and precisions.

 Regardless. Making extra build steps with pre-calculated values or whatever is
 of course a workable solution, but it also raises the barrier of entry. You
 can't just, for example, select a compiler option in your IDE and have it just
 work. You have to go out of your way to make it work the way you want it to.
And
 if there's one thing you can count on, it's end users being lazy.
From your description, there would have been a problem *regardless* of the precision used for constant folding. This is because the target hardware used truncation, and constant folding (in all compilers I know of) use rounding, and no compiler I know of allows controlling the rounding mode in constant folding.
May 18 2016
next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
 The constant folding part was where, then?
Probably on the EE, executed with different precision (32-bit) and passed over to the VU via one of its registers. Manu spent more time with that code than I did and can probably give exact details. But pasting the code, given it's proprietary code from a 15 year old engine, will be difficult at best considering the code is likely on a backup tape somewhere.
 You're also asking for a mode where the compiler for one 
 machine is supposed to behave like hand-coded assembler for 
 another machine with a different instruction set.
Actually, I'm asking for something exactly like the arch option for MSVC/-mfmath option for GCC/etc, and have it respect that for CTFE.
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 1:30 AM, Ethan Watson wrote:
 You're also asking for a mode where the compiler for one machine is supposed
 to behave like hand-coded assembler for another machine with a different
 instruction set.
Actually, I'm asking for something exactly like the arch option for MSVC/-mfmath option for GCC/etc, and have it respect that for CTFE.
MSVC doesn't appear to have a switch that does what you ask for: https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx
May 18 2016
next sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:
 MSVC doesn't appear to have a switch that does what you ask for
I'm still not entirely sure what the /fp switch does for x64 builds. The documentation is not clear in the slightest and I haven't been able to find any concrete information. As near as I can tell it has no effect as the original behaviour was tied to how it handles the x87 control words. But it might also be possible that the SSE instructions emitted can differ depending on what operation you're trying to do. I have not dug deep to see exactly how the code gen differs. I can take a guess that /fp:precise was responsible for promoting my float to a double to call CRT functions, but I have not tested that so that's purely theoretical at the moment. Of course, while this conversation has mostly been for compile time constant folding, the example of passing a value from the EE and treating it as a constant in the VU is still analagous to calculating a value at compile time in D at higher precision than the instruction set the runtime code is compiled to work with. /arch:sse2 is the default with MSVC x64 builds (Xbox One defaults to /arch:avx), and it sounds like the DMD has defaulted to sse2 for a long time. The exception being the compile time behaviour. That compile time behaviour conforming to the the runtime behaviour is an option I want, with the default being whatever is decided in here. Executing code at compile time at a higher precision than what SSE dictates is effectively undesired behaviour for our use cases. And in cases where we compile code for another architecture on x64 (let's say ARM code with NEON instructions, as it's the most common case thanks to iOS development) then it would be forced to fallback to the default. Fine for most use cases as well. It would be up to the user to compile their ARM code on an ARM processor to get the code execution match if they need it.
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 2:54 AM, Ethan Watson wrote:
 On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:
 MSVC doesn't appear to have a switch that does what you ask for
I'm still not entirely sure what the /fp switch does for x64 builds. The documentation is not clear in the slightest and I haven't been able to find any concrete information. As near as I can tell it has no effect as the original behaviour was tied to how it handles the x87 control words. But it might also be possible that the SSE instructions emitted can differ depending on what operation you're trying to do. I have not dug deep to see exactly how the code gen differs. I can take a guess that /fp:precise was responsible for promoting my float to a double to call CRT functions, but I have not tested that so that's purely theoretical at the moment. Of course, while this conversation has mostly been for compile time constant folding, the example of passing a value from the EE and treating it as a constant in the VU is still analagous to calculating a value at compile time in D at higher precision than the instruction set the runtime code is compiled to work with. /arch:sse2 is the default with MSVC x64 builds (Xbox One defaults to /arch:avx), and it sounds like the DMD has defaulted to sse2 for a long time. The exception being the compile time behaviour. That compile time behaviour conforming to the the runtime behaviour is an option I want, with the default being whatever is decided in here. Executing code at compile time at a higher precision than what SSE dictates is effectively undesired behaviour for our use cases. And in cases where we compile code for another architecture on x64 (let's say ARM code with NEON instructions, as it's the most common case thanks to iOS development) then it would be forced to fallback to the default. Fine for most use cases as well. It would be up to the user to compile their ARM code on an ARM processor to get the code execution match if they need it.
Again, even if the precision matches, the rounding will NOT match, and you will get different results randomly dependent on the exact operand values. If those differences matter, then you'll randomly be up all night debugging it. If you're willing to try the approach I mentioned, it'll cost you a bit more time up front, but may save a lot of agony later.
May 18 2016
parent reply Ethan Watson <gooberman gmail.com> writes:
On Wednesday, 18 May 2016 at 11:17:14 UTC, Walter Bright wrote:
 Again, even if the precision matches, the rounding will NOT 
 match, and you will get different results randomly dependent on 
 the exact operand values.
We've already been burned by middlewares/APIS toggling MMX flags on and off and not cleaning up after themselves, and as such we strictly control those flags going in to and out of such areas. We even have a little class with implementations for x87 (thoroughly deprecated) and SSE that is used in a RAII manner, copying the MMX flag on construction and restoring it on destruction. I appreciate that it sounds like I'm starting to stretch to hold to my point, but I imagine we'd also be able to control such things with the compiler - or at least know what flags it uses so that we can ensure consistent behaviour between compilation and runtime.
May 18 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 4:30 AM, Ethan Watson wrote:
 I appreciate that it sounds like I'm starting to stretch to hold to my point,
 but I imagine we'd also be able to control such things with the compiler - or
at
 least know what flags it uses so that we can ensure consistent behaviour
between
 compilation and runtime.
All compilers I know of use "round to nearest" for constant folding, and that is not adjustable.
May 19 2016
prev sibling parent reply ixid <adamsibson hotmail.com> writes:
On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:
 On 5/18/2016 1:30 AM, Ethan Watson wrote:
 You're also asking for a mode where the compiler for one 
 machine is supposed
 to behave like hand-coded assembler for another machine with 
 a different
 instruction set.
Actually, I'm asking for something exactly like the arch option for MSVC/-mfmath option for GCC/etc, and have it respect that for CTFE.
MSVC doesn't appear to have a switch that does what you ask for: https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx
Apologies if this has been addressed in the thread, it's a difficult structure to follow for technical discussion. You seem positive about software implementations of float. What are your thoughts on having the compile time implementation of a given type mirror the behaviour of the runtime version? Fundamentally whatever rules are chosen it would seem better to have fewer rules for people to remember.
May 18 2016
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 18 May 2016 at 21:28, ixid via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 18 May 2016 at 08:55:03 UTC, Walter Bright wrote:
 On 5/18/2016 1:30 AM, Ethan Watson wrote:
 You're also asking for a mode where the compiler for one machine is
 supposed
 to behave like hand-coded assembler for another machine with a different
 instruction set.
Actually, I'm asking for something exactly like the arch option for MSVC/-mfmath option for GCC/etc, and have it respect that for CTFE.
MSVC doesn't appear to have a switch that does what you ask for: https://msdn.microsoft.com/en-us/library/e7s85ffb.aspx
Apologies if this has been addressed in the thread, it's a difficult structure to follow for technical discussion. You seem positive about software implementations of float. What are your thoughts on having the compile time implementation of a given type mirror the behaviour of the runtime version? Fundamentally whatever rules are chosen it would seem better to have fewer rules for people to remember.
That's precisely the suggestion; that compile time execution of a given type mirror the runtime, that is, matching precisions in this case. ...within reason; as Walter has pointed out consistently, it's very difficult to be PERFECT for all the reasons he's been repeating, but there's still a massive difference between the runtime executing a bunch of float code, and the compile time executing it all promoted to 80bits. Results will drift apart very quickly.
May 18 2016
parent reply ixid <adamsibson hotmail.com> writes:
On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
 That's precisely the suggestion; that compile time execution of 
 a
 given type mirror the runtime, that is, matching precisions in 
 this
 case.
 ...within reason; as Walter has pointed out consistently, it's 
 very
 difficult to be PERFECT for all the reasons he's been 
 repeating, but
 there's still a massive difference between the runtime 
 executing a
 bunch of float code, and the compile time executing it all 
 promoted to
 80bits. Results will drift apart very quickly.
What do you think can be productively done to improve the situation? I am beginning to think a Wiki-like structure would be better for these discussions where each major debater has their thoughts on a specific issue in a relevantly headed section so there is more clarity.
May 18 2016
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 18 May 2016 at 21:53, ixid via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 18 May 2016 at 11:38:23 UTC, Manu wrote:
 That's precisely the suggestion; that compile time execution of a
 given type mirror the runtime, that is, matching precisions in this
 case.
 ...within reason; as Walter has pointed out consistently, it's very
 difficult to be PERFECT for all the reasons he's been repeating, but
 there's still a massive difference between the runtime executing a
 bunch of float code, and the compile time executing it all promoted to
 80bits. Results will drift apart very quickly.
What do you think can be productively done to improve the situation? I am beginning to think a Wiki-like structure would be better for these discussions where each major debater has their thoughts on a specific issue in a relevantly headed section so there is more clarity.
I've said so many times; I think it would be more useful if CTFE operated on the *specified type*. That is all. That should produce results as close as reasonably possible to the runtime. If I want CTFE to operate at real precision (as is often the case), float functions just take an 'isFloatingPoint!T' and users can execute them at whatever precision they like. Most float functions already have this form.
May 18 2016
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
 Trying to make D behave exactly like various C++ compilers do, 
 with all their semi-documented behavior and semi-documented 
 switches that affect constant folding behavior, is a hopeless 
 task.

 I doubt various C++ compilers are this compatible, even if they 
 follow the same ABI.
They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128 bits soft floats in their innards.
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 3:15 AM, deadalnix wrote:
 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
 Trying to make D behave exactly like various C++ compilers do, with all their
 semi-documented behavior and semi-documented switches that affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if they follow the
 same ABI.
They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128 bits soft floats in their innards.
Looks like LLVM had the same idea as myself. Anyhow, this pretty much destroys the idea that I have proposed some sort of cowboy FP that's going to wreck FP programs. (What is arbitrary precision FB?)
May 18 2016
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 18 May 2016 04:11:08 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:

 On 5/18/2016 3:15 AM, deadalnix wrote:
 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:  
 Trying to make D behave exactly like various C++ compilers do,
 with all their semi-documented behavior and semi-documented
 switches that affect constant folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if they
 follow the same ABI.
  
They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128 bits soft floats in their innards.
Looks like LLVM had the same idea as myself. Anyhow, this pretty much destroys the idea that I have proposed some sort of cowboy FP that's going to wreck FP programs. (What is arbitrary precision FB?)
A claim from GMP, a library used by GCC:
 GMP is a free library for arbitrary precision arithmetic, operating on
 signed integers, rational numbers, and floating-point numbers. There
 is
 no practical limit to the precision except the ones implied by the
 available memory in the machine GMP runs on.
It's difficult to find reliable information, but I think GCC always uses the target precision for constant folding: https://gcc.gnu.org/onlinedocs/gcc-6.1.0/gccint/Floating-Point.html#Floating-Point
 Because different representation systems may offer different amounts
 of
 range and precision, all floating point constants must be represented
 in the target machine's format. Therefore, the cross compiler cannot
 safely use the host machine's floating point arithmetic; it must
 emulate the target's arithmetic. To ensure consistency, GCC always
 uses
 emulation to work with floating point values, even when the host and
 target floating point formats are identical. 
May 18 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:
 On 5/18/2016 3:15 AM, deadalnix wrote:
 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote:
 Trying to make D behave exactly like various C++ compilers 
 do, with all their
 semi-documented behavior and semi-documented switches that 
 affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if 
 they follow the
 same ABI.
They aren't. For instance, GCC use arbitrary precision FB, and LLVM uses 128 bits soft floats in their innards.
Looks like LLVM had the same idea as myself. Anyhow, this pretty much destroys the idea that I have proposed some sort of cowboy FP that's going to wreck FP programs. (What is arbitrary precision FB?)
Typo: arbitrary precision FP. Meaning some soft float that grows as big as necessary to not lose precision à la BitInt but for floats.
May 18 2016
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 18 May 2016 11:48:49 +0000
schrieb deadalnix <deadalnix gmail.com>:

 On Wednesday, 18 May 2016 at 11:11:08 UTC, Walter Bright wrote:
 On 5/18/2016 3:15 AM, deadalnix wrote: =20
 On Wednesday, 18 May 2016 at 08:21:18 UTC, Walter Bright wrote: =20
 Trying to make D behave exactly like various C++ compilers=20
 do, with all their
 semi-documented behavior and semi-documented switches that=20
 affect constant
 folding behavior, is a hopeless task.

 I doubt various C++ compilers are this compatible, even if=20
 they follow the
 same ABI.
 =20
They aren't. For instance, GCC use arbitrary precision FB, and=20 LLVM uses 128 bits soft floats in their innards. =20
Looks like LLVM had the same idea as myself. Anyhow, this pretty much destroys the idea that I have proposed=20 some sort of cowboy FP that's going to wreck FP programs. (What is arbitrary precision FB?) =20
=20 Typo: arbitrary precision FP. Meaning some soft float that grows=20 as big as necessary to not lose precision =C3=A0 la BitInt but for=20 floats. =20
Do you have a link explaining GCC actually uses such a soft float? For example https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says "This file should be rewritten to use an arbitrary precision..."
May 18 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:
 Do you have a link explaining GCC actually uses such a soft 
 float? For example 
 https://github.com/gcc-mirror/gcc/blob/master/gcc/fold-const.c#L20 still says
"This file should be rewritten to use an arbitrary precision..."
Alright, it seems that GCC doesn't use arbitrary precision float everywhere :)
May 18 2016
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:
 Do you have a link explaining GCC actually uses such a soft 
 float?
I'm confused as to why the compiler would be using soft floats instead of hard floats.
May 18 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 18 May 2016 at 19:20:20 UTC, jmh530 wrote:
 On Wednesday, 18 May 2016 at 12:39:21 UTC, Johannes Pfau wrote:
 Do you have a link explaining GCC actually uses such a soft 
 float?
I'm confused as to why the compiler would be using soft floats instead of hard floats.
Cross compilation.
May 18 2016
parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 18 May 2016 at 19:30:12 UTC, deadalnix wrote:
 I'm confused as to why the compiler would be using soft floats 
 instead of hard floats.
Cross compilation.
Ah, looking back on the discussion, I see the comments about cross compilation and soft floats. Making more sense now... So if compiling on x86 for x86, you could just use hard floats, but if compiling on x86 for some other system, then use soft floats to mimic what the result would be as if you had compiled on that system. Correct? But what if you are compiling for a system whose float behavior matches the system you're compiling on? So for instance, suppose you are only using 32bit floats and not allowing anything fancy like 80bit intermediate calculations. And you're compiling for a system that treats floats the same way. Then, you could theoretically use hard floats in the compiler and the results would be the same.
May 18 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 4:48 AM, deadalnix wrote:
 Typo: arbitrary precision FP. Meaning some soft float that grows as big as
 necessary to not lose precision à la BitInt but for floats.
0.10 is not representable in a binary format regardless of precision.
May 18 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:
 On 5/18/2016 4:48 AM, deadalnix wrote:
 Typo: arbitrary precision FP. Meaning some soft float that 
 grows as big as
 necessary to not lose precision à la BitInt but for floats.
0.10 is not representable in a binary format regardless of precision.
You should ask the gcc guys how they do it, but you can surely represent this as a fraction, so I see no major blocker.
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 1:22 PM, deadalnix wrote:
 On Wednesday, 18 May 2016 at 20:14:22 UTC, Walter Bright wrote:
 On 5/18/2016 4:48 AM, deadalnix wrote:
 Typo: arbitrary precision FP. Meaning some soft float that grows as big as
 necessary to not lose precision à la BitInt but for floats.
0.10 is not representable in a binary format regardless of precision.
You should ask the gcc guys how they do it, but you can surely represent this as a fraction,
Right.
 so I see no major blocker.
Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by definition, not representable as a ratio.
May 18 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:
 Now try the square root of 2. Or pi, e, etc. The irrational 
 numbers are, by definition, not representable as a ratio.
Continued fraction? :-)
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:
 On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:
 Now try the square root of 2. Or pi, e, etc. The irrational numbers are, by
 definition, not representable as a ratio.
Continued fraction? :-)
Somehow I don't think gcc is using Mathematica for constant folding.
May 18 2016
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, May 18, 2016 at 04:28:13PM -0700, Walter Bright via Digitalmars-d wrote:
 On 5/18/2016 4:17 PM, Joseph Rushton Wakeling wrote:
 On Wednesday, 18 May 2016 at 23:09:28 UTC, Walter Bright wrote:
 Now try the square root of 2. Or pi, e, etc. The irrational
 numbers are, by definition, not representable as a ratio.
Continued fraction? :-)
Somehow I don't think gcc is using Mathematica for constant folding.
http://apt.cs.manchester.ac.uk/ftp/pub/apt/papers/drl_ieee01.pdf Side-note: continued fractions of quadratic irrationals (a + b*sqrt(c)) are periodic, so it's possible to do exact arithmetic with them using only finite storage. T -- All problems are easy in retrospect.
May 18 2016
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, May 18, 2016 at 04:09:28PM -0700, Walter Bright via Digitalmars-d wrote:
[...]
 Now try the square root of 2. Or pi, e, etc. The irrational numbers
 are, by definition, not representable as a ratio.
This is somewhat tangential, but in certain applications it is perfectly possible to represent certain irrationals exactly. For example, if you're working with quadratic fractions of the form a + b*sqrt(r) where r is fixed, you can exactly represent all such numbers as a pair of rational coefficients. These numbers are closed under addition, subtraction, multiplication, and division, so you have a nice closed system that can handle numbers that contain square roots. It is possible to have multiple square roots (e.g., a + b*sqrt(r) + c*sqrt(s) + d*sqrt(r*s)), but the tuple gets exponentially long with each different root, so it quickly becomes unwieldy (not to mention slow). Similar techniques can be used for exactly representing roots of cubic equations (including cube roots) -- a single cubic root can be represented as a 3-tuple -- or higher. Again, these are closed under +, *, -, /, so you can do quite a lot of interesting things with them without sacrificing exact arithmetic. Though again, things quickly become unwieldy. Higher order roots are also possible, though of questionable value since storage and performance costs quickly become too high. Transcendentals like pi or e are out of reach by this method, though. You'd need a variable-length vector to represent numbers that contain pi or e as a factor, because their powers never become integral, so you potentially need to represent an arbitrary power of pi in order to obtain closure under + and *. Of course, with increasing sophistication the computation and storage costs increase accordingly, but the point is that if your application is known to only need a certain subset of irrationals, it may well be possible to represent them exactly within reasonable costs. (There's also RealLib, that can exactly represent *all* computable reals, but may not be feasible depending on what your application does.) T -- Let's eat some disquits while we format the biskettes.
May 18 2016
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 18 May 2016 at 18:21, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/18/2016 12:56 AM, Ethan Watson wrote:
 In any case, the problem Manu was having was with C++.
VU code was all assembly, I don't believe there was a C/C++ compiler for it.
The constant folding part was where, then?
The comparison was a 24bit fpu doing runtime work but where some constant input data was calculated with a separate 32bit fpu. The particulars were not ever intended to be relevant to the conversation, except the fact that 2 differently precisioned float units were producing output that then had to reconcile. The analogy was to CTFE doing all its work at 80bits, and then the processor doing work with the types explicitly stated by the programmer; a runtime calculation compared against the same compile time calculate is likely to be quite radically different. I don't care about the precision, I just care that they're really different. Ideally, CTFE would produce a result that is as similar to the runtime result as reasonably possible, and I expect using the stated types to do the calculations would get much much closer. I don't know if a couple of least-significant bits of difference would have caused problems for us, I suspect not, but I know that doing math at radically different precisions (ie, 32bits vs 80bits) does lead to radically different results, not just a couple of bits. That is my concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit fpu's.
May 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 4:27 AM, Manu via Digitalmars-d wrote:
 The comparison was a 24bit fpu doing runtime work but where some
 constant input data was calculated with a separate 32bit fpu. The
 particulars were not ever intended to be relevant to the conversation,
 except the fact that 2 differently precisioned float units were
 producing output that then had to reconcile.

 The analogy was to CTFE doing all its work at 80bits, and then the
 processor doing work with the types explicitly stated by the
 programmer; a runtime calculation compared against the same compile
 time calculate is likely to be quite radically different. I don't care
 about the precision, I just care that they're really different.
 Ideally, CTFE would produce a result that is as similar to the runtime
 result as reasonably possible, and I expect using the stated types to
 do the calculations would get much much closer.
 I don't know if a couple of least-significant bits of difference would
 have caused problems for us, I suspect not, but I know that doing math
 at radically different precisions (ie, 32bits vs 80bits) does lead to
 radically different results, not just a couple of bits. That is my
 concern wrt reproduction of my anecdote from PS2 and Gamecubes 24bit
 fpu's.
"radically different results" means you were getting catastrophic loss of precision in the runtime results, which is just what doing the CTFE at a higher precision is attempting to avoid (and apparently successfully). I get that the game did not care about the value produced, even if it was total garbage. I suspect that games are the only category of FP apps where there is no concern for the values computed. I do not understand the tolerance for bad results in scientific, engineering, medical, or finance applications. The only way to *reliably* get the same results at compile time as at runtime, regardless of what technology is put into the language/compiler, is to actually run the code on the target hardware, save the results, and insert those values into the code. It isn't that hard to do, and will save you from endless ghosts popping up that waste your debugging time.
May 18 2016
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.
I don't think anyone has suggested tolerance for bad results in any of those applications. What _has_ been argued for is that in order to _prevent_ bad results it's necessary for the programmer to have control and clarity over the choice of precision as much as possible. If I'm writing a numerical simulation or calculation using insufficient floating-point precision, I don't _want_ to be saved by under-the-hood precision increases -- I would like it to break because then I will be forced to improve either the floating-point precision or the choice of algorithm (or both). To be clear: the fact that D makes it a priority to offer me the highest possible floating-point precision is awesome. But precision is not the only factor in generating accurate scientific results.
May 18 2016
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
Wakeling wrote:
 On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.
I don't think anyone has suggested tolerance for bad results in any of those applications.
I don't think its about tolerance for bad results, so much as the ability to make the trade-off between speed and precision when you need to. Just thinking of finance: a market maker has to provide quotes on potentially thousands of instruments in real-time. This might involve some heavy calculations for options pricing. When dealing with real-time tick data (the highest frequency of financial data), sometimes you take shortcuts that you wouldn't be willing to do if you were working with lower frequency data. It's not that you don't care about precision. It's just that sometimes it's more important to be fast than accurate. I'm not a market maker and don't work with high frequency data. I usually look at low enough frequency data so that I actually do generally care more about accurate results than speed. Nevertheless, sometimes with hefty simulations that take several hours or days to run, I might be willing to take some short cuts to get a general idea of the results. Then, when I implement the strategy, I might do something different.
May 18 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Wednesday, 18 May 2016 at 22:16:44 UTC, jmh530 wrote:
 On Wednesday, 18 May 2016 at 21:49:34 UTC, Joseph Rushton 
 Wakeling wrote:
 On Wednesday, 18 May 2016 at 20:29:27 UTC, Walter Bright wrote:
 I do not understand the tolerance for bad results in 
 scientific, engineering, medical, or finance applications.
I don't think anyone has suggested tolerance for bad results in any of those applications.
I don't think its about tolerance for bad results, so much as the ability to make the trade-off between speed and precision when you need to.
It isn't only about speed. It is about correctness as well. Compilers should not change the outcome (including precision and rounding) unless the programmer has explicitly requested it, and if you do the effects should be well documented so that the effect is clear to the programmer. This is a well known and universally accepted principle in compiler design. Take for instance the documentation page for a professional level compiler targeting embedded programming: http://processors.wiki.ti.com/index.php/Floating_Point_Optimization It gives the programmer explicit control over what kind of deviations the compiler can create. If you look at Intel's compiler you'll see that they even turn off fused-multiply-add in strict mode because it skips a single rounding between the multiply and the add. https://software.intel.com/sites/default/files/article/326703/fp-control-2012-08.pdf Some languages allow constant folding of literals in _expressions_ to use infinite precision, but once you have bound it to a variable it should use the same precision, and you always have to use the same rounding mode. This means that if you should check the rounding mode before using precomputed values. If you cannot emulate the computation then you can simply run the computations prior to entering main and put the "constant computations" in global variables. That said, compilers may have some problematic settings enabled by default in order to look good in benchmarks/test suites. In order to be IEEE compliant you cannot even optimize "0.0 - x" to "-x", you also cannot optimize "x - 0.0" to "x". Such issues make compiler vendors provide many different floating point options as command line flags.
May 19 2016
prev sibling parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 18 May 2016 at 07:56:58 UTC, Ethan Watson wrote:
 Not in the standards, no. But older gaming hardware was never 
 known to be standards-conformant.

 As it turns out, the original hardware manuals can be found on 
 the internet.
Hmmm... I can't help but look at this and think about x86 instructions of old, and how they aren't utilized usually fully or properly in language specs to take advantage of them, like the jump carry, overflow bits/checks, or the nature of how the CPUs handle arithmetic. I'm referring more to multiplication where the result of 2 16-bit multiplies will result in a 32bit output (AX&DX, is the same for 32/64bit instructions), but most likely the upper 32bits are just outright ignored rather than making use of those possible features. Heh, I'd love to see more hardware level abstraction that's built into the language. Almost like: try {} // Considers the result of 1 line of basic math to be caught by: carry {} //only activates if carry is set overflow {} //if overflowed during some math modulus(m){} //get the remainder as m after a division operation mult(dx) {} //get upper 32/64/whatever after a multiply and set as dx Of course I'd understand if some hardware doesn't offer such support, so the else could be thrown in to allow a workaround code to detect such an event, or only allow it if it's a compliant architecture. Although workaround detection is always possible, just not as fast as hardware supplied. Although I'm not fully familiar with all the results a FPU result could give, or if such a system would be beneficial to the current discussion on floats. I would prefer not to inject fixed x86 instructions if I can avoid it.
May 18 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 18 May 2016 at 08:38:07 UTC, Era Scarecrow wrote:
  try {}    // Considers the result of 1 line of basic math to 
 be caught by:
  carry     {} //only activates if carry is set
  overflow  {} //if overflowed during some math
  modulus(m){} //get the remainder as m after a division 
 operation
  mult(dx)  {} //get upper 32/64/whatever after a multiply and 
 set as dx

  Of course I'd understand if some hardware doesn't offer such 
 support, so the else could be thrown in to allow a workaround 
 code to detect such an event, or only allow it if it's a 
 compliant architecture. Although workaround detection is always 
 possible, just not as fast as hardware supplied.
https://code.dlang.org/packages/checkedint https://dlang.org/phobos/core_checkedint.html
May 18 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:
 On Wednesday, 18 May 2016 at 08:38:07 UTC, Era Scarecrow wrote:
  try {}    // Considers the result of 1 line of basic math to 
 be caught by:
  carry     {} //only activates if carry is set
  overflow  {} //if overflowed during some math
  modulus(m){} //get the remainder as m after a division 
 operation
  mult(dx)  {} //get upper 32/64/whatever after a multiply and 
 set as dx

 Of course I'd understand if some hardware doesn't offer such 
 support, so the else could be thrown in to allow a workaround 
 code to detect such an event, or only allow it if it's a 
 compliant architecture. Although workaround detection is 
 always possible, just not as fast as hardware supplied.
https://code.dlang.org/packages/checkedint https://dlang.org/phobos/core_checkedint.html
Glancing at the checkedInt I really don't see it as being the same as what I'm talking about. Overflow/carry for add perhaps, but unless it breaks down to a single instruction for the compiler to determine if it needs to do something, I see it as a failure (at best, a workaround). That's just my thoughts. CheckedInt simply _doesn't_ cover what I was talking about. Obtaining the modulus for 0 cost/instructions after doing a division which is in the hardware's opcode side effects (unless the compiler recognizes the pattern and offers it as an optimization), or having the full result of a multiply on hand (that exceeds it's built-in size, long.max*long.max = 128bit result, which the hardware hands to you if you check the register it stores the other half of the result in). Perhaps what I want is more limited to handling certain tasks (making software math libraries) but I'd still like/want to see access to the other effects of these opcodes.
May 18 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 18 May 2016 at 11:46:37 UTC, Era Scarecrow wrote:
 On Wednesday, 18 May 2016 at 10:25:10 UTC, tsbockman wrote:
 https://code.dlang.org/packages/checkedint
 https://dlang.org/phobos/core_checkedint.html
Glancing at the checkedInt I really don't see it as being the same as what I'm talking about. Overflow/carry for add perhaps, but unless it breaks down to a single instruction for the compiler to determine if it needs to do something, I see it as a failure (at best, a workaround). That's just my thoughts. CheckedInt simply _doesn't_ cover what I was talking about.
The functions in druntime's `core.checkedint` are intrinsics that map directly to the hardware overflow/carry instructions. The DUB package I linked provides various wrapper functions and data structures to make it easier to use the `core.checkedint` intrinsics (among other things). The performance cost of the wrappers is low with proper inlining, which GDC and LDC are able to provide. (DMD is another story...)
 Obtaining the modulus for 0 cost/instructions after doing a 
 division which is in the hardware's opcode side effects (unless 
 the compiler recognizes the pattern and offers it as an 
 optimization), or having the full result of a multiply on hand 
 (that exceeds it's built-in size, long.max*long.max = 128bit 
 result, which the hardware hands to you if you check the 
 register it stores the other half of the result in).
I agree that intrinsics for this would be nice. I doubt that any current D platform is actually computing the full 128 bit result for every 64 bit multiply though - that would waste both power and performance, for most programs.
May 18 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste both 
 power and performance, for most programs.
Except the 128 result is _already_ there for 0 cost (at least for x86 instructions that I'm aware). There's bound to be enough cases (say pseudo random number generation, encryption, or numerical processing above 64bits) I'd like access to it supported by the language and not having to inject instructions using the asm command. This is also the same with the division where the number could be a 128bit number divided by a 64bit divisor. We could get the main benefit of the 128bit cent with very few instructions if we simply guarantee one of the two arguments smaller than 65 bits (although the dividend and remainder both need to be 64 bits or smaller)
May 18 2016
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:
 On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste 
 both power and performance, for most programs.
Except the 128 result is _already_ there for 0 cost (at least for x86 instructions that I'm aware).
Can you give me a source for this, or at least the name of the relevant op code? (I'm new to x86 assembly.)
 There's bound to be enough cases (say pseudo random number 
 generation, encryption, or numerical processing above 64bits) 
 I'd like access to it supported by the language and not having 
 to inject instructions using the asm command.
Of course it would be useful to have in the language; I wasn't disputing that. I'd like to have as much support for 128-bit integers in the language as possible. Among other things, this would greatly simplify getting 128-bit floating-point working. I'm just surprised that the CPU would really calculate the upper 64 bits of a multiply without being explicitly asked to.
May 18 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:
 On Wednesday, 18 May 2016 at 19:53:10 UTC, Era Scarecrow wrote:
 On Wednesday, 18 May 2016 at 19:36:59 UTC, tsbockman wrote:
 I agree that intrinsics for this would be nice. I doubt that 
 any current D platform is actually computing the full 128 bit 
 result for every 64 bit multiply though - that would waste 
 both power and performance, for most programs.
Except the 128 result is _already_ there for 0 cost (at least for x86 instructions that I'm aware).
Can you give me a source for this, or at least the name of the relevant op code? (I'm new to x86 assembly.)
http://www.mathemainzel.info/files/x86asmref.html#mul http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html There's div, idiv, mul, and imul which follow this exact pattern. Although the instruction mentioned in the following pages is meant for 32bit or less, the pattern used is no different. (mathemainzel.info) Usage MUL src Modifies flags CF OF (AF,PF,SF,ZF undefined) Unsigned multiply of the accumulator by the source. If "src" is a byte value, then AL is used as the other multiplicand and the result is placed in AX. If "src" is a word value, then AX is multiplied by "src" and DX:AX receives the result. If "src" is a double word value, then EAX is multiplied by "src" and EDX:EAX receives the result. The 386+ uses an early out algorithm which makes multiplying any size value in EAX as fast as in the 8 or 16 bit registers. (intel.com) Downloading the 64 intel manual on opcodes says the same thing, only the registers become RDX:RAX with 64bit instructions. Quadword RAX r/m64 RDX:RAX
May 18 2016
parent tsbockman <thomas.bockman gmail.com> writes:
On Wednesday, 18 May 2016 at 22:06:43 UTC, Era Scarecrow wrote:
 On Wednesday, 18 May 2016 at 21:02:03 UTC, tsbockman wrote:
 Can you give me a source for this, or at least the name of the 
 relevant op code? (I'm new to x86 assembly.)
http://www.mathemainzel.info/files/x86asmref.html#mul http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html
Thanks.
 [...]
 Downloading the 64 intel manual on opcodes says the same thing, 
 only the registers become RDX:RAX with 64bit instructions.

 Quadword RAX r/m64 RDX:RAX
I will look into adding intrinsics for full-width multiply and combined division-modulus.
May 18 2016
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 17.05.2016 23:07, Walter Bright wrote:
 On 5/17/2016 11:08 AM, Timon Gehr wrote:
 Right. Hence, the 80-bit CTFE results have to be converted to the final
 precision at some point in order to commence the runtime computation.
 This means
 that additional rounding happens, which was not present in the
 original program.
 The additional total roundoff error this introduces can exceed the
 roundoff
 error you would have suffered by using the lower precision in the
 first place,
 sometimes completely defeating precision-enhancing improvements to an
 algorithm.
I'd like to see an example of double rounding "completely defeating" an algorithm,
I have given this example, and I have explained it. However, let me provide one of the examples I have given before, in a more concrete fashion. Unfortunately, there is no way to use "standard" D to illustrate the problem, as there is no way to write an implementation that is guaranteed not to be broken, so let us assume hypothetically for now that we are using D', where all computations are performed at the specified precision. I'm copying the code from: https://en.wikipedia.org/wiki/Kahan_summation_algorithm $ cat kahanDemo.d module kahanDemo; double sum(double[] arr){ double s=0.0; foreach(x;arr) s+=x; return s; } double kahan(double[] arr){ double sum = 0.0; double c = 0.0; foreach(x;arr){ double y=x-c; double t=sum+y; c = (t-sum)-y; sum=t; } return sum; } double kahanBroken(double[] arr){ double sum = 0; double c= 0.0; foreach(x;arr){ real y=x-c; real t=sum+y; c = (t-sum)-y; sum=t; } return sum; } void main(){ double[] data=[1e16,1,-9e15]; import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } In D, the compiler is in principle allowed to transform the non-broken version to the broken version. (And maybe, it will soon be allowed to transform the baseline version to the Kahan version. Who knows.) Now let's see what DMD does: $ dmd --version DMD64 D Compiler v2.071.0 Copyright (c) 1999-2015 by Digital Mars written by Walter Bright $ dmd -m64 -run kahanDemo.d 1000000000000000.000000 1000000000000001.000000 1000000000000000.000000 Nice, this is what I expect. $ dmd -m64 -O -run kahanDemo.d 1000000000000000.000000 1000000000000001.000000 1000000000000000.000000 Still great. $ dmd -m32 -run kahanDemo.d 1000000000000000.000000 1000000000000001.000000 1000000000000000.000000 Liking this. $ dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Screw you, DMD! And suddenly, I need to compile and test my code with all combinations of compiler flags, and even then I am not sure the compiler is not intentionally screwing me over. How is this remotely acceptable?
 and why an unusual case of producing a slightly worse
It's not just slightly worse, it can cut the number of useful bits in half or more! It is not unusual, I have actually run into those problems in the past, and it can break an algorithm that is in Phobos today!
 answer trumps the usual case of producing better answers.
 ...
The 'usual case of producing better answers' /is not actually desirable/, because the compiler does not guarantee that it happens all the time! I don't want my code to rely on something to happen that might not always happen. I want to be sure that my code is correct. I cannot conveniently do so if you don't tell me in advance what it does, and/or if the behaviour has a lot of abstraction-breaking special cases.
 There are other reasons why I think that this kind of
 implementation-defined
 behaviour is a terribly bad idea, eg.:

 - it breaks common assumptions about code, especially how it behaves
 under
 seemingly innocuous refactorings, or with a different set of compiler
 flags.
As pointed out, this already happens with just about every language. It happens with all C/C++ compilers I'm aware of.
I'm not claiming those languages don't have broken floating point semantics. I have sometimes been using inline assembler in C++ to get the results I want. It's painful and unnecessary.
 It happens as the default behavior of the x86.
I know. I don't care. It is a stupid idea. See above.
 And as pointed out, refactoring (x+y)+z to x+(y+z)
 often produces different results, and surprises a lot of people.
 ...
As far as I can tell, you are saying: "You think A is bad, but A is similar to B, and B is bad, but B is hard to fix, hence A is actually good." I disagree. For floating point, '+' means: "add precisely, then round". It is indeed potentially surprising and not very helpful for generic code that floating point types and integral types use the same syntax for conceptually different things (i.e., the language commits operator overloading abuse), but we are stuck with that now. Implementation-defined behaviour can actually be eliminated in a backward-compatible way. Refactorings as simple as moving some expression into its own function should be possible without surprisingly wreaking havoc. Implementations can (and do) choose strange and useless behaviours. The fact that types do not actually specify what kind of value will be used at runtime is a pointless waste of type safety.
 - it breaks reproducibility, which is sometimes more important that
 being close
 to the infinite precision result (which you cannot guarantee with any
 finite
 floating point type anyway).
   (E.g. in a game, it is enough if the result seems plausible, but it
 should be
 the same for everyone. For some scientific experiments, the ideal case
 is to
 have 100% reproducibility of the computation, even if it is horribly
 wrong, such
 that other scientists can easily uncover and diagnose the problem, for
 example.)
Nobody is proposing a D feature that does not produce reproducible results
Do you disagree with the notion that implementation-defined behaviour is almost by definition detrimental to reproducibility? (I'm not trying to say that it makes reproducing results impossible. It is a continuum.)
 with the same program on the same inputs.
I'm talking about computations, not just programs, and I ideally want consistent behaviour across compilers/compiler versions/compiler flags (at least those flags not specifically designed to change language semantics in precisely that way). If you can additionally give me refactorability (i.e. a lack of unprincipled interplay between language features that should be orthogonal [1]), that would be really awesome. The result should ideally not depend on e.g. whether a computation is run at compile-time or run-time, or whatever other irrelevant implicit detail that is changed during refactoring or when switching machines. (e.g. someone might be running a 32-bit system, and another person (or the same person) might be running a 64-bit system, and they get different floating-point results. It just adds a lot of friction and pointless work.)
 This complaint is a strawman, as I've pointed out multiple times.
 ...
A strawman is an argument one incorrectly claims that the other party has made. Here, I think the underlying problem is that there is a misunderstanding. (I.e. you think I'm saying something I didn't intend to say, or vice-versa. [2]) This is often the case, hence I try to avoid calling out strawmen by name and instead try to put my argument differently. (E.g., when you seemingly started to claim that my argument was something like "using lower precision for the entire computation throughout should be expected to yield more accurate results" and then ridiculed it.)
 In fact, the results would be MORE portable than with C/C++,
I know that it is better than completely broken. I'm sorry, but I have higher standards. It should be not broken.
 because the
 FP behavior is completely implementation defined, and compilers take
 advantage of that.
I guess the reason why it is completely implementation defined is the desire that there should be completely standard-compliant C/C++ compilers for each platform, and the fact that implementers didn't want to support IEEE 754 everywhere. One way to implement a specification is to weaken the specification. [1] This is the single most important thing programming languages developed in academia often get right. [2] To clarify: I know that if I compile a program on systems with identical states, on an identical compiler with identical flags, I will (or should) get identical binaries that then produce identical results on conforming hardware. (See how I didn't need to say "identical hardware"? I want to be there!) It is not always desirable or practical to keep around all that additional state though.
May 18 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of useful 
 bits in half or more! It is not unusual, I have actually run 
 into those problems in the past, and it can break an algorithm 
 that is in Phobos today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand. As Don's talk pointed out, all floating-point calculations will see loss of precision starting there. In this case, not increasing precision gets the more accurate result, but other examples could be constructed that _heavily_ favor increasing precision. In fact, almost any real-world, non-toy calculation would favor it. In any case, nobody should depend on the precision out that far being accurate or "reliable."
May 18 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that _heavily_ 
 favor increasing precision.  In fact, almost any real-world, 
 non-toy calculation would favor it.
Please stop saying this. It is very wrong. Algorithms that need higher accuracy need error correction mechanisms, not unpredictable precision and rounding. Unpredictable precision and rounding makes adding error correction difficult so it does not improve accuracy, it harms accuracy when you need it.
May 19 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that _heavily_ 
 favor increasing precision.  In fact, almost any real-world, 
 non-toy calculation would favor it.
Please stop saying this. It is very wrong.
I will keep saying it because it is _not_ wrong.
 Algorithms that need higher accuracy need error correction 
 mechanisms, not unpredictable precision and rounding. 
 Unpredictable precision and rounding makes adding error 
 correction difficult so it does not improve accuracy, it harms 
 accuracy when you need it.
And that is what _you_ need to stop saying: there's _nothing unpredictable_ about what D does. You may find it unintuitive, but that's your problem. The notion that "error correction" can fix the inevitable degradation of accuracy with each floating-point calculation is just laughable.
May 19 2016
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:
 On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
 wrote:
 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
 In this case, not increasing precision gets the more accurate 
 result, but other examples could be constructed that 
 _heavily_ favor increasing precision.  In fact, almost any 
 real-world, non-toy calculation would favor it.
Please stop saying this. It is very wrong.
I will keep saying it because it is _not_ wrong.
Can you then please read this paper in it's entirety before continuing saying it. Because changing precision breaks properties of the semantics of IEEE floating point. What Every Computer Scientist Should Know About Floating-Point Arithmetic https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377 «Conventional wisdom maintains that extended-based systems must produce results that are at least as accurate, if not more accurate than those delivered on single/double systems, since the former always provide at least as much precision and often more than the latter. Trivial examples such as the C program above as well as more subtle programs based on the examples discussed below show that this wisdom is naive at best: some apparently portable programs, which are indeed portable across single/double systems, deliver incorrect results on extended-based systems precisely because the compiler and hardware conspire to occasionally provide more precision than the program expects.»
 And that is what _you_ need to stop saying: there's _nothing 
 unpredictable_ about what D does.  You may find it unintuitive, 
 but that's your problem.
No. It is not my problem as I would never use a system with this kind of semantics for anything numerical. It is a problem for D. Not for me.
 The notion that "error correction" can fix the inevitable 
 degradation of accuracy with each floating-point calculation is 
 just laughable.
Well, it is not laughable to computer scientists that accuracy depends on knowledge about precision and rounding... And I am a computer scientists, in case you have forgotten...
May 19 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 19 May 2016 at 11:00:31 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 19 May 2016 at 08:37:55 UTC, Joakim wrote:
 On Thursday, 19 May 2016 at 08:28:22 UTC, Ola Fosheim Grøstad 
 wrote:
 On Thursday, 19 May 2016 at 06:04:15 UTC, Joakim wrote:
 In this case, not increasing precision gets the more 
 accurate result, but other examples could be constructed 
 that _heavily_ favor increasing precision.  In fact, almost 
 any real-world, non-toy calculation would favor it.
Please stop saying this. It is very wrong.
I will keep saying it because it is _not_ wrong.
Can you then please read this paper in it's entirety before continuing saying it. Because changing precision breaks properties of the semantics of IEEE floating point. What Every Computer Scientist Should Know About Floating-Point Arithmetic https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html#3377 «Conventional wisdom maintains that extended-based systems must produce results that are at least as accurate, if not more accurate than those delivered on single/double systems, since the former always provide at least as much precision and often more than the latter. Trivial examples such as the C program above as well as more subtle programs based on the examples discussed below show that this wisdom is naive at best: some apparently portable programs, which are indeed portable across single/double systems, deliver incorrect results on extended-based systems precisely because the compiler and hardware conspire to occasionally provide more precision than the program expects.»
The example he refers to is laughable because it also checks for equality.
 The notion that "error correction" can fix the inevitable 
 degradation of accuracy with each floating-point calculation 
 is just laughable.
Well, it is not laughable to computer scientists that accuracy depends on knowledge about precision and rounding... And I am a computer scientists, in case you have forgotten...
Computer scientists are no good if they don't know any science.
May 19 2016
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:
 The example he refers to is laughable because it also checks 
 for equality.
With good reason, because it's intended to illustrate the point that two calculations that _look_ identical in code, that intuitively should produce identical results, actually don't, because of under-the-hood precision differences.
May 19 2016
parent Joakim <dlang joakim.fea.st> writes:
On Thursday, 19 May 2016 at 12:00:33 UTC, Joseph Rushton Wakeling 
wrote:
 On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:
 The example he refers to is laughable because it also checks 
 for equality.
With good reason, because it's intended to illustrate the point that two calculations that _look_ identical in code, that intuitively should produce identical results, actually don't, because of under-the-hood precision differences.
And that's my point too, that floating point will always subvert your expectations, particularly such simplistic notions of equality, which is why you should always use approxEqual or it's analogues.
May 19 2016
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Thursday, 19 May 2016 at 11:33:38 UTC, Joakim wrote:
 Computer scientists are no good if they don't know any science.
Even the computer scientists that does not know any science are infinitely better than those who refuse to read papers and debate on a rational level. Blind D zealotry at its finest.
May 19 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 19.05.2016 08:04, Joakim wrote:
 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand.
Argh... // ... void main(){ //double[] data=[1e16,1,-9e15]; import std.range; double[] data=1e16~repeat(1.0,100000000).array~(-9e15); import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } dmd -run kahanDemo.d 1000000000000000.000000 1000000100000000.000000 1000000000000000.000000 dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Better? Obviously there is more structure in the data that I invent manually than in a real test case where it would go wrong. The problems carry over though.
 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting there.
 ...
This is implicitly assuming a development model where the programmer first writes down the computation as it would be correct in the real number system and then naively replaces every operation by the rounding equivalent and hopes for the best. It is a useful rule if that is what you're doing. One might be doing something else. Consider the following paper for an example where the last bit in the significant actually carries useful information for many of the values used in the program. http://www.jaist.ac.jp/~s1410018/papers/qd.pdf
 In this case, not increasing precision gets the more accurate result,
 but other examples could be constructed that _heavily_ favor increasing
 precision.
Sure. In such cases, you should use higher precision. What is the problem? This is already supported (the compiler is not allowed to use lower precision than requested).
 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that far being
 accurate or "reliable."
IEEE floating point has well-defined behaviour and there is absolutely nothing wrong with code that delivers more accurate results just because it is actually aware of the actual semantics of the operations being carried out.
May 19 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 On 19.05.2016 08:04, Joakim wrote:
 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of useful 
 bits in
 half or more! It is not unusual, I have actually run into 
 those
 problems in the past, and it can break an algorithm that is 
 in Phobos
 today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand.
Argh... // ... void main(){ //double[] data=[1e16,1,-9e15]; import std.range; double[] data=1e16~repeat(1.0,100000000).array~(-9e15); import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } dmd -run kahanDemo.d 1000000000000000.000000 1000000100000000.000000 1000000000000000.000000 dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Better? Obviously there is more structure in the data that I invent manually than in a real test case where it would go wrong. The problems carry over though.
I looked over your code a bit. If I define sum and c as reals in "kahanBroken" at runtime, this problem goes away. Since that's what the CTFE rule is actually doing, ie extending all floating-point to reals at compile-time, I don't see what you're complaining about. Try it, run even your original naive summation algorithm through CTFE and it will produce the result you want: enum double[] ctData=[1e16,1,-9e15]; enum ctSum = sum(ctData); writefln("%f", ctSum);
 As Don's talk pointed out,
 all floating-point calculations will see loss of precision 
 starting there.
 ...
This is implicitly assuming a development model where the programmer first writes down the computation as it would be correct in the real number system and then naively replaces every operation by the rounding equivalent and hopes for the best.
No, it is intrinsic to any floating-point calculation.
 It is a useful rule if that is what you're doing. One might be 
 doing something else. Consider the following paper for an 
 example where the last bit in the significant actually carries 
 useful information for many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf
Did you link to the wrong paper? ;) I skimmed it and that paper explicitly talks about error bounds all over the place. The only mention of "the last bit" is when they say they calculated their constants in arbitrary precision before rounding them for runtime use, which is ironically similar to what Walter suggested doing for D's CTFE also.
 In this case, not increasing precision gets the more accurate 
 result,
 but other examples could be constructed that _heavily_ favor 
 increasing
 precision.
Sure. In such cases, you should use higher precision. What is the problem? This is already supported (the compiler is not allowed to use lower precision than requested).
I'm not the one with the problem, you're the one complaining.
 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that 
 far being
 accurate or "reliable."
IEEE floating point has well-defined behaviour and there is absolutely nothing wrong with code that delivers more accurate results just because it is actually aware of the actual semantics of the operations being carried out.
You just made the case for Walter doing what he did. :)
May 20 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 20.05.2016 11:14, Joakim wrote:
 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 On 19.05.2016 08:04, Joakim wrote:
 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand.
Argh... // ... void main(){ //double[] data=[1e16,1,-9e15]; import std.range; double[] data=1e16~repeat(1.0,100000000).array~(-9e15); import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } dmd -run kahanDemo.d 1000000000000000.000000 1000000100000000.000000 1000000000000000.000000 dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Better? Obviously there is more structure in the data that I invent manually than in a real test case where it would go wrong. The problems carry over though.
I looked over your code a bit. If I define sum and c as reals in "kahanBroken" at runtime, this problem goes away.
Yes. That's absolutely obvious, and I have noted it before, but thanks. Maybe try to understand why this problem occurs in the first place.
 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point to reals at
 compile-time, I don't see what you're complaining about.  Try it, run
 even your original naive summation algorithm through CTFE and it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...
This example wasn't specifically about CTFE, but just imagine that only part of the computation is done at CTFE, all local variables are transferred to runtime and the computation is completed there.
 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting
 there.
 ...
This is implicitly assuming a development model where the programmer first writes down the computation as it would be correct in the real number system and then naively replaces every operation by the rounding equivalent and hopes for the best.
No, it is intrinsic to any floating-point calculation. ...
How do you even define accuracy if you don't specify an infinitely precise reference result?
 It is a useful rule if that is what you're doing. One might be doing
 something else. Consider the following paper for an example where the
 last bit in the significant actually carries useful information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf
Did you link to the wrong paper? ;)
No. That paper uses multiple doubles per approximated real value to implement arithmetic that is more precise than using just plain doubles. If any bit in the first double is off, this is no better than using a single double.
 I skimmed it and that paper
 explicitly talks about error bounds all over the place.
It is enough to read the abstract to figure out what the problem is. This demonstrates a non-contrived case where CTFE using enhanced precision throughout can break your program. Compute something as a double-double at compile-time, and when it is transferred to runtime you lose all the nice extra precision, because bits in the middle of the (conceptual) mantissa are lost.
 The only mention of "the last bit" is
This part is actually funny. Thanks for the laugh. :-) I was going to say that your text search was too naive, but then I double-checked your claim and there are actually two mentions of "the last bit", and close by to the other mention, the paper says that "the first double a_0 is a double-precision approximation to the number a, accurate to almost half an ulp."
 when they say they calculated their
 constants in arbitrary precision before rounding them for runtime use,
 which is ironically similar to what Walter suggested doing for D's CTFE
 also.
 ...
Nothing "ironic" about that. It is sometimes a good idea and I can do this explicitly and make sure the rounding is done correctly, just like they did. Also, it is a lot more flexible if I can specify the exact way the computation is done and the result is rounded. 80 bits might not be enough anyway. There is no reason for the language to apply potentially incorrect "hand holding" here. Again, please understand that my point is not that lower precision is better. My point is that doing the same thing in every context and allowing the programmer to specify what happens is better.
 In this case, not increasing precision gets the more accurate result,
 but other examples could be constructed that _heavily_ favor increasing
 precision.
Sure. In such cases, you should use higher precision. What is the problem? This is already supported (the compiler is not allowed to use lower precision than requested).
I'm not the one with the problem, you're the one complaining. ...
So you see no problem with my requested semantics for the built-in floating point types?
 In fact, almost any real-world, non-toy calculation would
 favor it.

 In any case, nobody should depend on the precision out that far being
 accurate or "reliable."
IEEE floating point has well-defined behaviour and there is absolutely nothing wrong with code that delivers more accurate results just because it is actually aware of the actual semantics of the operations being carried out.
You just made the case for Walter doing what he did. :)
No.
May 20 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:
 On 20.05.2016 11:14, Joakim wrote:
 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 On 19.05.2016 08:04, Joakim wrote:
 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of 
 useful bits in
 half or more! It is not unusual, I have actually run into 
 those
 problems in the past, and it can break an algorithm that is 
 in Phobos
 today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand.
Argh... // ... void main(){ //double[] data=[1e16,1,-9e15]; import std.range; double[] data=1e16~repeat(1.0,100000000).array~(-9e15); import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } dmd -run kahanDemo.d 1000000000000000.000000 1000000100000000.000000 1000000000000000.000000 dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Better? Obviously there is more structure in the data that I invent manually than in a real test case where it would go wrong. The problems carry over though.
I looked over your code a bit. If I define sum and c as reals in "kahanBroken" at runtime, this problem goes away.
Yes. That's absolutely obvious, and I have noted it before, but thanks. Maybe try to understand why this problem occurs in the first place.
Yet you're the one arguing against increasing precision everywhere in CTFE.
 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point 
 to reals at
 compile-time, I don't see what you're complaining about.  Try 
 it, run
 even your original naive summation algorithm through CTFE and 
 it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...
This example wasn't specifically about CTFE, but just imagine that only part of the computation is done at CTFE, all local variables are transferred to runtime and the computation is completed there.
Why would I imagine that? And this whole discussion is about what happens if you change the precision of all variables to real when doing CTFE, so what's the point of giving an example that isn't "specifically" about that? And if any part of it is done at runtime using the algorithms you gave, which you yourself admit works fine if you use the right higher-precision types, you don't seem to have a point at all.
 As Don's talk pointed out,
 all floating-point calculations will see loss of precision 
 starting
 there.
 ...
This is implicitly assuming a development model where the programmer first writes down the computation as it would be correct in the real number system and then naively replaces every operation by the rounding equivalent and hopes for the best.
No, it is intrinsic to any floating-point calculation. ...
How do you even define accuracy if you don't specify an infinitely precise reference result?
There is no such thing as an infinitely precise result. All one can do is compute using even higher precision and compare it to lower precision.
 It is a useful rule if that is what you're doing. One might 
 be doing
 something else. Consider the following paper for an example 
 where the
 last bit in the significant actually carries useful 
 information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf
Did you link to the wrong paper? ;)
No. That paper uses multiple doubles per approximated real value to implement arithmetic that is more precise than using just plain doubles. If any bit in the first double is off, this is no better than using a single double.
 I skimmed it and that paper
 explicitly talks about error bounds all over the place.
It is enough to read the abstract to figure out what the problem is. This demonstrates a non-contrived case where CTFE using enhanced precision throughout can break your program. Compute something as a double-double at compile-time, and when it is transferred to runtime you lose all the nice extra precision, because bits in the middle of the (conceptual) mantissa are lost.
That is a very specific case where they're implementing higher-precision algorithms using lower-precision registers. If you're going to all that trouble, you should know not to blindly run the same code at compile-time.
 The only mention of "the last bit" is
This part is actually funny. Thanks for the laugh. :-) I was going to say that your text search was too naive, but then I double-checked your claim and there are actually two mentions of "the last bit", and close by to the other mention, the paper says that "the first double a_0 is a double-precision approximation to the number a, accurate to almost half an ulp."
Is there a point to this paragraph?
 when they say they calculated their
 constants in arbitrary precision before rounding them for 
 runtime use,
 which is ironically similar to what Walter suggested doing for 
 D's CTFE
 also.
 ...
Nothing "ironic" about that. It is sometimes a good idea and I can do this explicitly and make sure the rounding is done correctly, just like they did. Also, it is a lot more flexible if I can specify the exact way the computation is done and the result is rounded. 80 bits might not be enough anyway. There is no reason for the language to apply potentially incorrect "hand holding" here. Again, please understand that my point is not that lower precision is better. My point is that doing the same thing in every context and allowing the programmer to specify what happens is better.
I understand your point that sometimes the programmer wants more control. But as long as the way CTFE extending precision is consistently done and clearly communicated, those people can always opt out and do it some other way.
 In this case, not increasing precision gets the more 
 accurate result,
 but other examples could be constructed that _heavily_ favor 
 increasing
 precision.
Sure. In such cases, you should use higher precision. What is the problem? This is already supported (the compiler is not allowed to used lower precision than requested).
I'm not the one with the problem, you're the one complaining. ...
So you see no problem with my requested semantics for the built-in floating point types?
I think it's an extreme minority use case that may not merit the work, though I don't know how much work it would require. However, I also feel that way about Walter's suggested move to 128-bit CTFE, as dmd is x86-only anyway.
May 20 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 20.05.2016 13:32, Joakim wrote:
 On Friday, 20 May 2016 at 11:02:45 UTC, Timon Gehr wrote:
 On 20.05.2016 11:14, Joakim wrote:
 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 On 19.05.2016 08:04, Joakim wrote:
 On Wednesday, 18 May 2016 at 17:10:25 UTC, Timon Gehr wrote:
 It's not just slightly worse, it can cut the number of useful bits in
 half or more! It is not unusual, I have actually run into those
 problems in the past, and it can break an algorithm that is in Phobos
 today!
I wouldn't call that broken. Looking at the hex output by replacing %f with %A in writefln, it appears the only differences in all those results is the last byte in the significand.
Argh... // ... void main(){ //double[] data=[1e16,1,-9e15]; import std.range; double[] data=1e16~repeat(1.0,100000000).array~(-9e15); import std.stdio; writefln("%f",sum(data)); // baseline writefln("%f",kahan(data)); // kahan writefln("%f",kahanBroken(data)); // broken kahan } dmd -run kahanDemo.d 1000000000000000.000000 1000000100000000.000000 1000000000000000.000000 dmd -m32 -O -run kahanDemo.d 1000000000000000.000000 1000000000000000.000000 1000000000000000.000000 Better? Obviously there is more structure in the data that I invent manually than in a real test case where it would go wrong. The problems carry over though.
I looked over your code a bit. If I define sum and c as reals in "kahanBroken" at runtime, this problem goes away.
Yes. That's absolutely obvious, and I have noted it before, but thanks. Maybe try to understand why this problem occurs in the first place.
Yet you're the one arguing against increasing precision everywhere in CTFE. ...
High precision is usually good (use high precision, up to arbitrary precision or even symbolic arithmetic whenever it improves your results and you can afford it). *Implicitly* increasing precision during CTFE is bad. *Explicitly* using higher precision during CTFE than at running time /may or may not/ be good. In case it is good, there is often no reason to stop at 80 bits.
 Since that's what the
 CTFE rule is actually doing, ie extending all floating-point to reals at
 compile-time, I don't see what you're complaining about.  Try it, run
 even your original naive summation algorithm through CTFE and it will
 produce the result you want:

 enum double[] ctData=[1e16,1,-9e15];
 enum ctSum = sum(ctData);
 writefln("%f", ctSum);
 ...
This example wasn't specifically about CTFE, but just imagine that only part of the computation is done at CTFE, all local variables are transferred to runtime and the computation is completed there.
Why would I imagine that?
Because that's the most direct way to go from that example to one where implicit precision enhancement during CTFE only is bad.
 And this whole discussion is about what
 happens if you change the precision of all variables to real when doing
 CTFE, so what's the point of giving an example that isn't "specifically"
 about that?
 ...
Walter's request wasn't specifically about that, and it's the first thing I came up with:
 On 17.05.2016 23:07, Walter Bright wrote:
 I'd like to see an example of double rounding "completely defeating" an
 algorithm,
I don't have infinite amounts of time. The (significant) time spent writing up all of this competes with time spent doing things that are guaranteed to yield (often immediate) tangible benefits.
 And if any part of it is done at runtime using the algorithms you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,
What's "right" about them? That the compiler will not implicitly transform some of them to even higher precision in order to break the algorithm again? (I don't think that is even guaranteed.)
 you don't seem to have a point at all.
 ...
Be assured that I have a point. If you spend some time reading, or ask some constructive questions, I might be able to get it across to you. Otherwise, we might as well stop arguing.
 As Don's talk pointed out,
 all floating-point calculations will see loss of precision starting
 there.
 ...
This is implicitly assuming a development model where the programmer first writes down the computation as it would be correct in the real number system and then naively replaces every operation by the rounding equivalent and hopes for the best.
No, it is intrinsic to any floating-point calculation. ...
How do you even define accuracy if you don't specify an infinitely precise reference result?
There is no such thing as an infinitely precise result. All one can do is compute using even higher precision and compare it to lower precision. ...
If I may ask, how much mathematics have you been exposed to?
 It is a useful rule if that is what you're doing. One might be doing
 something else. Consider the following paper for an example where the
 last bit in the significant actually carries useful information for
 many of the values used in the program.

 http://www.jaist.ac.jp/~s1410018/papers/qd.pdf
Did you link to the wrong paper? ;)
No. That paper uses multiple doubles per approximated real value to implement arithmetic that is more precise than using just plain doubles. If any bit in the first double is off, this is no better than using a single double.
 I skimmed it and that paper
 explicitly talks about error bounds all over the place.
It is enough to read the abstract to figure out what the problem is. This demonstrates a non-contrived case where CTFE using enhanced precision throughout can break your program. Compute something as a double-double at compile-time, and when it is transferred to runtime you lose all the nice extra precision, because bits in the middle of the (conceptual) mantissa are lost.
That is a very specific case where they're implementing higher-precision algorithms using lower-precision registers.
If I argue in the abstract, people request examples. If I provide examples, people complain that they are too specific.
 If you're going to all that
 trouble, you should know not to blindly run the same code at compile-time.
 ...
The point of CTFE is to be able to run any code at compile-time that adheres to a well-defined set of restrictions. Not using floating point is not one of them.
 The only mention of "the last bit" is
This part is actually funny. Thanks for the laugh. :-) I was going to say that your text search was too naive, but then I double-checked your claim and there are actually two mentions of "the last bit", and close by to the other mention, the paper says that "the first double a_0 is a double-precision approximation to the number a, accurate to almost half an ulp."
Is there a point to this paragraph?
I don't think further explanations are required here. Maybe be more careful next time.
 when they say they calculated their
 constants in arbitrary precision before rounding them for runtime use,
 which is ironically similar to what Walter suggested doing for D's CTFE
 also.
 ...
Nothing "ironic" about that. It is sometimes a good idea and I can do this explicitly and make sure the rounding is done correctly, just like they did. Also, it is a lot more flexible if I can specify the exact way the computation is done and the result is rounded. 80 bits might not be enough anyway. There is no reason for the language to apply potentially incorrect "hand holding" here. Again, please understand that my point is not that lower precision is better. My point is that doing the same thing in every context and allowing the programmer to specify what happens is better.
I understand your point that sometimes the programmer wants more control.
Thanks!
 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,
It never will be clearly communicated to everyone and it will also hit people by accident who would have been aware of it. What is so incredibly awesome about /implicit/ 80 bit precision as to justify the loss of control? If I want to use high precision for certain constants precomputed at compile time, I can do it just as well, possibly even at more than 80 bits such as to actually obtain accuracy up to the last bit. Also, maybe I will need to move the computation to startup at runtime some time in the future because of some CTFE limitation, and then the additional implicit gain from 80 bit precision will be lost and cause a regression. The compiler just has no way to guess what precision is actually needed for each operation.
 those people can always opt out and do it some other way.
 ...
Sure, but now they need to _carefully_ maintain different implementations for CTFE and runtime, for an ugly misfeature. It's a silly magic trick that is not actually very useful and prone to errors.
May 21 2016
parent reply Joakim <dlang joakim.fea.st> writes:
Sorry, I stopped reading this thread after my last response, as I 
felt I was wasting too much time on this discussion, so I didn't 
read your response till now.

On Saturday, 21 May 2016 at 14:38:20 UTC, Timon Gehr wrote:
 On 20.05.2016 13:32, Joakim wrote:
 Yet you're the one arguing against increasing precision 
 everywhere in CTFE.
 ...
High precision is usually good (use high precision, up to arbitrary precision or even symbolic arithmetic whenever it improves your results and you can afford it). *Implicitly* increasing precision during CTFE is bad. *Explicitly* using higher precision during CTFE than at running time /may or may not/ be good. In case it is good, there is often no reason to stop at 80 bits.
It is not "implicitly increasing," Walter has said it will always be done for CTFE, ie it is explicit behavior for all compile-time calculation. And he agrees with you about not stopping at 80 bits, which is why he wanted to increase the precision of compile-time calculation even more.
 This example wasn't specifically about CTFE, but just imagine 
 that
 only part of the computation is done at CTFE, all local 
 variables are
 transferred to runtime and the computation is completed there.
Why would I imagine that?
Because that's the most direct way to go from that example to one where implicit precision enhancement during CTFE only is bad.
Obviously, but you still have not said why one would need to do that in some real situation, which is what I was asking for.
 And if any part of it is done at runtime using the algorithms 
 you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,
What's "right" about them? That the compiler will not implicitly transform some of them to even higher precision in order to break the algorithm again? (I don't think that is even guaranteed.)
What's right is that their precision is high enough to possibly give you the accuracy you want, and increasing their precision will only better that.
 you don't seem to have a point at all.
 ...
Be assured that I have a point. If you spend some time reading, or ask some constructive questions, I might be able to get it across to you. Otherwise, we might as well stop arguing.
I think you don't really have a point, as your argumentation and examples are labored.
 No, it is intrinsic to any floating-point calculation.
 ...
How do you even define accuracy if you don't specify an infinitely precise reference result?
There is no such thing as an infinitely precise result. All one can do is compute using even higher precision and compare it to lower precision. ...
If I may ask, how much mathematics have you been exposed to?
I suspect a lot more than you have. Note that I'm talking about calculation and compute, which can only be done at finite precision. One can manipulate symbolic math with all kinds of abstractions, but once you have to insert arbitrarily but finitely precise inputs and _compute_ outputs, you have to round somewhere for any non-trivial calculation.
 That is a very specific case where they're implementing 
 higher-precision
 algorithms using lower-precision registers.
If I argue in the abstract, people request examples. If I provide examples, people complain that they are too specific.
Yes, and? The point of providing examples is to illustrate a general need with a specific case. If your specific case is too niche, it is not a general need, ie the people you're complaining about can make both those statements and still make sense.
 If you're going to all that
 trouble, you should know not to blindly run the same code at 
 compile-time.
 ...
The point of CTFE is to be able to run any code at compile-time that adheres to a well-defined set of restrictions. Not using floating point is not one of them.
My point is that potentially not being able to use CTFE for floating-point calculation that is highly specific to the hardware is a perfectly reasonable restriction.
 The only mention of "the last bit" is
This part is actually funny. Thanks for the laugh. :-) I was going to say that your text search was too naive, but then I double-checked your claim and there are actually two mentions of "the last bit", and close by to the other mention, the paper says that "the first double a_0 is a double-precision approximation to the number a, accurate to almost half an ulp."
Is there a point to this paragraph?
I don't think further explanations are required here. Maybe be more careful next time.
Not required because you have some unstated assumptions that we are supposed to read from your mind? Specifically, you have not said why doing the calculation of that "double-precision approximation" at a higher precision and then rounding would necessarily throw their algorithms off.
 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,
It never will be clearly communicated to everyone and it will also hit people by accident who would have been aware of it. What is so incredibly awesome about /implicit/ 80 bit precision as to justify the loss of control? If I want to use high precision for certain constants precomputed at compile time, I can do it just as well, possibly even at more than 80 bits such as to actually obtain accuracy up to the last bit.
On the other hand, what is so bad about CTFE-calculated constants being computed at a higher precision and then rounded down? Almost any algorithm would benefit from that.
 Also, maybe I will need to move the computation to startup at 
 runtime some time in the future because of some CTFE 
 limitation, and then the additional implicit gain from 80 bit 
 precision will be lost and cause a regression. The compiler 
 just has no way to guess what precision is actually needed for 
 each operation.
Another scenario that I find far-fetched.
 those people can always opt out and do it some other way.
 ...
Sure, but now they need to _carefully_ maintain different implementations for CTFE and runtime, for an ugly misfeature. It's a silly magic trick that is not actually very useful and prone to errors.
I think the idea is to give compile-time calculations a boost in precision and accuracy, thus improving the constants computed at compile-time for almost every runtime algorithm. There may be some algorithms that have problems with this, but I think Walter and I are saying they're so few not to worry about, ie the benefits greatly outweigh the costs.
Aug 22 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 22.08.2016 20:26, Joakim wrote:
 Sorry, I stopped reading this thread after my last response, as I felt I
 was wasting too much time on this discussion, so I didn't read your
 response till now.
 ...
No problem. Would have been fine with me if it stayed that way.
 On Saturday, 21 May 2016 at 14:38:20 UTC, Timon Gehr wrote:
 On 20.05.2016 13:32, Joakim wrote:
 Yet you're the one arguing against increasing precision everywhere in
 CTFE.
 ...
High precision is usually good (use high precision, up to arbitrary precision or even symbolic arithmetic whenever it improves your results and you can afford it). *Implicitly* increasing precision during CTFE is bad. *Explicitly* using higher precision during CTFE than at running time /may or may not/ be good. In case it is good, there is often no reason to stop at 80 bits.
It is not "implicitly increasing,"
Yes it is. I don't state anywhere that I want the precision to increase. The default assumption is that CTFE behaves as (close to as reasonably possible to) runtime execution.
 Walter has said it will always be
 done for CTFE, ie it is explicit behavior for all compile-time
 calculation.
Well, you can challenge the definition of words I am using if you want, but what's the point?
  And he agrees with you about not stopping at 80 bits,
 which is why he wanted to increase the precision of compile-time
 calculation even more.
 ...
I'd rather not think of someone reaching that conclusion as agreeing with me.
 This example wasn't specifically about CTFE, but just imagine that
 only part of the computation is done at CTFE, all local variables are
 transferred to runtime and the computation is completed there.
Why would I imagine that?
Because that's the most direct way to go from that example to one where implicit precision enhancement during CTFE only is bad.
Obviously, but you still have not said why one would need to do that in some real situation, which is what I was asking for. ...
It seems you think your use cases are real, but mine are not, so there is no way to give you a "real" example. I can just hope that Murphy's law strikes and you eventually run into the problems yourself.
 And if any part of it is done at runtime using the algorithms you gave,
 which you yourself admit works fine if you use the right
 higher-precision types,
What's "right" about them? That the compiler will not implicitly transform some of them to even higher precision in order to break the algorithm again? (I don't think that is even guaranteed.)
What's right is that their precision is high enough to possibly give you the accuracy you want, and increasing their precision will only better that. ...
I have explained why this is not true. (There is another explanation further below.)
 No, it is intrinsic to any floating-point calculation.
 ...
How do you even define accuracy if you don't specify an infinitely precise reference result?
There is no such thing as an infinitely precise result. All one can do is compute using even higher precision and compare it to lower precision. ...
If I may ask, how much mathematics have you been exposed to?
I suspect a lot more than you have.
I would not expect anyone familiar with the real number system to make a remark like "there is no such thing as an infinitely precise result".
 Note that I'm talking about
 calculation and compute, which can only be done at finite precision.
I wasn't, and it was my post pointing out the implicit assumption that floating point algorithms are thought of as operating on real numbers that started this subthread, if you remember. Then you said that my point was untrue without any argumentation, and I asked a very specific question in order to figure out how you reached your conclusion. Then you wrote a comment that didn't address my question at all and was obviously untrue from where I stood. Therefore I suspected that we might be using incompatible terminology, hence I asked how familiar you are with mathematical language, which you didn't answer either.
 One can manipulate symbolic math with all kinds of abstractions, but
 once you have to insert arbitrarily but finitely precise inputs and
 _compute_ outputs, you have to round somewhere for any non-trivial
 calculation.
 ...
You don't need to insert any concrete values to make relevant definitions and draw conclusions. My question was how you define accuracy, because this is crucial for understanding and/or refuting your point. It's a reasonable question that you ought to be able to answer if you use the term in an argument repeatedly.
 That is a very specific case where they're implementing higher-precision
 algorithms using lower-precision registers.
If I argue in the abstract, people request examples. If I provide examples, people complain that they are too specific.
Yes, and? The point of providing examples is to illustrate a general need with a specific case. If your specific case is too niche, it is not a general need, ie the people you're complaining about can make both those statements and still make sense. ...
I think the problem is that they don't see the general need from the example.
 If you're going to all that
 trouble, you should know not to blindly run the same code at
 compile-time.
 ...
The point of CTFE is to be able to run any code at compile-time that adheres to a well-defined set of restrictions. Not using floating point is not one of them.
My point is that potentially not being able to use CTFE for floating-point calculation that is highly specific to the hardware is a perfectly reasonable restriction. ...
I meant that the restriction is not enforced by the language definition. I.e. it is not a compile-time error to compute with built-in floating point types in CTFE. Anyway, it is unfortunate but true that performance requirements might make it necessary to allow the results to be slightly hardware-specific; I agree that some compromises might be necessary. Arbitrarily using higher precision even in cases where the target hardware actually supports all features of IEEE floats and doubles does not seem like a good compromise though, it's completely unforced.
 The only mention of "the last bit" is
This part is actually funny. Thanks for the laugh. :-) I was going to say that your text search was too naive, but then I double-checked your claim and there are actually two mentions of "the last bit", and close by to the other mention, the paper says that "the first double a_0 is a double-precision approximation to the number a, accurate to almost half an ulp."
Is there a point to this paragraph?
I don't think further explanations are required here. Maybe be more careful next time.
Not required because you have some unstated assumptions that we are supposed to read from your mind?
Because anyone with a suitable pdf reader can verify that "the last bit" is mentioned twice inside that pdf document, and that the mention that you didn't see supports my point.
 Specifically, you have not said why
 doing the calculation of that "double-precision approximation" at a
 higher precision and then rounding would necessarily throw their
 algorithms off.
 ...
I did somewhere in this thread. (Using ASCII-art graphics even.) Basically, the number is represented using two doubles with a non-overlapping mantissa. I'll try to explain using a decimal floating-point type to maybe illustrate it better. E.g. assume that the higher-precision type has a 4-digit mantissa, and the lower-precision type has a 3-digit mantissa: The computation could have resulted in the double-double (1234e2, 56.78) representing the number 123456.78 (which is the exact sum of the two components.) If we now round both components to lower precision independently, we are left with (123e3, 56.7) which represents the number 123056.7, which has only 3 accurate mantissa digits. If OTOH, we had used the lower-precision type from the start, we would get a more accurate result, such as (123e3, 457) representing the number 123457. This might be slightly counter-intuitive, but it is not that uncommon for floating point-specific algorithms to actually rely on floating point specifics. Here, the issue is that the compiler has no way to know the correct way to transform the set of higher-precision floating point numbers to a corresponding set of lower-precision floating point numbers; it does not know how the values are actually interpreted by the program.
 But as long as the way CTFE extending precision is
 consistently done and clearly communicated,
It never will be clearly communicated to everyone and it will also hit people by accident who would have been aware of it. What is so incredibly awesome about /implicit/ 80 bit precision as to justify the loss of control? If I want to use high precision for certain constants precomputed at compile time, I can do it just as well, possibly even at more than 80 bits such as to actually obtain accuracy up to the last bit.
On the other hand, what is so bad about CTFE-calculated constants being computed at a higher precision and then rounded down? Almost any algorithm would benefit from that. ...
Some will subtly break, for some it won't matter and the others will work for reasons mostly hidden to the programmer and might therefore break later. Sometimes the programmer is aware of the funny language semantics and exploiting it cleverly, using 'float' during CTFE deliberately for performing 80-bit computations, confusing readers about the actual precision being used.
 Also, maybe I will need to move the computation to startup at runtime
 some time in the future because of some CTFE limitation, and then the
 additional implicit gain from 80 bit precision will be lost and cause
 a regression. The compiler just has no way to guess what precision is
 actually needed for each operation.
Another scenario that I find far-fetched. ...
Well, it's not. (The CTFE limitation could be e.g. performance.) Basically, anytime that a programmer has wrong assumptions about why their code works correctly, this is slightly dangerous. It's not a good thing if the compiler tries to outsmart the programmer, because the compiler is not (supposed to be) smarter than the programmer.
 those people can always opt out and do it some other way.
 ...
Sure, but now they need to _carefully_ maintain different implementations for CTFE and runtime, for an ugly misfeature. It's a silly magic trick that is not actually very useful and prone to errors.
I think the idea is to give compile-time calculations a boost in precision and accuracy, thus improving the constants computed at compile-time for almost every runtime algorithm. There may be some algorithms that have problems with this, but I think Walter and I are saying they're so few not to worry about, ie the benefits greatly outweigh the costs.
There are no benefits, because I can just explicitly compute at the precision I need myself, and I would prefer others to do the same, such that I have some clues about their reasoning when reading their code. Give me what I ask for. If you think I asked for the wrong thing, give me that wrong thing. If it is truly the wrong thing, I will see it and fix it. If you still disagree, that's fine, just don't claim that I don't have a point, thanks.
Aug 22 2016
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 
 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000

 Better?
Ignore if you think it's not relevant. I am a bit lost in this thread. I thought this was about floating point CTFE, but I think what you are seeing here (in the 2nd case) is the DMD optimizer allowing reassociation of fp expressions? LDC does not allow that (per default) and produces the 1000000100000000.000000 result also with optimization on. -Johan
May 20 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 20.05.2016 14:34, Johan Engelen wrote:
 On Thursday, 19 May 2016 at 18:22:48 UTC, Timon Gehr wrote:
 dmd -run kahanDemo.d
 1000000000000000.000000
 1000000100000000.000000
 1000000000000000.000000

 dmd -m32 -O -run kahanDemo.d
 1000000000000000.000000
 1000000000000000.000000
 1000000000000000.000000

 Better?
Ignore if you think it's not relevant. I am a bit lost in this thread. I thought this was about floating point CTFE,
Yes, this is how it started out. :) At some point, I made the comment that the compiler should never be allowed to use a higher precision than the one specified, which broadened the scope of the discussion somewhat. But the kinds of problems caused by using higher precision than requested can be similar for CTFE and runtime.
 but I think what you are seeing here (in the 2nd case) is the DMD
 optimizer allowing reassociation of fp expressions?
I don't believe it does (Walter knows /very/ well that a+(b+c) is not the same as (a+b)+c), but I have not looked at the assembly output. The result becomes wrong when 80 bit precision is used behind the scenes for some (but not all) of the computation, so this could be the explanation.
 LDC does not allow
 that (per default) and produces the 1000000100000000.000000 result also
 with optimization on.

 -Johan
Thanks!
May 21 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2016 10:10 AM, Timon Gehr wrote:
 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;
double y = roundToDouble(x - c);
         double t=sum+y;
double t = roundToDouble(sum + y);
         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }
Those are the only two points in the program that depend on rounding. If you're implementing Kahan, you'd know that, so there shouldn't be anything difficult about it. There's no reason to make the rest of the program suffer inaccuracies.
May 19 2016
next sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Thursday, 19 May 2016 at 07:09:30 UTC, Walter Bright wrote:
 On 5/18/2016 10:10 AM, Timon Gehr wrote:
 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;
double y = roundToDouble(x - c);
         double t=sum+y;
double t = roundToDouble(sum + y);
         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }
Those are the only two points in the program that depend on rounding. If you're implementing Kahan, you'd know that, so there shouldn't be anything difficult about it. There's no reason to make the rest of the program suffer inaccuracies.
People are trying to get across that, if they wanted to maximize accuracy, they would request the most precise type explicitly. D has 'real' for that. This thread has shown unequivocally that the semantics you are insisting on is bound to cause endless confusion and complaints.
May 19 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2016 12:49 AM, Max Samukha wrote:
 People are trying to get across that, if they wanted to maximize accuracy, they
 would request the most precise type explicitly. D has 'real' for that. This
 thread has shown unequivocally that the semantics you are insisting on is bound
 to cause endless confusion and complaints.
C++ programs do the same thing, and have for decades, and such behavior is allowed by the C++ Standard. People have been confused by and complained about floating point behavior at least since I started programming 40 years ago. I have not invented anything new here.
May 19 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 20.05.2016 08:25, Walter Bright wrote:
 On 5/19/2016 12:49 AM, Max Samukha wrote:
 People are trying to get across that, if they wanted to maximize
 accuracy, they
 would request the most precise type explicitly. D has 'real' for that.
 This
 thread has shown unequivocally that the semantics you are insisting on
 is bound
 to cause endless confusion and complaints.
C++ programs do the same thing, and have for decades, and such behavior is allowed by the C++ Standard. ...
And reported as a gcc bug regularly, and there seems to be a flag to turn x87-induced strangeness off completely. Also, the C99 standard rules out compiler-specific results. (And allows the program to query whether it is being compiled with the set of flags it requires: https://en.wikipedia.org/wiki/C99#IEEE.C2.A0754_floating_point_support .)
 People have been confused by and complained about floating point
 behavior at least since I started programming 40 years ago.
 ...
That's mostly an incidental problem. (Sharing language syntax with non-rounding operations, needing to map to mis-designed hardware, etc.)
 I have not invented anything new here.
You may not have invented anything new, but the old invention is broken and in the (slow and painful) process of getting fixed. There is no reason to build new things that use the same principle, as there is already a better way available.
May 21 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 19.05.2016 09:09, Walter Bright wrote:
 On 5/18/2016 10:10 AM, Timon Gehr wrote:
 double kahan(double[] arr){
     double sum = 0.0;
     double c = 0.0;
     foreach(x;arr){
         double y=x-c;
double y = roundToDouble(x - c);
Those two lines producing different results is unexpected, because you are explicitly saying that y is a double, and the first line also does implicit rounding (probably -- on all compilers and targets that will be relevant in the near future -- to double). It's obviously bad language design on multiple levels.
         double t=sum+y;
double t = roundToDouble(sum + y);
         c = (t-sum)-y;
         sum=t;
     }
     return sum;
 }
Those are the only two points in the program that depend on rounding.
If you say so. I would like to see an example that demonstrates that the first roundToDouble is required. Also, in case that the compiler chooses to internally use higher precision for all local variables in that program, the 'roundToDouble's you have inserted will reduce precision in comparison to the case where they are not inserted, reducing magical precision enhancement that would otherwise happen. (I.e., assuming that some of the ideas are valid and it is in fact desirable to have variable precision enhancement depending on target, if I find a precision bug I can fix by adding roundToDouble, this introduces a potential regression on other targets. And as none of it was guaranteed to work in the first place, the resulting mess is all my fault.)
 If you're implementing Kahan,
And if you are not? (I find the standard assumption that counterexamples to wrong statements are one-off special cases tiresome. This is not usually the case, even if you cannot construct other examples right away.)
 you'd know that,
I would, but what often happens in practice is that people don't. For example because they wouldn't expect roundToDouble to be required anywhere in code that only uses doubles. They are right.
 so there shouldn't be anything difficult about it.
 ...
https://issues.dlang.org/show_bug.cgi?id=13474 Creating useful programs in D shouldn't require knowing (or thinking about) an excess amount of D-specific implicit arcane details of the language semantics. Being easy to pick up and use is a major selling point for D. I'm not convinced there is nothing difficult about it. It's certainly a generator of incidental complexity. Also, error prone is not the same as difficult. Almost everybody makes some mistakes occasionally.
 There's no reason to make the rest of the program suffer inaccuracies.
Indeed there isn't. I'm not suggesting to allow using a precision lower than the one specified. If you are talking about programs that "benefit" from automatic extension of precision: do you think their authors are aware that e.g. their 32 bit release builds with gdc/ldc are producing wrong results?
May 19 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2016 1:26 PM, Timon Gehr wrote:
 Those two lines producing different results is unexpected, because you are
 explicitly saying that y is a double, and the first line also does implicit
 rounding (probably -- on all compilers and targets that will be relevant in the
 near future -- to double).
 [...]
 It's obviously bad language design on multiple levels.
Floating point semantics simply are not that simple, on any compiler, language or hardware. I recommend reading what the C/C++ Standards say about it, and look at the plethora of compiler switches for VC++/g++/clang++. The people who design these things are not fools, and there are good reasons for the way things are. Sorry, I don't agree.
 If you say so. I would like to see an example that demonstrates that the first
 roundToDouble is required.
That's beside the point. If there are spots in the program that require rounding, what is wrong with having to specify it? Why compromise speed & accuracy in the rest of the code that does not require it? (And yes, rounding in certain spots can and does compromise both speed and accuracy.)
 And if you are not? (I find the standard assumption that counterexamples to
 wrong statements are one-off special cases tiresome.
You might, but I've implemented a number of FP algorithms, and none of them required rounding to a lower precision. I've also seen many that failed due to premature rounding.
 Creating useful programs in D shouldn't require knowing (or thinking about) an
 excess amount of D-specific implicit arcane details of the language semantics.
 Being easy to pick up and use is a major selling point for D.
As pointed out innumerable times here, the same issues are in C and C++ (with even more variability!). When designing an algorithm that you know requires rounding in a certain place, you are already significantly beyond implementing a naive algorithm, and it would be time to start paying attention to how FP actually works. I'm curious if you know of any language that meets your requirements. (Java 1.0 did, but Sun was forced to abandon that.)
May 19 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 20.05.2016 08:12, Walter Bright wrote:
 On 5/19/2016 1:26 PM, Timon Gehr wrote:
 Those two lines producing different results is unexpected, because you
 are
 explicitly saying that y is a double, and the first line also does
 implicit
 rounding (probably -- on all compilers and targets that will be
 relevant in the
 near future -- to double).
> [...]
 It's obviously bad language design on multiple levels.
Floating point semantics simply are not that simple,
This is about how the language defines floating point semantics. They can be that simple if the specification says so.
 on any compiler,
gdc/ldc seem to get it right. David, Iain, is this the case?
 language
Anybody can invent a language and IEEE 754 semantics are possible to support. Recent hardware does it.
 or hardware. I recommend reading what the C/C++ Standards say
 about it, and look at the plethora of compiler switches for
 VC++/g++/clang++.
 ...
I don't really need to look at examples of how to do it wrong, but I will read whatever relevant link you can give me carefully. In D you don't need compiler switches to change the semantics of your code. We have more or less sane conditional compilation.
 The people who design these things are not fools,
I don't know those people, but I would like to know their thoughts about the current discussion.
 and there are good reasons for the way things are.
 ...
It is not unusual that there is some good reason for a design mistake.
 Sorry, I don't agree.

 If you say so. I would like to see an example that demonstrates that
 the first
 roundToDouble is required.
That's beside the point.
Absolutely not.
 If there are spots in the program that require
 rounding, what is wrong with having to specify it?
I explained what is wrong with it. What is wrong with having to specify the precision that is /required for your code to run correctly/?
 Why compromise speed & accuracy in the rest of the code
You don't compromise speed or accuracy if the rest of the code just correctly specifies the precision it should be run at.
 that does not require it? (And yes,
 rounding in certain spots can and does compromise both speed and accuracy.)
 ...
Because it is the only sane thing to do, as explained from many different angles in this thread. If it does compromise accuracy, that means the code is bad (it relies on precision enhancements that are not guaranteed to happen, hence it is hardly correct).
 And if you are not? (I find the standard assumption that
 counterexamples to
 wrong statements are one-off special cases tiresome.
You might, but I've implemented a number of FP algorithms, and none of them required rounding to a lower precision.
double x=..., z=...; double y=roundToDouble(x+z); Lower than what precision? It is not a 'lower' precision, it is the precision that was specified.
 I've also seen many that
 failed due to premature rounding.
 ...
Yes. Your results might be garbage more often if you run at too low precision. I and everyone else are very aware of this. I am not arguing against any of this. There seems to be a total communication failure here.
 Creating useful programs in D shouldn't require knowing (or thinking
 about) an
 excess amount of D-specific implicit arcane details of the language
 semantics.
 Being easy to pick up and use is a major selling point for D.
As pointed out innumerable times here, the same issues are in C and C++ (with even more variability!).
As also pointed out here, it does not matter. I'm *not* trying to make the point that C and C++ do it better or anything like that. (But well, in practice, the actual commonly used C and C++ implementations probably do get it right by default.)
 When designing an algorithm that you know
 requires rounding in a certain place, you are already significantly
 beyond implementing a naive algorithm, and it would be time to start
 paying attention to how FP actually works.

 I'm curious if you know of any language that meets your requirements.
 (Java 1.0 did, but Sun was forced to abandon that.)
x86_64 assembly language.
May 20 2016
parent HorseWrangler <horse wrangler.org> writes:
On Friday, 20 May 2016 at 11:22:48 UTC, Timon Gehr wrote:
 On 20.05.2016 08:12, Walter Bright wrote:
 I'm curious if you know of any language that meets your 
 requirements.
 (Java 1.0 did, but Sun was forced to abandon that.)
x86_64 assembly language.
Similar discussion for Rust: https://internals.rust-lang.org/t/pre-rfc-dealing-with-broken-floating-point/2673/1
May 20 2016
prev sibling next sibling parent reply Tobias =?UTF-8?B?TcO8bGxlcg==?= <troplin bluewin.ch> writes:
On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:
 On 5/19/2016 1:26 PM, Timon Gehr wrote:
 Those two lines producing different results is unexpected, 
 because you are
 explicitly saying that y is a double, and the first line also 
 does implicit
 rounding (probably -- on all compilers and targets that will 
 be relevant in the
 near future -- to double).
 [...]
 It's obviously bad language design on multiple levels.
Floating point semantics simply are not that simple, on any compiler, language or hardware. I recommend reading what the C/C++ Standards say about it, and look at the plethora of compiler switches for VC++/g++/clang++. The people who design these things are not fools, and there are good reasons for the way things are. Sorry, I don't agree.
Let me cite Prof. John L Gustafson from his Book "The End of Error" (it's worth reading): --- The lesson taught by all these attempts at clever hidden scratchpad work is that computer users want *consistent* answers, and want to be *permitted the trade-of between speed and accuracy* depending on the situation. There is a subtle arrogance in any computer system that takes the position "You asked for a quick, cheap, approximate calculation, but I know what is good for you and will instead do a better job even if it takes more time and storage and energy and gives you a different answer. You're welcome." ---
May 20 2016
parent reply Tobias M <troplin bluewin.ch> writes:
On Friday, 20 May 2016 at 12:32:40 UTC, Tobias Müller wrote:
 Let me cite Prof. John L Gustafson
Not "Prof." but "Dr.", sorry about that. Still an authority, though.
May 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
May 20 2016
next sibling parent reply Tobias =?UTF-8?B?TcO8bGxlcg==?= <troplin bluewin.ch> writes:
On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
Actually cited this *because* of you mentioning Kahan several times. And because you said "The people who design these things are not fools, and there are good reasons for the way things are." Obviously, the designer of x87 and IEEE 754 (which both explicitly allow such "magic" behind the back of the programmer) thinks this is a good thing. But after reading the book, I tend to agree with Gustafson. In the book this is elaborated more in detail, I just cited the conclusion.
May 21 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/21/2016 2:26 AM, Tobias Müller wrote:
 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
Actually cited this *because* of you mentioning Kahan several times. And because you said "The people who design these things are not fools, and there are good reasons for the way things are."
I meant two things by this: 1. Do the homework before disagreeing with someone who literally wrote the book and designed the hardware for it. 2. Complaining that the x87 is not IEEE compliant, when the guy that designed the x87 wrote the spec at the same time, suggests a misunderstanding the spec. I.e. again, gotta do the homework first. Dismissing several decades of FP designs, and every programming language, as being "obviously wrong" and "insane" is an extraordinary claim, and so requires extraordinary evidence. After all, what would your first thought be when a sophomore physics student tells you that Feynman got it all wrong? It's good to revisit existing dogma now and then, and challenge the underlying assumptions of it, but again, you gotta understand the existing dogma all the way down first. If you don't, you're very likely to miss something fundamental and produce a design that is less usable.
May 21 2016
next sibling parent reply Tobias M <troplin bluewin.ch> writes:
On Saturday, 21 May 2016 at 17:58:49 UTC, Walter Bright wrote:
 On 5/21/2016 2:26 AM, Tobias Müller wrote:
 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
Actually cited this *because* of you mentioning Kahan several times. And because you said "The people who design these things are not fools, and there are good reasons for the way things are."
I meant two things by this: 1. Do the homework before disagreeing with someone who literally wrote the book and designed the hardware for it. 2. Complaining that the x87 is not IEEE compliant, when the guy that designed the x87 wrote the spec at the same time, suggests a misunderstanding the spec. I.e. again, gotta do the homework first.
Sorry but this is a misrepresentation. I never claimed that the x87 doesn't conform to the IEEE standard. That's completely missing the point. Again.
 Dismissing several decades of FP designs, and every programming 
 language, as being "obviously wrong" and "insane" is an 
 extraordinary claim, and so requires extraordinary evidence.

 After all, what would your first thought be when a sophomore 
 physics student tells you that Feynman got it all wrong? It's 
 good to revisit existing dogma now and then, and challenge the 
 underlying assumptions of it, but again, you gotta understand 
 the existing dogma all the way down first.

 If you don't, you're very likely to miss something fundamental 
 and produce a design that is less usable.
The point is, that is IS possible to provide fairly reasonable and consistent semantics within the existing standards (C, C++, IEEE, ...). They provide a certain degree of freedom to accomodate for different hardware, but this doesn't mean that software should use this freedom to do arbitrary things. Regarding the decades of FP design, the initial edition of K&R C contained the following clause: "Notice that all floats in an expression are converted to double; all floating point arithmethic in C is done in double precision". That passus was removed quite quickly because users complained about it.
May 21 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/21/2016 11:36 AM, Tobias M wrote:
 Sorry but this is a misrepresentation. I never claimed that the x87 doesn't
 conform to the IEEE standard.
My point was directed to more than just you. Sorry I didn't make that clear.
 The point is, that is IS possible to provide fairly reasonable and consistent
 semantics within the existing standards (C, C++, IEEE, ...).
That implies what I propose, which is what many C/C++ compilers do, is unreasonable, inconsistent, not Standard compliant, and not IEEE. I.e. that the x87 is not conformant :-) Read the documentation on the FP switches for VC++, g++, clang, etc. You'll see there are tradeoffs. There is no "obvious, sane" way to do it. There just isn't.
 They provide a
 certain degree of freedom to accomodate for different hardware, but this
doesn't
 mean that software should use this freedom to do arbitrary things.
Nobody is suggesting doing arbitrary things, but to write portable fp, take into account what the Standard says rather than what your version of the compiler does with various default and semi-documented switches.
 Regarding the decades of FP design, the initial edition of K&R C contained the
 following clause:
 "Notice that all floats in an expression are converted to double; all floating
 point arithmethic in C is done in double precision".
 That passus was removed quite quickly because users complained about it.
It was changed to allow floats to be computed as floats, not require it. And the reason at the time, as I recall, was to get faster floating point ops, not because anyone desired reduced precision.
May 21 2016
parent Tobias M <troplin bluewin.ch> writes:
On Saturday, 21 May 2016 at 21:56:02 UTC, Walter Bright wrote:
 On 5/21/2016 11:36 AM, Tobias M wrote:
 Sorry but this is a misrepresentation. I never claimed that 
 the x87 doesn't
 conform to the IEEE standard.
My point was directed to more than just you. Sorry I didn't make that clear.
 The point is, that is IS possible to provide fairly reasonable 
 and consistent
 semantics within the existing standards (C, C++, IEEE, ...).
That implies what I propose, which is what many C/C++ compilers do, is unreasonable, inconsistent, not Standard compliant, and not IEEE. I.e. that the x87 is not conformant :-)
I'm trying to understand what you want to say here, but I just don't get it. Can you maybe formulate it differently?
 Read the documentation on the FP switches for VC++, g++, clang, 
 etc. You'll see there are tradeoffs. There is no "obvious, 
 sane" way to do it.

 There just isn't.
As I see it, the only real trade off is speed/optimization vs correctness.
 They provide a
 certain degree of freedom to accomodate for different 
 hardware, but this doesn't
 mean that software should use this freedom to do arbitrary 
 things.
Nobody is suggesting doing arbitrary things, but to write portable fp, take into account what the Standard says rather than what your version of the compiler does with various default and semi-documented switches.
https://gcc.gnu.org/wiki/FloatingPointMath Seems relatively straight forward to me and well documented to me... Dangerous optimizations like reordering expressions are all opt-in. Sure it's probably not 100% consistent across implementations/platforms, but it's also *that* bad. And it's certainly not an excuse to make it even worse. And yes, I think that in such an underspecified domain like FP, you cannot just rely on the standard but have to take the individual implementations into account. Again, this is not ideal, but let's not make it even worse.
 Regarding the decades of FP design, the initial edition of K&R 
 C contained the
 following clause:
 "Notice that all floats in an expression are converted to 
 double; all floating
 point arithmethic in C is done in double precision".
 That passus was removed quite quickly because users complained 
 about it.
It was changed to allow floats to be computed as floats, not require it. And the reason at the time, as I recall, was to get faster floating point ops, not because anyone desired reduced precision.
I don't think that anyone has argued that lower precision is better. But a compiler should just do what it is told, not trying to be too clever.
May 21 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 21.05.2016 19:58, Walter Bright wrote:
 On 5/21/2016 2:26 AM, Tobias Müller wrote:
 On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
Actually cited this *because* of you mentioning Kahan several times. And because you said "The people who design these things are not fools, and there are good reasons for the way things are."
I meant two things by this: 1. Do the homework before disagreeing with someone who literally wrote the book and designed the hardware for it. ...
Sure.
 2. Complaining that the x87 is not IEEE compliant, when the guy that
 designed the x87 wrote the spec at the same time, suggests a
 misunderstanding the spec.
Who claimed that the x87 is not IEEE compliant? Anyway, this is easy to resolve. IEEE 754-2008 requires FMA. x87 has no FMA. Also, in practice, it is used to produce non-compliant results, as it has a default mode of being used that gives you results that differ from the specified results for single and double precision sources and destinations. From IEEE 754-2008: "1.2 Purpose 1.2.0 This standard provides a method for computation with floating-point numbers that will yield the same result whether the processing is done in hardware, software, or a combination of the two." I.e. the only stated purpose of IEEE 754 is actually reproducibility. "shall indicates mandatory requirements strictly to be followed in order to conform to the standard and from which no deviation is permitted (“shall” means “is required to”" "3.1.2 Conformance 3.1.2 .0 A conforming implementation of any supported format shall provide means to initialize that format and shall provide conversions between that format and all other supported formats. A conforming implementation of a supported arithmetic format shall provide all the operations of this standard defined in Clause 5, for that format. A conforming implementation of a supported interchange format shall provide means to read and write that format using a specific encoding defined in this clause, for that format. A programming environment conforms to this standard, in a particular radix, by implementing one or more of the basic formats of that radix as both a supported arithmetic format and a supported interchange format." For the original IEEE 754-1985, the x87 seems to support all of those clauses for float, double and extended (if you use it in the right way, which is inefficient, as you need to spill the result to memory after each operation, and it is not the default way), but it also supports further operations that fulfill similar functions in a non-conforming manner, and compiler implementations use it that way. Another way to read it is that the x86 conforms by supporting float and double as interchange formats and extended as arithmetic format. What is the correct way to interpret the x87 as conforming?
 I.e. again, gotta do the homework first.

 Dismissing several decades of FP designs, and every programming
 language, as being "obviously wrong" and "insane"
No need to unduly generalize the scope of my claims.
 is an extraordinary claim,
Not really. There don't seem to be so many popular systems in computing that are not broken in one way or another. (And even when they are, they are a lot more useful than nothing. I'd rather have the x87 than no hardware floating point support at all.)
 and so requires extraordinary evidence.
 ...
There are several decades of experience with the x87 to draw from. SSE does not suffer from those issues anymore. This is because flaws in the design were identified and fixed. Why is this evidence not extraordinary enough?
 After all, what would your first thought be when a sophomore physics
 student tells you that Feynman got it all wrong?
It's an oddly unspecific statement (what about his work is wrong?) and it's a completely different case. Feynman getting everything wrong is not a popular sentiment in the Physics community AFAICT.
 It's good to revisit
 existing dogma now and then, and challenge the underlying assumptions of
 it, but again, you gotta understand the existing dogma all the way down
 first.
 ...
What parts of "existing dogma" justify all the problems that x87 causes? Also, if e.g.implicit 80 bit precision for CTFE floats truly is mandated by "floating point dogma" then "floating point dogma" clashes with "language design dogma".
 If you don't, you're very likely to miss something fundamental and
 produce a design that is less usable.
I haven't invented anything new either. SSE already fixes the issues. (But yes, it is probably hard to support systems that only have the x87 in a sane way. Tradeoffs might be necessary. Deliberately copying mistakes that x87 made to other contexts is not the right course of action though.)
May 21 2016
prev sibling next sibling parent Tobias M <troplin bluewin.ch> writes:
On Friday, 20 May 2016 at 22:22:57 UTC, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority, may I present Kahan who concurrently designed the IEEE 754 spec and the x87.
Since I'm just in the mood of appealing to authorities, let me cite Wikipedia: The argument from authority (Latin: argumentum ad verecundiam) also appeal to authority, is a common argument form which can be fallacious, *such as when an authority is cited on a topic outside their area of expertise, when the authority cited is not a true expert or if the cited authority is stating a contentious or controversial position.* (Emphasis is mine)
May 21 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 21.05.2016 00:22, Walter Bright wrote:
 On 5/20/2016 5:36 AM, Tobias M wrote:
 Still an authority, though.
If we're going to use the fallacy of appeal to authority,
Authorities are not always wrong, the fallacy is to argue that they are right *because* they are authorities. However, in this case, I don't think you have provided a quote from Kahan that is applicable to the case we have here. The Gustafson quote is formulated generally enough to apply to the current case directly.
 may I present
 Kahan who concurrently designed the IEEE 754 spec and the x87.
The x87 is by far not the slam-dunk design you seem to make it out to be. (And you don't need to take my word for it. Otherwise, why would hardware designers prefer to phase it out in favour of systems that don't do the implicit extra scratch space thing?) https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323 Bug report for this problem in gcc. The discussion raises an issue I wasn't aware of so far (or have forgotten about): Apparently the x87 does not even round to double correctly, it only rounds the mantissa but does not deal with out-of-range-exponents correctly. What a mess. http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html x87 experience report. http://www.vedetta.com/x87-fpu-php-bug-causes-infinite-loop-affected-websites-vulnerable-to-dos-via-php-get-function-2.2250738585072011e-308 DOS attack possible due to usage of x87 instructions. http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it Many complaints about x87, and some concerns about 80 bit floating point support in the future. http://www.vinc17.org/research/extended.en.html Good summary of some x87 problems. (Slightly broken English though.) http://www.realworldtech.com/physx87/ x87 instructions used to make the GPU look faster compared to the CPU. (Not /directly/ a x87 design problem.) http://www.cims.nyu.edu/~dbindel/class/cs279/87stack.pdf http://www.cims.nyu.edu/~dbindel/class/cs279/stack87.pdf Kahan criticizing the design of the register stack on the x87. I haven't found any statement from Kahan on gcc bug 323, but this would be really interesting, if anyone can find anything. I think there is ample evidence that x87 is ultimately a design failure, even though the intentions were good and based on real expertise.
May 21 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 21.05.2016 15:45, Timon Gehr wrote:
 On 21.05.2016 00:22, Walter Bright wrote:
 ...
 may I present
 Kahan who concurrently designed the IEEE 754 spec and the x87.
The x87 is by far not the slam-dunk design you seem to make it out to be. ...
https://hal.archives-ouvertes.fr/hal-00128124v5/document Check out section 5 for some convincing examples showing why the x87 is horrible.
May 21 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/21/2016 10:03 AM, Timon Gehr wrote:
 Check out section 5 for some convincing examples showing why the x87 is
horrible.
The polio vaccine winds up giving a handful of people polio, too. It's good to list traps for the unwary in FP usage. It's disingenuous to list only problems with one design and pretend there are no traps in another design.
May 21 2016
next sibling parent Fool <fool dlang.org> writes:
Reasons have been alleged. What's your final decision?
May 21 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 21.05.2016 20:14, Walter Bright wrote:
 On 5/21/2016 10:03 AM, Timon Gehr wrote:
 Check out section 5 for some convincing examples showing why the x87
 is horrible.
The polio vaccine winds up giving a handful of people polio, too. ...
People don't get vaccinated without consent.
 It's good to list traps for the unwary in FP usage. It's disingenuous to
 list only problems with one design and pretend there are no traps in
 another design.
Some designs are much better than others.
May 21 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Saturday, 21 May 2016 at 22:05:31 UTC, Timon Gehr wrote:
 On 21.05.2016 20:14, Walter Bright wrote:
 It's good to list traps for the unwary in FP usage. It's 
 disingenuous to
 list only problems with one design and pretend there are no 
 traps in
 another design.
Some designs are much better than others.
Indeed. There are actually _only_ problems with D's take on floating point. It even prevents implementing higher precision double-double and quad-double math libraries by error-correction techniques where you get 106 bit/212 bit mantissas: C++ w/2 x 64 bits adder and conservative settings --> GOOD ACCURACY/ ~106 significant bits: #include <iostream> int main() { const double a = 1.23456; const double b = 1.3e-18; double hi = a+b; double lo = -((a - ((a+b) - ((a+b) - a))) - (b + ((a+b) - a))); std::cout << hi << std::endl; // 1.23456 std::cout << lo << std::endl; // 1.3e-18 SUCCESS! std::cout << (hi-a) <<std::endl; // 0 } D w/2 x 64/80 bits adder --> BAD ACCURACY import std.stdio; void main() { const double a = 1.23456; const double b = 1.3e-18; double hi = a+b; double lo = -((a - ((a+b) - ((a+b) - a))) - (b + ((a+b) - a))); writeln(hi); // 1.23456 writeln(lo); // 2.60104e-18 FAILURE! writeln(hi-a); // 0 } Add to this that compiler-backed emulation of 128 bit floats are twice as fast as 80-bit floats in hardware... that's how incredibly slow it 80 bit floats are on modern hardware. I don't even understand why this is a topic as there is not one single rational to keep it the way it is. Not one.
Jun 06 2016
prev sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Friday, 20 May 2016 at 06:12:44 UTC, Walter Bright wrote:

 If you say so. I would like to see an example that 
 demonstrates that the first
 roundToDouble is required.
That's beside the point. If there are spots in the program that require rounding, what is wrong with having to specify it?
Because the programmer has already specified it with a type that requires rounding!
 Why compromise speed & accuracy in the rest of the code that 
 does not require it? (And yes, rounding in certain spots can 
 and does compromise both speed and accuracy.)
Accuracy and speed won't be compromised if the programmer chooses an appropriate type - 'real' for the highest precision supported by the implementation. BTW, you need to remove 'hardware' from the definition of 'real' in the spec.
May 20 2016
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 18.05.2016 19:10, Timon Gehr wrote:
 implementation-defined behaviour
Maybe that wasn't the right term (it's worse than that; I think the documentation of the implementation is not even required to tell you precisely what it does).
May 19 2016
prev sibling parent Fool <fool dlang.org> writes:
On Tuesday, 17 May 2016 at 21:07:21 UTC, Walter Bright wrote:
 [...] why an unusual case of producing a slightly worse answer 
 trumps the usual case of producing better answers.
'Sometimes worse' is not 'better', but that's not the point, here. Even if you managed to consistently produce not less accurate results: if the results computed at compile time differ from those computed at run time you cannot safely substitute run time calls with compile time calls without altering the result. Even slightest differences can lead to completely different outcomes. For example, if we are interested in the number of real roots of a quadratic polynomial we compute its discriminant D. There are no solutions if D is negative, there is one solution if D is zero and there are two solutions if D is positive. In a floating point environment the crucial test is often implemented with a certain tolerance and good implementations compute the discriminant with increased precision. In any case, the critical point is that we have to make a decision based on a single computed value. A difference of one unit in the last place can change the result. Note that this in not about correctness of the result but about reproducibility and substitutability across run time and compile time boundaries. Please make a decision, document it, and move on. Fool
May 21 2016
prev sibling parent reply Max Samukha <maxsamukha gmail.com> writes:
On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:

 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended 
 precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not 
 reasonable
 that CTFE has higher precision applied than the same code at 
 runtime.
 CTFE must give the exact same result as runtime execution of 
 the function.
You are not even guaranteed to get the same result on two different x86 implementations. AMD64: "The processor produces a floating-point result defined by the IEEE standard to be infinitely precise. This result may not be representable exactly in the destination format, because only a subset of the continuum of real numbers finds exact representation in any particular floating-point format."
May 15 2016
next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 15:49, Max Samukha via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:

 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the function.
You are not even guaranteed to get the same result on two different x86 implementations. AMD64: "The processor produces a floating-point result defined by the IEEE standard to be infinitely precise. This result may not be representable exactly in the destination format, because only a subset of the continuum of real numbers finds exact representation in any particular floating-point format."
I think that is to be interpreted that the result will be the closest representation possible (rounding up), and that's not fuzzy. I've never heard of an IEEE float unit that gives different results than another one...?
May 15 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 07:49, Max Samukha wrote:
 On Monday, 16 May 2016 at 04:02:54 UTC, Manu wrote:

 extended x = 1.3;
 x + y;

 If that code were to CTFE, I expect the CTFE to use extended precision.
 My point is, CTFE should surely follow the types and language
 semantics as if it were runtime generated code... It's not reasonable
 that CTFE has higher precision applied than the same code at runtime.
 CTFE must give the exact same result as runtime execution of the
 function.
You are not even guaranteed to get the same result on two different x86 implementations.
Without reading the x86 specification, I think it is safe to claim that you actually are guaranteed to get the same result.
 AMD64:

 "The processor produces a floating-point result defined by the IEEE
 standard to be infinitely precise.
 This result may not be representable exactly in the destination format,
 because only a subset of the
 continuum of real numbers finds exact representation in any particular
 floating-point format."
This just says that results of computations will need to be rounded to fit into constant-size storage.
May 16 2016
parent reply Max Samukha <maxsamukha gmail.com> writes:
On Monday, 16 May 2016 at 19:01:19 UTC, Timon Gehr wrote:

 You are not even guaranteed to get the same result on two 
 different x86
 implementations.
Without reading the x86 specification, I think it is safe to claim that you actually are guaranteed to get the same result.
Correct. Sorry for the noise.
 AMD64:

 "The processor produces a floating-point result defined by the 
 IEEE
 standard to be infinitely precise.
 This result may not be representable exactly in the 
 destination format,
 because only a subset of the
 continuum of real numbers finds exact representation in any 
 particular
 floating-point format."
This just says that results of computations will need to be rounded to fit into constant-size storage.
Right.
May 17 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 17.05.2016 20:09, Max Samukha wrote:
 On Monday, 16 May 2016 at 19:01:19 UTC, Timon Gehr wrote:

 You are not even guaranteed to get the same result on two different x86
 implementations.
Without reading the x86 specification, I think it is safe to claim that you actually are guaranteed to get the same result.
Correct. Sorry for the noise.
Interestingly, I was actually wrong: https://hal.archives-ouvertes.fr/hal-00128124v5/document I think that document is worth a read in any case. Page 18: "Note, also, that different processors within the same architecture can implement the same transcendental functions with different accuracies. We already noted the difference between the AMD-K5 and the K6 and following processors with respect to angular reduction. Intel also notes that the algorithms’ precision was improved between the 80387 / i486DX processors and the Pentium processors. [Int, 1997,§7.5.10]" The language should ideally not leak such differences to the user.
May 21 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 06:02, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 I'm not interested in C/C++, I gave some anecdotes where it's gone
 wrong for me too, but regardless; generally, they do match, and I
 can't think of a single modern example where that's not true. If you
 *select* fast-math, then you may generate code that doesn't match, but
 that's a deliberate selection.

 If I want 'real' math (in CTFE or otherwise), I will type "real". It
 is completely unreasonable to reinterpret the type that the user
 specified. CTFE should execute code the same way runtime would execute
 the code (without -ffast-math, and conformant ieee hardware). This is
 not a big ask.

 Incidentally, I made the mistake of mentioning this thread (due to my
 astonishment that CTFE ignores float types) out loud to my
 colleagues... and they actually started yelling violently out loud.
 One of them has now been on a 10 minute angry rant with elevated tone
 and waving his arms around about how he's been shafted by this sort
 behaviour so many times before. I wish I recorded it, I'd love to have
 submit it as evidence.
It isn't all bad. Most the time you'll never notice. :-) I can't think of a case of the top of my head where too much precision caused a surprise. It's always when there is too little: https://issues.dlang.org/show_bug.cgi?id=16026 And I think the it's about that time of the year when I remind people of gcc bug 323, and this lovely blog post. http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html
May 15 2016
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 06:48:19 UTC, Iain Buclaw wrote:
 I can't think of a case of the top of my head where too much 
 precision caused a surprise.  It's always when there is too 
 little:
Wrong. You obviously don't do much system level programming using floats.
May 16 2016
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 09:06, Ola Fosheim Grøstad via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 16 May 2016 at 06:48:19 UTC, Iain Buclaw wrote:
 I can't think of a case of the top of my head where too much precision
 caused a surprise.  It's always when there is too little:
Wrong. You obviously don't do much system level programming using floats.
Feel free to give me any bug report or example. None that you've done so far in this thread relate to CTFE.
May 16 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 07:16:41 UTC, Iain Buclaw wrote:
 Feel free to give me any bug report or example.  None that 
 you've done so far in this thread relate to CTFE.
Of course it is the same issue. Constant folding is "compile-time-function-evaluation", e.g. an expression is a function. Compiler internals does not change that.
May 16 2016
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 11:48 PM, Iain Buclaw via Digitalmars-d wrote:
 I can't think of a case of the top of my head where too much precision
 caused a surprise.  It's always when there is too little:

 https://issues.dlang.org/show_bug.cgi?id=16026

 And I think the it's about that time of the year when I remind people
 of gcc bug 323, and this lovely blog post.

 http://blog.jwhitham.org/2015/04/gcc-bug-323-journey-to-heart-of.html
Manu, read those! More: https://randomascii.wordpress.com/2012/03/21/intermediate-floating-point-precision/ Ironically, this Microsoft article argues for greater precision for intermediate calculations, although Microsoft ditched 80 bits: https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx
May 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 3:31 AM, Walter Bright wrote:
 Ironically, this Microsoft article argues for greater precision for
 intermediate calculations, although Microsoft ditched 80 bits:

 https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx
Do you have an explanation on why Microsoft ditched 80-bit floats? -- Andrei
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:30 AM, Andrei Alexandrescu wrote:
 On 5/16/16 3:31 AM, Walter Bright wrote:
 Ironically, this Microsoft article argues for greater precision for
 intermediate calculations, although Microsoft ditched 80 bits:

 https://msdn.microsoft.com/en-us/library/aa289157(VS.71).aspx
Do you have an explanation on why Microsoft ditched 80-bit floats? -- Andrei
At the time they were porting NT to some other architecture which didn't have 80 bits. I suspect they dumped it in order to enhance code compatibility between the x86 compiler and the other compiler. I think Microsoft tools had an internal mandate to make the differences between the machines as small as possible. The lack of 80 bit SIMD support likely gave it a good shove off the curb as well. Even dmd no longer generates x87 code for float/double for 64 bit targets, meaning no more 80 bit temporaries. It's kinda sad, really. The x87 was a triumph of engineering when it came out. The comments that "64 bits is all anyone will ever need" are not made by people who do numerical work, where one constantly battles catastrophic loss of precision. I've often wondered how NASA calculates trajectories out to Jupiter, because it has to be done iteratively, and that means cumulative loss of precision. I wrote some orbit software for fun in college, and catastrophic loss of precision would make the orbits go completely bonkers after just one orbit. I invented several schemes to fix it, but nothing worked. When I was doing numerical analysis for my you-know-what job, I had to invert matrices all the time. I only knew the math algorithm, and that would produce nonsense on anything larger than 14*14 or so using doubles. Better algorithms exist that compensate for the errors, but this was pre-internet and I didn't know where to get such a solution. Those two experiences shaped my attitudes about the value of precision, as well as the TTL boards I designed where the design had to still work if faster parts were swapped in.
May 16 2016
prev sibling next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 13 May 2016 at 07:12, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/12/2016 4:32 PM, Marco Leise wrote:
 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results
I've actually been thinking of writing a 128 bit float emulator, and then using that in the compiler internals to do all FP computation with.
No. Do not. I've worked on systems where the compiler and the runtime don't share floating point precisions before, and it was a nightmare.
I have some bad news for you about CTFE then. This already happens in DMD even though float is not emulated. :-o
May 13 2016
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 14 May 2016 at 00:00, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 13 May 2016 at 07:12, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 13 May 2016 at 11:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/12/2016 4:32 PM, Marco Leise wrote:
 - Unless CTFE uses soft-float implementation, depending on
   compiler and flags used to compile a D compiler, resulting
   executable produces different CTFE floating-point results
I've actually been thinking of writing a 128 bit float emulator, and then using that in the compiler internals to do all FP computation with.
No. Do not. I've worked on systems where the compiler and the runtime don't share floating point precisions before, and it was a nightmare.
I have some bad news for you about CTFE then. This already happens in DMD even though float is not emulated. :-o
O_O Are you saying 'float' in CTFE is not 'float'? I protest this about as strongly as I can muster...
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:
 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...
I imagine you'll be devastated when you discover that the C++ Standard does not require 32 bit floats to have 32 bit precision either, and never did. :-)
May 15 2016
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:
 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...
I imagine you'll be devastated when you discover that the C++ Standard does not require 32 bit floats to have 32 bit precision either, and never did. :-)
I've never read the C++ standard, but I have more experience with a wide range of real-world compilers than most, and it is rarely very violated. The times it is, we've known about it, and it has made us all very, very angry.
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:
 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.
It has on every C/C++ compiler for x86 machines that used the x87, which was true until SIMD, and is still true for x86 CPUs that don't target SIMD.
 The times it is, we've known about it, and it has made us
 all very, very angry.
The C/C++ standard is written that way for a reason. I'd like to hear what terrible problem is caused by having more accurate values.
May 15 2016
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 14:31, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:
 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.
It has on every C/C++ compiler for x86 machines that used the x87, which was true until SIMD, and is still true for x86 CPUs that don't target SIMD.
It has what? Reinterpreted your constant-folding to execute in 80bits internally for years? Again, if that's true, I expect that's only true in the context that the compiler also takes care to maintain the IEEE conformant bit pattern, or at very least, it works because the opportunity for FP constant folding in C++ is almost non-existent compared to CTFE, such that it's never resulted in a problem case in my experience. In D, we will (do) use CTFE for table generation all the time (this has never been done in C++). If those tables were generated with entirely different precision than the runtime functions, that's just begging for trouble.
 The times it is, we've known about it, and it has made us
 all very, very angry.
The C/C++ standard is written that way for a reason. I'd like to hear what terrible problem is caused by having more accurate values.
In the majority of my anecdotes, if they don't match, there are cracks/seams in the world. That is a show-stopping bug. We have had many late nights, even product launch delays due to by these problems. They have been a nightmare to solve in the past. Obviously the solution in this case is a relatively simple work-around; don't use CTFE (ie, lean on the LLVM runtime codegen instead to do the right thing with the float precision), but that's a tragic solution to a problem that should never happen in the first place.
May 15 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 10:24 PM, Manu via Digitalmars-d wrote:
 On 16 May 2016 at 14:31, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/15/2016 9:05 PM, Manu via Digitalmars-d wrote:
 I've never read the C++ standard, but I have more experience with a
 wide range of real-world compilers than most, and it is rarely very
 violated.
It has on every C/C++ compiler for x86 machines that used the x87, which was true until SIMD, and is still true for x86 CPUs that don't target SIMD.
It has what? Reinterpreted your constant-folding to execute in 80bits internally for years? Again, if that's true, I expect that's only true in the context that the compiler also takes care to maintain the IEEE conformant bit pattern, or at very least, it works because the opportunity for FP constant folding in C++ is almost non-existent compared to CTFE, such that it's never resulted in a problem case in my experience.
Check out my other message to you quoting the VC++ manual. It does constant folding at higher precision. So does gcc: http://stackoverflow.com/questions/7295861/enabling-strict-floating-point-mode-in-gcc
 In D, we will (do) use CTFE for table generation all the time (this
 has never been done in C++). If those tables were generated with
 entirely different precision than the runtime functions, that's just
 begging for trouble.
You can generate the tables at runtime at program startup by using a 'static this()' constructor.
 In the majority of my anecdotes, if they don't match, there are
 cracks/seams in the world. That is a show-stopping bug. We have had
 many late nights, even product launch delays due to by these problems.
 They have been a nightmare to solve in the past.
 Obviously the solution in this case is a relatively simple
 work-around; don't use CTFE (ie, lean on the LLVM runtime codegen
 instead to do the right thing with the float precision), but that's a
 tragic solution to a problem that should never happen in the first
 place.
You're always going to have tragic problems if you expect FP to behave like conventional mathematics. Just look at all the switches VC++ has that influence FP behavior. Do you really understand all that? Tell me you understand all the gcc floating point switches? https://gcc.gnu.org/wiki/FloatingPointMath How about g++ getting different results based on optimization switches? http://stackoverflow.com/questions/7517588/different-floating-point-result-with-optimization-enabled-compiler-bug ------- Or, you could design the code so that there aren't cracks because it is more tolerant of slight differences, i.e. look for so many mantissa bits matching instead of all of them matching. This can be done by subtracting the operands and looking at the magnitude of the difference. The fact that you have had "many late nights" and were not using D means that the compiler switches you were using were not adequate or were not doing what you thought they were doing.
May 15 2016
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:
 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:
 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...
I imagine you'll be devastated when you discover that the C++ Standard does not require 32 bit floats to have 32 bit precision either, and never did. :-)
I've never read the C++ standard, but I have more experience with a wide range of real-world compilers than most, and it is rarely very violated. The times it is, we've known about it, and it has made us all very, very angry.
Holy shit, it's just occurred to me that 'real' is only 64bits on arm (and every non-x86 platform)... That means a compiler running on an arm host will produce a different binary than a compiler running on an x86 host!! O_O
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 9:06 PM, Manu via Digitalmars-d wrote:
 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...
There are some that support 128 bits.
 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O
That's probably why VC++ dropped 80 bit long double support entirely. Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU, we're going to give you only 64 bits." It's also why I'd like to build a 128 soft fp emulator in dmd for all compile time float operations.
May 15 2016
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 14:37, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/15/2016 9:06 PM, Manu via Digitalmars-d wrote:
 Holy shit, it's just occurred to me that 'real' is only 64bits on arm
 (and every non-x86 platform)...
There are some that support 128 bits.
 That means a compiler running on an arm host will produce a different
 binary than a compiler running on an x86 host!! O_O
That's probably why VC++ dropped 80 bit long double support entirely. Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU, we're going to give you only 64 bits."
No, you'll give me 80bit _when I type "real"_. Otherwise, if I type 'double', you'll give me that. I don't understand how that can be controversial. I know you love the x87, but I'm pretty sure you're among a small minority. Personally, I don't want a single line of code that goes anywhere near the x87 to be emit in any binary I produce. It's a deprecated old crappy piece of hardware, and transfers between x87 and sse regs are slow.
 It's also why I'd like to build a 128 soft fp emulator in dmd for all
 compile time float operations.
And I did also realise the reason for your desire to implement 128bit soft-float the same moment I realised this. The situation that different DMD builds operate at different precisions internally (based on the host arch) is a whole new level of astonishment. I support this soft-float, but please, build a soft-float for all precisions, and use them everywhere that a hardware float for _the specified precision_ is not available ;)
May 15 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/15/2016 10:37 PM, Manu via Digitalmars-d wrote:
 No, you'll give me 80bit _when I type "real"_. Otherwise, if I type
 'double', you'll give me that. I don't understand how that can be
 controversial.
Because, as I explained, that results in a 2x or more speed degradation (along with the loss of accuracy).
 I know you love the x87, but I'm pretty sure you're among a small
 minority. Personally, I don't want a single line of code that goes
 anywhere near the x87 to be emit in any binary I produce. It's a
 deprecated old crappy piece of hardware, and transfers between x87 and
 sse regs are slow.
I used to do numerics work professionally. Most of the troubles I had were catastrophic loss of precision. Accumulated roundoff errors when doing numerical integration or matrix inversion are major problems. 80 bits helps dramatically with that.
 It's also why I'd like to build a 128 soft fp emulator in dmd for all
 compile time float operations.
And I did also realise the reason for your desire to implement 128bit soft-float the same moment I realised this. The situation that different DMD builds operate at different precisions internally (based on the host arch) is a whole new level of astonishment.
Then you're probably also astonished at the links I posted to you on how g++ and VC++ behave with FP.
May 15 2016
next sibling parent Mike James <foo bar.com> writes:
On Monday, 16 May 2016 at 06:46:59 UTC, Walter Bright wrote:
 On 5/15/2016 10:37 PM, Manu via Digitalmars-d wrote:
 [...]
Because, as I explained, that results in a 2x or more speed degradation (along with the loss of accuracy).
 [...]
I used to do numerics work professionally. Most of the troubles I had were catastrophic loss of precision. Accumulated roundoff errors when doing numerical integration or matrix inversion are major problems. 80 bits helps dramatically with that.
A classic example... :-) http://www.stsci.edu/~lbradley/seminar/butterfly.html
May 16 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 2:46 AM, Walter Bright wrote:
 I used to do numerics work professionally. Most of the troubles I had
 were catastrophic loss of precision. Accumulated roundoff errors when
 doing numerical integration or matrix inversion are major problems. 80
 bits helps dramatically with that.
Aren't good algorithms helping dramatically with that? Also, do you have a theory that reconciles your assessment of the importance of 80-bit math with the fact that the computing world is moving away from it? http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it Andrei
May 16 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 16 May 2016 at 10:29:02 UTC, Andrei Alexandrescu wrote:
 On 5/16/16 2:46 AM, Walter Bright wrote:
 I used to do numerics work professionally. Most of the 
 troubles I had
 were catastrophic loss of precision. Accumulated roundoff 
 errors when
 doing numerical integration or matrix inversion are major 
 problems. 80
 bits helps dramatically with that.
Aren't good algorithms helping dramatically with that? Also, do you have a theory that reconciles your assessment of the importance of 80-bit math with the fact that the computing world is moving away from it? http://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it Andrei
Regardless of the compiler actually doing it or not, the argument that extra precision is a problem is self defeating. I don't think argument for speed have been raised so far.
May 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 7:33 AM, deadalnix wrote:
 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.
I agree that this whole "we need less precision" argument would be difficult to accept.
 I don't think argument
 for speed have been raised so far.
This may be the best angle in this discussion. For all I can tell 80 bit is slow as molasses and on the road to getting slower. Isn't that enough of an argument to move away from it? Andrei
May 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 4:35 AM, Andrei Alexandrescu wrote:
 This may be the best angle in this discussion. For all I can tell 80 bit is
slow
 as molasses and on the road to getting slower. Isn't that enough of an argument
 to move away from it?
We are talking CTFE here, not runtime.
May 16 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 8:19 AM, Walter Bright wrote:
 On 5/16/2016 4:35 AM, Andrei Alexandrescu wrote:
 This may be the best angle in this discussion. For all I can tell 80
 bit is slow
 as molasses and on the road to getting slower. Isn't that enough of an
 argument
 to move away from it?
We are talking CTFE here, not runtime.
I have big plans with floating-point CTFE and all are elastic: the faster CTFE FP is, the more and better things we can do. Things that other languages can't dream to do, like interpolation tables for transcendental functions. So a slowdown of FP CTFE would be essentially a strategic loss. -- Andrei
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 6:15 AM, Andrei Alexandrescu wrote:
 On 5/16/16 8:19 AM, Walter Bright wrote:
 We are talking CTFE here, not runtime.
I have big plans with floating-point CTFE and all are elastic: the faster CTFE FP is, the more and better things we can do. Things that other languages can't dream to do, like interpolation tables for transcendental functions. So a slowdown of FP CTFE would be essentially a strategic loss. -- Andrei
Based on my experience with soft float on DOS, and the fact that CPUs are what, 1000 times faster today, I have a hard time thinking of a case where that would be a big problem. I can't see someone running a meteorological prediction using CTFE :-)
May 16 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 05/16/2016 01:16 PM, Walter Bright wrote:
 On 5/16/2016 6:15 AM, Andrei Alexandrescu wrote:
 On 5/16/16 8:19 AM, Walter Bright wrote:
 We are talking CTFE here, not runtime.
I have big plans with floating-point CTFE and all are elastic: the faster CTFE FP is, the more and better things we can do. Things that other languages can't dream to do, like interpolation tables for transcendental functions. So a slowdown of FP CTFE would be essentially a strategic loss. -- Andrei
Based on my experience with soft float on DOS
I seriously think all experience accumulated last century needs to be thoroughly scrutinized. The world of software has changed and is changing fast enough to make all tricks older than a a decade virtually useless. Although core structures and algorithms stay the same, even some fundamentals are changing - e.g. chasing pointers is no longer faster than seemingly slow operations on implicit data structures on arrays, and so on and so forth. Yes, there was a time when one setvbuf() call would make I/O one order of magnitude faster. Those days are long gone, and the only value of that knowledge industry is tepid anecdotes. Fast compile-time FP is good, and the faster it is, the better it is. It's a huge differentiating factor for us. A little marginal precision is not. Please don't spend time on making compile-time FP slower. Thanks, Andrei
May 16 2016
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 21:35, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/16/16 7:33 AM, deadalnix wrote:
 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.
I agree that this whole "we need less precision" argument would be difficult to accept.
 I don't think argument
 for speed have been raised so far.
This may be the best angle in this discussion. For all I can tell 80 bit is slow as molasses and on the road to getting slower. Isn't that enough of an argument to move away from it?
It really has though! At Remedy, the first thing we noticed when compiling code with DMD is that our float code produced x87 ops, and with the x64 calling convention (floats in SSE regs), this means pushing every function argument to the stack, loading them into x87 regs, doing the work (slowly), pushing them back, and popping them into a return reg. The codegen is insanely long and inefficient. The only reason it wasn't a critical blocker was because of the relative small level of use of D, and that it wasn't in any time-critical path. If Ethan and Remedy want to expand their use of D, the compiler CAN NOT emit x87 code. It's just a matter of time before a loop is in a hot path.
May 16 2016
next sibling parent Ethan Watson <gooberman gmail.com> writes:
On Tuesday, 17 May 2016 at 02:00:24 UTC, Manu wrote:
 If Ethan and Remedy want to expand their use of D, the compiler 
 CAN
 NOT emit x87 code. It's just a matter of time before a loop is 
 in a
 hot path.
I really need to see what the codegen for the latest DMD looks like, I have no idea what the current state is. But this isn't just true for Remedy, it's true for any engine programmer in another company thinking of using D. In context of this entire discussion though, a compiler switch to give the codegen I want is my preference. I have no desire to dictate how other people should use D/DMD, I just want the option to use it the way I need to.
May 17 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 7:00 PM, Manu via Digitalmars-d wrote:
 If Ethan and Remedy want to expand their use of D, the compiler CAN
 NOT emit x87 code. It's just a matter of time before a loop is in a
 hot path.
dmd no longer emits x87 code for float/double on 64 bit models, and hasn't for years.
May 17 2016
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 17 May 2016 at 12:00, Manu <turkeyman gmail.com> wrote:
 On 16 May 2016 at 21:35, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/16/16 7:33 AM, deadalnix wrote:
 Regardless of the compiler actually doing it or not, the argument that
 extra precision is a problem is self defeating.
I agree that this whole "we need less precision" argument would be difficult to accept.
 I don't think argument
 for speed have been raised so far.
This may be the best angle in this discussion. For all I can tell 80 bit is slow as molasses and on the road to getting slower. Isn't that enough of an argument to move away from it?
It really has though! At Remedy, the first thing we noticed when compiling code with DMD is that our float code produced x87 ops, and with the x64 calling convention (floats in SSE regs), this means pushing every function argument to the stack, loading them into x87 regs, doing the work (slowly), pushing them back, and popping them into a return reg. The codegen is insanely long and inefficient. The only reason it wasn't a critical blocker was because of the relative small level of use of D, and that it wasn't in any time-critical path. If Ethan and Remedy want to expand their use of D, the compiler CAN NOT emit x87 code. It's just a matter of time before a loop is in a hot path.
That said, I recall a conversation where Walter said this has been addressed? I haven't worked on this in some time now, so I haven't tested where it's at. Just saying that complaints about x87 performance have definitely been made.
May 16 2016
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:29 AM, Andrei Alexandrescu wrote:
 Aren't good algorithms helping dramatically with that?
Yup. But they are not textbook math algorithms, tend to be complex and strange, and are not very discoverable by regular programmers (I tried and failed). Extended precision is a simple, straightforward fix to precision problems in straightforward code.
May 16 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 12:37 AM, Walter Bright wrote:
 Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU,
 we're going to give you only 64 bits."
I'm not sure about this. My understanding is that all SSE has hardware for 32 and 64 bit floats, and the the 80-bit hardware is pretty much cut-and-pasted from the x87 days without anyone really looking in improving it. And that's been the case for more than a decade. Is that correct? I'm looking for example at http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see that on all Intel and compatible hardware, the speed of 80-bit floating point operations ranges between much slower and disastrously slower. I think it's time to revisit our attitudes to floating point, which was formed last century in the heydays of x87. My perception is the world has moved to SSE and 32- and 64-bit float; the "real" type is a distraction for D; the whole let's do things in 128-bit during compilation is a time waster; and many of the original things we want to do with floating point are different without a distinction, and a further waste of our resources. Andrei
May 16 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:
 I'm not sure about this. My understanding is that all SSE has hardware for 32
 and 64 bit floats, and the the 80-bit hardware is pretty much cut-and-pasted
 from the x87 days without anyone really looking in improving it. And that's
been
 the case for more than a decade. Is that correct?
I believe so.
 I'm looking for example at
 http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see that on all
 Intel and compatible hardware, the speed of 80-bit floating point operations
 ranges between much slower and disastrously slower.
It's not a totally fair comparison. A matrix inversion algorithm that compensates for cumulative precision loss involves executing a lot more FP instructions (don't know the ratio).
 I think it's time to revisit our attitudes to floating point, which was formed
 last century in the heydays of x87. My perception is the world has moved to SSE
 and 32- and 64-bit float; the "real" type is a distraction for D; the whole
 let's do things in 128-bit during compilation is a time waster; and many of the
 original things we want to do with floating point are different without a
 distinction, and a further waste of our resources.
Some counter points: 1. Go uses 256 bit soft float for constant folding. 2. Speed is hardly the only criterion. Quickly getting the wrong answer (and not just a few bits off, but total loss of precision) is of no value. 3. Supporting 80 bit reals does not take away from the speed of floats/doubles at runtime. 4. Removing 80 bit reals will consume resources (adapting the test suite, rewriting the math library, ...). 5. Other languages not supporting it means D has a capability they don't have. My experience with selling products is that if you have an exclusive feature that a particular customer needs, it's a slam dunk sale. 6. My other experience with feature sets is if you drop things that make your product different, and concentrate on matching feature checklists with Major Brand X, customers go with Major Brand X. 7. 80 bit reals are there and they work. The support is mature, and is rarely worked on, i.e. it does not consume resources. 8. Removing it would break an unknown amount of code, and there's no reasonable workaround for those that rely on it.
May 16 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/16/16 8:37 AM, Walter Bright wrote:
 On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:
 I'm looking for example at
 http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see
 that on all
 Intel and compatible hardware, the speed of 80-bit floating point
 operations
 ranges between much slower and disastrously slower.
It's not a totally fair comparison. A matrix inversion algorithm that compensates for cumulative precision loss involves executing a lot more FP instructions (don't know the ratio).
It is rare to need to actually compute the inverse of a matrix. Most of the time it's of interest to solve a linear equation of the form Ax = b, for which a variety of good methods exist that don't entail computing the actual inverse. I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a lot of other evidence. This is what happened with input vs. forward C++ iterators as the main motivator for a variety of concepts designs.
 Some counter points:
Glad to see these!
 1. Go uses 256 bit soft float for constant folding.
Go can afford it because it does no interesting things during compilation. We can't.
 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no value.
Of course. But it turns out the precision argument loses to the speed argument. A. It's been many many years and very few if any people commend D for its superior approach to FP precision. B. In contrast, a bunch of folks complain about anything slow, be it during compilation or at runtime. Good algorithms lead to good precision, not 16 additional bits. Precision is overrated. Speed isn't.
 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.
Fast compile-time floats are of strategic importance to us. Give me fast FP during compilation, I'll make it go slow (whilst put to do amazing work).
 4. Removing 80 bit reals will consume resources (adapting the test
 suite, rewriting the math library, ...).
I won't argue with that! Just let's focus on the right things: good fast streamlined computing using the appropriate hardware.
 5. Other languages not supporting it means D has a capability they don't
 have. My experience with selling products is that if you have an
 exclusive feature that a particular customer needs, it's a slam dunk sale.
Again: I'm not seeing people coming out of the woodwork to praise D's precision. What they would indeed enjoy is amazing FP use during compilation, and that can be done only if CTFE FP is __FAST__. That _is_, indeed the capability others are missing!
 6. My other experience with feature sets is if you drop things that make
 your product different, and concentrate on matching feature checklists
 with Major Brand X, customers go with Major Brand X.
This is true in principle but too vague to be relevant. Again, what evidence do you have that D's additional precision is revered? I see none, over like a decade.
 7. 80 bit reals are there and they work. The support is mature, and is
 rarely worked on, i.e. it does not consume resources.
Yeah, I just don't want it used in any new code. Please. It's using lead for building boats.
 8. Removing it would break an unknown amount of code, and there's no
 reasonable workaround for those that rely on it.
Let's do what everybody did to x87: continue supporting, but slowly and surely drive it to obsolescence. That's the right way. Andrei
May 16 2016
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 16 May 2016 at 14:32:55 UTC, Andrei Alexandrescu wrote:
 Let's do what everybody did to x87: continue supporting, but 
 slowly and surely drive it to obsolescence.

 That's the right way.
This is a long thread that has covered a lot of ground. There has been a lot of argument, but few concrete take-a-ways (this was one, why I'm quoting it). As far as I see it, the main topics have been: 1) Change in precision during comparisons (the original) 2) Change in precision to real during intermediate floating point calculations 3) Floating point precision in CTFE On #1, there was discussion initially about compiler warnings as a solution. I have not seen any concrete recommendations. Personally, I am not a fan of implicit conversions. It's one more set of rules I need to remember. I would prefer a compile-time error with a recommendation to explicitly cast to whatever is the best precision. It may even make sense for the error to also recommend approxEqual. On #2, I don't see any clear agreement on anything. Walter emphasized the importance of precision, while others emphasized determinism and speed. As someone who sometimes does matrix inversions and gets eigenvalues, I'm sympathetic to Walter's points about precision. For some applications, additional precision is essential. However, others clearly want the option to disable this behavior. Walter stressed Java's difficulty in resolving the issue. What about their solution? They have strictfp, which allows exactly the behavior people like Manu seem to want. In D, you could make it an attribute. The benefit of adding the new annotation is that no code would get broken. You could write strictfp: at the top of a file instead of a compiler flag. You could also add a defaultfp (or !strictfp or strictfp(false)) as its complement. I'm not sure whether it should be applied recursively. Perhaps? A recursive approach can be difficult. However, a non-recursive approach would be limited in some ways. You'd have to do some work on std.math to get it working. On #3, resolving #2 might address some of this. Regardless, this topic seems to be the least fully formed. Improved CTFE seems universally regarded as a positive.
May 16 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:
 It is rare to need to actually compute the inverse of a matrix. Most of the
time
 it's of interest to solve a linear equation of the form Ax = b, for which a
 variety of good methods exist that don't entail computing the actual inverse.
I was solving n equations with n unknowns.
 I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a lot of
 other evidence. This is what happened with input vs. forward C++ iterators as
 the main motivator for a variety of concepts designs.
What I did was implement the algorithm out of my calculus textbook. Sure, it's a naive algorithm - but it is highly unlikely that untrained FP programmers know intuitively how to deal with precision loss. I bring up our very own Phobos sum algorithm, which was re-implemented later with the Kahan method to reduce precision loss.
 1. Go uses 256 bit soft float for constant folding.
Go can afford it because it does no interesting things during compilation. We can't.
The we can't is conjecture at the moment.
 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no value.
Of course. But it turns out the precision argument loses to the speed argument. A. It's been many many years and very few if any people commend D for its superior approach to FP precision. B. In contrast, a bunch of folks complain about anything slow, be it during compilation or at runtime.
D's support for reals does not negatively impact the speed of float or double computations.
 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.
Fast compile-time floats are of strategic importance to us. Give me fast FP during compilation, I'll make it go slow (whilst put to do amazing work).
I still have a hard time seeing what you plan to do at compile time that would involve tens of millions of FP calculations.
 6. My other experience with feature sets is if you drop things that make
 your product different, and concentrate on matching feature checklists
 with Major Brand X, customers go with Major Brand X.
This is true in principle but too vague to be relevant. Again, what evidence do you have that D's additional precision is revered? I see none, over like a decade.
Fortran supports Quad (128 bit floats) as standard. https://en.wikipedia.org/wiki/Quadruple-precision_floating-point_format
May 16 2016
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 16 May 2016 at 18:57:24 UTC, Walter Bright wrote:
 On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:
 It is rare to need to actually compute the inverse of a 
 matrix. Most of the time
 it's of interest to solve a linear equation of the form Ax = 
 b, for which a
 variety of good methods exist that don't entail computing the 
 actual inverse.
I was solving n equations with n unknowns.
LU decomposition is the more common approach. atlab has a backslash operator to solve systems of equations with LU decomposition: http://www.mathworks.com/help/matlab/ref/inv.html#bu6sfy8-1
May 16 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 05/16/2016 02:57 PM, Walter Bright wrote:
 On 5/16/2016 7:32 AM, Andrei Alexandrescu wrote:
 It is rare to need to actually compute the inverse of a matrix. Most
 of the time
 it's of interest to solve a linear equation of the form Ax = b, for
 which a
 variety of good methods exist that don't entail computing the actual
 inverse.
I was solving n equations with n unknowns.
That's the worst way to go about it. I've seen students fail exams over it. Solving a linear equations sytems by computing the inverse is inferior to just about any other method. See e.g. http://www.mathworks.com/help/matlab/ref/inv.html "It is seldom necessary to form the explicit inverse of a matrix. A frequent misuse of inv arises when solving the system of linear equations Ax = b. One way to solve the equation is with x = inv(A)*b. A better way, from the standpoint of both execution time and numerical accuracy, is to use the matrix backslash operator x = A\b. This produces the solution using Gaussian elimination, without explicitly forming the inverse. See mldivide for further information." You have long been advocating that the onus is on the engineer to exercise good understanding of what's going on when using domain-specific code such as UTF, linear algebra, etc. So if you exercised it now, you need to discount this argument.
 I emphasize the danger of this kind of thinking: 1-2 anecdotes trump a
 lot of
 other evidence. This is what happened with input vs. forward C++
 iterators as
 the main motivator for a variety of concepts designs.
What I did was implement the algorithm out of my calculus textbook. Sure, it's a naive algorithm - but it is highly unlikely that untrained FP programmers know intuitively how to deal with precision loss.
As someone else said: a few bits of extra precision ain't gonna help them. I thought that argument was closed.
 I bring
 up our very own Phobos sum algorithm, which was re-implemented later
 with the Kahan method to reduce precision loss.
Kahan is clear, ingenous, and understandable and a great part of the stdlib. I don't see what the point is here. Naive approaches aren't going to take anyone far, regardless of precision.
 1. Go uses 256 bit soft float for constant folding.
Go can afford it because it does no interesting things during compilation. We can't.
The we can't is conjecture at the moment.
We can't and we shouldn't invest time in investigating whether we can. It's a waste even if the project succeeded 100% and exceeded anyone's expectations.
 2. Speed is hardly the only criterion. Quickly getting the wrong answer
 (and not just a few bits off, but total loss of precision) is of no
 value.
Of course. But it turns out the precision argument loses to the speed argument. A. It's been many many years and very few if any people commend D for its superior approach to FP precision. B. In contrast, a bunch of folks complain about anything slow, be it during compilation or at runtime.
D's support for reals does not negatively impact the speed of float or double computations.
Then let's not do more of it.
 3. Supporting 80 bit reals does not take away from the speed of
 floats/doubles at runtime.
Fast compile-time floats are of strategic importance to us. Give me fast FP during compilation, I'll make it go slow (whilst put to do amazing work).
I still have a hard time seeing what you plan to do at compile time that would involve tens of millions of FP calculations.
Give those to me and you'll be surprised. Andrei
May 16 2016
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 22:10, Andrei Alexandrescu wrote:
 I bring
 up our very own Phobos sum algorithm, which was re-implemented later
 with the Kahan method to reduce precision loss.
Kahan is clear, ingenous, and understandable and a great part of the stdlib. I don't see what the point is here. Naive approaches aren't going to take anyone far, regardless of precision.
Kahan summation is not guaranteed to work in D.
May 17 2016
prev sibling parent reply Ethan Watson <gooberman gmail.com> writes:
On Monday, 16 May 2016 at 14:32:55 UTC, Andrei Alexandrescu wrote:
 It is rare to need to actually compute the inverse of a matrix.
Unless you're doing game/graphics work ;-) 4x3 or 4x4 matrices are commonly used to represent transforms in 3D space in every 3D polygon-based rendering pipeline I know of. It's even a requirement for fixed-function OpenGL 1.x. Video games - also known around here as "The Exception To The Rule". (Side note: My own preference is to represent transforms as a quaternion and vector. Inverting such a transform is a simple matter of negating a few components. Generating a matrix from such a transform for rendering purposes is trivial compared to matrix inversion.)
May 17 2016
parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 17 May 2016 at 07:47:58 UTC, Ethan Watson wrote:
 Unless you're doing game/graphics work ;-) 4x3 or 4x4 matrices 
 are commonly used to represent transforms in 3D space in every 
 3D polygon-based rendering pipeline I know of. It's even a 
 requirement for fixed-function OpenGL 1.x.

 Video games - also known around here as "The Exception To The 
 Rule".

 (Side note: My own preference is to represent transforms as a 
 quaternion and vector. Inverting such a transform is a simple 
 matter of negating a few components. Generating a matrix from 
 such a transform for rendering purposes is trivial compared to 
 matrix inversion.)
I don't know much about computer graphics, but if you're solving equations, then you can use the techniques mentioned above. Nevertheless, I'm not really sure what would be the fastest approach to inverting small matrices. I would definitely try the LU or Cholesky approaches. It might be that for a small matrix a Gaussian reduction approach would be fast. There are some analytic tricks you could use if you have some information about them, like when you can represent them as blocks. If some of the blocks are zero or identity matrices, then it simplifies the calculations too.
May 17 2016
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, May 16, 2016 at 05:37:58AM -0700, Walter Bright via Digitalmars-d wrote:
 On 5/16/2016 3:27 AM, Andrei Alexandrescu wrote:
[...]
I think it's time to revisit our attitudes to floating point, which
was formed last century in the heydays of x87. My perception is the
world has moved to SSE and 32- and 64-bit float; the "real" type is a
distraction for D; the whole let's do things in 128-bit during
compilation is a time waster; and many of the original things we want
to do with floating point are different without a distinction, and a
further waste of our resources.
Some counter points: 1. Go uses 256 bit soft float for constant folding. 2. Speed is hardly the only criterion. Quickly getting the wrong answer (and not just a few bits off, but total loss of precision) is of no value.
An algorithm that uses 80-bit but isn't written properly to account for FP total precision loss will also produce wrong results, just like an algorithm that uses 64-bit and isn't written to account for FP total precision loss. [...]
 5. Other languages not supporting it means D has a capability they
 don't have. My experience with selling products is that if you have an
 exclusive feature that a particular customer needs, it's a slam dunk
 sale.
It's also the case that maintaining a product with feature X that nobody uses is an unnecessary drain on developmental resources.
 6. My other experience with feature sets is if you drop things that
 make your product different, and concentrate on matching feature
 checklists with Major Brand X, customers go with Major Brand X.
[...] If I were presented with product A having an archaic feature X that works slowly and that I don't even need in the first place, vs. product B having exactly the feature set I need without the baggage of archaic feature X, I'd go with product B. T -- Real Programmers use "cat > a.out".
May 16 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 7:37 AM, H. S. Teoh via Digitalmars-d wrote:
 An algorithm that uses 80-bit but isn't written properly to account for
 FP total precision loss will also produce wrong results, just like an
 algorithm that uses 64-bit and isn't written to account for FP total
 precision loss.
That is correct. But in most routine cases, it's enough more precision that going to more heroic algorithm changes become unnecessary.
 If I were presented with product A having an archaic feature X that
 works slowly and that I don't even need in the first place, vs. product
 B having exactly the feature set I need without the baggage of archaic
 feature X, I'd go with product B.
I have business experience with what people actually choose, and it's often not what they say they would choose. Sorry :-) Social proof (i.e. what your colleagues use) is a very, very important factor. But having a needed feature not available in the more popular product trumps social proof. It doesn't need to appeal to everyone, but it can be the wedge that drives a product into the mainstream. The needs of numerical analysts have often been neglected by the C/C++ community. The community has been very slow to recognize IEEE, by about a decade. It wasn't until fairly recently that C/C++ compilers even handled NaN properly. It's no surprise that FORTRAN reigned as the choice of numerical analysts. (On the other hand, the needs of game developers have received strong support.) The dismissal of concerns about precision as "archaic" is something I have seen for a long time. As my calculator anecdote illustrates, even engineers do not recognize loss of precision when it happens. I'm not talking about a few bits in last place, I'm talking about not recognizing total loss of precision. I sometimes wonder how many utter garbage FP results are being generated and treated as correct answers by researchers who confuse textbook math with FP math. The only field I can think of where a sprinkling of garbage results don't matter as long as it is fast is game programming. Even if you select doubles for speed, it's nice to be able to temporarily swap in reals as a check to see if the similar results are computed. If not, you surely have a loss of precision problem.
May 16 2016
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 16 May 2016 at 17:50:22 UTC, Walter Bright wrote:
 The needs of numerical analysts have often been neglected by 
 the C/C++ community. The community has been very slow to 
 recognize IEEE, by about a decade. It wasn't until fairly 
 recently that C/C++ compilers even handled NaN properly. It's 
 no surprise that FORTRAN reigned as the choice of numerical 
 analysts.

 (On the other hand, the needs of game developers have received 
 strong support.)
I have regression scripts that check audio computations against the previous baseline. The goal is to ensure the results are consistent between compilers and bitness, and to check for the validity of sound-altering optimization (usually within 80 dB peak difference is OK). And of course not to utterly break things during optimization work. I've been harmed once by the x87 promotion of a float to real, which made an algorithm sound better on 32-bit than 64-bit. On 64-bit DMD used SSE which avoided the internal precision boost. If I had tested only in 32-bit I could have inferred float precision was enough. This is one time I would have preferred if D would never promoted float to double or to real. I understand that the x87 code would go much slower if restricted precision was ensured. So how about using SSE like in 64-bit code? Free bits of precision may lead to think things are better than they are.
May 16 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 1:43 PM, Guillaume Piolat wrote:
 So how about using SSE like in 64-bit code?
It does for 32 bits on OSX, because SSE is guaranteed to be available on Macs. But this is not true in general, and so it does not for other x86 systems.
May 17 2016
prev sibling next sibling parent reply Daniel Murphy <yebbliesnospam gmail.com> writes:
On 16/05/2016 10:37 PM, Walter Bright wrote:
 Some counter points:

 1. Go uses 256 bit soft float for constant folding.
Then we should use 257 bit soft float!
May 16 2016
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.05.2016 17:11, Daniel Murphy wrote:
 On 16/05/2016 10:37 PM, Walter Bright wrote:
 Some counter points:

 1. Go uses 256 bit soft float for constant folding.
Then we should use 257 bit soft float!
I don't see how you reach that conclusion, as 258 bit soft float is clearly superior.
May 16 2016
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 18:31, Timon Gehr via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 16.05.2016 17:11, Daniel Murphy wrote:
 On 16/05/2016 10:37 PM, Walter Bright wrote:
 Some counter points:

 1. Go uses 256 bit soft float for constant folding.
Then we should use 257 bit soft float!
I don't see how you reach that conclusion, as 258 bit soft float is clearly superior.
You should be more pragmatic, 224 bits is all you need. :-)
May 16 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 8:11 AM, Daniel Murphy wrote:
 On 16/05/2016 10:37 PM, Walter Bright wrote:
 1. Go uses 256 bit soft float for constant folding.
Then we should use 257 bit soft float!
I like the cut of your jib!
May 16 2016
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 16 May 2016 at 15:11:21 UTC, Daniel Murphy wrote:
 On 16/05/2016 10:37 PM, Walter Bright wrote:
 Some counter points:

 1. Go uses 256 bit soft float for constant folding.
Then we should use 257 bit soft float!
Go uses at least 288 bits!!!!! (256 bits mantissa and 32 bits exponent). But Go has a completely different take on constant expressions, and much better than the C family that D belongs to (but still far from ideal). From https://golang.org/ref/spec: « Numeric constants represent exact values of arbitrary precision and do not overflow. Consequently, there are no constants denoting the IEEE-754 negative zero, infinity, and not-a-number values. Constants may be typed or untyped. Literal constants, true, false, iota, and certain constant expressions containing only untyped constant operands are untyped. A constant may be given a type explicitly by a constant declaration or conversion, or implicitly when used in a variable declaration or an assignment or as an operand in an expression. It is an error if the constant value cannot be represented as a value of the respective type. For instance, 3.0 can be given any integer or any floating-point type, while 2147483648.0 (equal to 1<<31) can be given the types float32, float64, or uint32 but not int32 or string. An untyped constant has a default type which is the type to which the constant is implicitly converted in contexts where a typed value is required, for instance, in a short variable declaration such as i := 0 where there is no explicit type. The default type of an untyped constant is bool, rune, int, float64, complex128 or string respectively, depending on whether it is a boolean, rune, integer, floating-point, complex, or string constant. Implementation restriction: Although numeric constants have arbitrary precision in the language, a compiler may implement them using an internal representation with limited precision. That said, every implementation must: Represent integer constants with at least 256 bits. Represent floating-point constants, including the parts of a complex constant, with a mantissa of at least 256 bits and a signed exponent of at least 32 bits. Give an error if unable to represent an integer constant precisely. Give an error if unable to represent a floating-point or complex constant due to overflow. Round to the nearest representable constant if unable to represent a floating-point or complex constant due to limits on precision. These requirements apply both to literal constants and to the result of evaluating constant expressions. » See also https://golang.org/ref/spec#Constant_expressions
May 17 2016
parent reply Matthias Bentrup <matthias.bentrup googlemail.com> writes:
If you try to make compile-time FP math behave exactly like 
run-time FP math, you'd not only have to use the same precision 
in the compiler, but also the same rounding mode, denormal 
handling etc., which can be changed at run time 
(http://dlang.org/phobos/core_stdc_fenv.html), so the exact same 
piece of code can yield different results based on the run-time 
FP-context.
May 17 2016
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 17 May 2016 at 14:05:54 UTC, Matthias Bentrup wrote:
 If you try to make compile-time FP math behave exactly like 
 run-time FP math, you'd not only have to use the same precision 
 in the compiler, but also the same rounding mode, denormal 
 handling etc., which can be changed at run time 
 (http://dlang.org/phobos/core_stdc_fenv.html), so the exact 
 same piece of code can yield different results based on the 
 run-time FP-context.
Yes, heading rounding mode is an absolute requirement. If the system don't heed the rounding mode then it will change the semantics and break the program when computing upper vs lower bounds. I've never liked the stateful approach to rounding. If I was to design a language I probably would make it part of the type or operator or something along those lines.
May 17 2016
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/16/2016 5:37 AM, Walter Bright wrote:
 [...]
9. Both clang and gcc offer 80 bit long doubles. It's Microsoft VC++ that is out of step. Not having an 80 bit type in D means diminished interoperability with very popular C and C++ compilers.
May 16 2016
prev sibling parent reply Wyatt <wyatt.epp gmail.com> writes:
On Monday, 16 May 2016 at 12:37:58 UTC, Walter Bright wrote:
 7. 80 bit reals are there and they work. The support is mature, 
 and is rarely worked on, i.e. it does not consume resources.
This may not be true for too much longer-- both Intel and AMD are slowly phasing the x86 FPU out. I think Intel already announced a server chip that omits it entirely, though I can't find the corroborating link. -Wyatt
May 17 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/17/2016 7:08 AM, Wyatt wrote:
 On Monday, 16 May 2016 at 12:37:58 UTC, Walter Bright wrote:
 7. 80 bit reals are there and they work. The support is mature, and is rarely
 worked on, i.e. it does not consume resources.
This may not be true for too much longer-- both Intel and AMD are slowly phasing the x86 FPU out. I think Intel already announced a server chip that omits it entirely, though I can't find the corroborating link.
Consider that clang and g++ support 80 bit long doubles. D is hardly unique. It's VC++ that does not.
May 17 2016
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 20:27, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 5/16/16 12:37 AM, Walter Bright wrote:
 Me, I think of that as "Who cares that you paid $$$ for an 80 bit CPU,
 we're going to give you only 64 bits."
I'm not sure about this. My understanding is that all SSE has hardware for 32 and 64 bit floats, and the the 80-bit hardware is pretty much cut-and-pasted from the x87 days without anyone really looking in improving it. And that's been the case for more than a decade. Is that correct?
It's not only correct, but it's also deprecated, and may be removed from silicone (ie, emulated/microcoded, is it's not already...) at some unknown point in the future.
May 16 2016
prev sibling next sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Monday, 16 May 2016 at 10:25:33 UTC, Andrei Alexandrescu wrote:
 On 5/16/16 12:37 AM, Walter Bright wrote:
 Me, I think of that as "Who cares that you paid $$$ for an 80 
 bit CPU,
 we're going to give you only 64 bits."
I'm not sure about this. My understanding is that all SSE has hardware for 32 and 64 bit floats, and the the 80-bit hardware is pretty much cut-and-pasted from the x87 days without anyone really looking in improving it. And that's been the case for more than a decade. Is that correct? I'm looking for example at http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/ and see that on all Intel and compatible hardware, the speed of 80-bit floating point operations ranges between much slower and disastrously slower. I think it's time to revisit our attitudes to floating point, which was formed last century in the heydays of x87. My perception is the world has moved to SSE and 32- and 64-bit float; the "real" type is a distraction for D; the whole let's do things in 128-bit during compilation is a time waster; and many of the original things we want to do with floating point are different without a distinction, and a further waste of our resources. It is a bit ironic that we worry about autodecoding (I'll destroy that later) whilst a landslide loss of speed and predictability in floating point math doesn't raise an eyebrow. Andrei
The AMD64 programmer's manual discourages the use of x87: "For media and scientific programs that demand floating-point operations, it is often easier and more powerful to use SSE instructions. Such programs perform better than x87 floating-point programs, because the YMM/XMM register file is flat rather than stack-oriented, there are twice as many registers (in 64-bit mode), and SSE instructions can operate on four or eight times the number of floating-point operands as can x87 instructions. This ability to operate in parallel on multiple pairs of floating-point elements often makes it possible to remove local temporary variables that would otherwise be needed in x87 floating-point code."
May 16 2016
prev sibling parent Ethan Watson <gooberman gmail.com> writes:
On Monday, 16 May 2016 at 10:25:33 UTC, Andrei Alexandrescu wrote:
 I'm not sure about this. My understanding is that all SSE has 
 hardware for 32 and 64 bit floats, and the the 80-bit hardware 
 is pretty much cut-and-pasted from the x87 days without anyone 
 really looking in improving it. And that's been the case for 
 more than a decade. Is that correct?
Pretty much. On the OS side, Windows has officially deprecated x87 for the 64-bit version in desktop mode, and it's flat out forbidden in kernel mode. All development focus from Intel has been on improving the SSE/AVX instruction set and pipeline. And on a gamedev side, we generally go for fast over precise. Or, more to the point, an acceptable loss in precision. The C++ codegen spits out SSE/AVX code by default in our builds, and I hand optimise with appropriate intrinsics certain functions that get inlined. SIMD is even more an appropriate point to bring up here - gaming is trending towards more parallel operations, operating on a single float at a time is not the correct way to get the best performance out of your system. This is one of those things where I can see the point for the D compiler to do things its own way - but only when it expects to operate in a pure D environment. We have heavy interop between C++ and D. If simple functions can give different results at compile time without a way for me to configure the compiler on both sides, what actual benefits does that give me?
May 16 2016
prev sibling next sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 06:06, Manu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:
 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:
 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...
I imagine you'll be devastated when you discover that the C++ Standard does not require 32 bit floats to have 32 bit precision either, and never did. :-)
I've never read the C++ standard, but I have more experience with a wide range of real-world compilers than most, and it is rarely very violated. The times it is, we've known about it, and it has made us all very, very angry.
Holy shit, it's just occurred to me that 'real' is only 64bits on arm (and every non-x86 platform)... That means a compiler running on an arm host will produce a different binary than a compiler running on an x86 host!! O_O
Which is why gcc/g++ (ergo gdc) uses floating point emulation. Getting consistent results at compile time regardless of whatever host/target/cross configuration trumps doing it natively.
May 15 2016
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 16 May 2016 at 16:09, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 16 May 2016 at 06:06, Manu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 16 May 2016 at 14:05, Manu <turkeyman gmail.com> wrote:
 On 16 May 2016 at 13:03, Walter Bright via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On 5/15/2016 7:01 PM, Manu via Digitalmars-d wrote:
 Are you saying 'float' in CTFE is not 'float'? I protest this about as
 strongly as I can muster...
I imagine you'll be devastated when you discover that the C++ Standard does not require 32 bit floats to have 32 bit precision either, and never did. :-)
I've never read the C++ standard, but I have more experience with a wide range of real-world compilers than most, and it is rarely very violated. The times it is, we've known about it, and it has made us all very, very angry.
Holy shit, it's just occurred to me that 'real' is only 64bits on arm (and every non-x86 platform)... That means a compiler running on an arm host will produce a different binary than a compiler running on an x86 host!! O_O
Which is why gcc/g++ (ergo gdc) uses floating point emulation. Getting consistent results at compile time regardless of whatever host/target/cross configuration trumps doing it natively.
Certainly. I understand this, and desire it in the frontend too.
May 16 2016
prev sibling parent Marco Leise <Marco.Leise gmx.de> writes:
Content-Disposition: inline

I wondered how the precision of typical SSE vs FPU code would
compare. SSE uses floats and doubles as the main mean to
control precision, while the FPU is configured via x87 control
word.

The attached program calculates 5^-441 iteratively using these
methods, then converts to double and prints the mantissa:

  Compiled with Digital Mars D in 64-bit
  float SSE : 00000000000000000000000000000000000000000000000000000
  float x87 : 00100000101010100111001110000000000000000000000000000
  double SSE: 00100000101010100111001101111011011110011011110010001
  double x87: 00100000101010100111001101111011011110011011110010000
  real x87  : 00100000101010100111001101111011011110011011110001111

Take-aways:
- SSE results generally differ from x87 results at the same
  precision
- x87 produces accurate single precision results, whereas
  the error with SSE is fatal (and that's not a flush-to-zero;
  the greatest power that still produces 1 bit of mantissa
  is 5^-64(!))

-- 
Marco
May 16 2016