digitalmars.D.learn - Accuracy of floating point calculations
- Twilight (14/14) Oct 29 2019 D calculation:
- Daniel Kozak (11/25) Oct 29 2019 If you use gdc or ldc you will get same results as c++, or you can use
- Daniel Kozak (2/11) Oct 29 2019 AFAIK dmd use real for floating point operations instead of double
- ixid (4/21) Oct 29 2019 Given x87 is deprecated and has been recommended against since
- H. S. Teoh (6/25) Oct 29 2019 Walter talked about this recently as one of the "misses" in D (one of
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (13/25) Oct 30 2019 Why should the real type be a wrong decision? Maybe the code generation
- H. S. Teoh (27/52) Oct 30 2019 It wasn't a wrong *decision* per se, but a wrong *prediction* of where
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (20/38) Oct 31 2019 Yes, which is wondering me as well... but all the AI stuff seems to
- H. S. Teoh (16/32) Oct 31 2019 Maybe you might be interested in this:
- =?iso-8859-1?Q?Robert_M._M=FCnch?= (7/11) Oct 31 2019 Well, our customers don't use PPC Laptops ;-) otherwise that would be co...
- Twilight (6/23) Oct 29 2019 Thanks for the clarification. It appears then that because of
- H. S. Teoh (36/57) Oct 29 2019 Yes, it will be somewhat more accurate, depending on the exact
- Daniel Kozak (15/48) Oct 29 2019 import std.math : pow;
D calculation:
writefln("%12.2F",log(1-0.9999)/log(1-(1-0.6)^^20));
837675572.38
C++ calculation:
cout<<setprecision(12)<< (log(1-0.9999)/log(1-pow(1-0.6,20)))
<<'\n';
837675573.587
As a second data point, changing 0.9999 to 0.75 yields
126082736.96 (Dlang) vs 126082737.142 (C++).
The discrepancy stood out as I was ultimately taking the ceil of
the results and noticed an off by one anomaly. Testing with
octave, www.desmos.com/scientific, and libreoffice(calc) gave
results consistent with the C++ result. Is the dlang calculation
within the error bound of what double precision should yield?
Oct 29 2019
Daniel Kozak <kozzi11 gmail.com>
On Tue, Oct 29, 2019 at 4:45 PM Twilight via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:
D calculation:
writefln("%12.2F",log(1-0.9999)/log(1-(1-0.6)^^20));
837675572.38
C++ calculation:
cout<<setprecision(12)<< (log(1-0.9999)/log(1-pow(1-0.6,20)))
<<'\n';
837675573.587
As a second data point, changing 0.9999 to 0.75 yields
126082736.96 (Dlang) vs 126082737.142 (C++).
The discrepancy stood out as I was ultimately taking the ceil of
the results and noticed an off by one anomaly. Testing with
octave, www.desmos.com/scientific, and libreoffice(calc) gave
results consistent with the C++ result. Is the dlang calculation
within the error bound of what double precision should yield?
If you use gdc or ldc you will get same results as c++, or you can use
C log directly:
import std.stdio;
import std.math : pow;
import core.stdc.math;
void main()
{
writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20));
}
Oct 29 2019
On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:
If you use gdc or ldc you will get same results as c++, or you can use
C log directly:
import std.stdio;
import std.math : pow;
import core.stdc.math;
void main()
{
writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20));
}
AFAIK dmd use real for floating point operations instead of double
Oct 29 2019
On Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:Given x87 is deprecated and has been recommended against since 2003 at the latest it's hard to understand why this could be seen as a good idea.If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }AFAIK dmd use real for floating point operations instead of double
Oct 29 2019
On Tue, Oct 29, 2019 at 04:54:23PM +0000, ixid via Digitalmars-d-learn wrote:On Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:Walter talked about this recently as one of the "misses" in D (one of the things he predicted wrongly when he designed it). T -- Philosophy: how to make a career out of daydreaming.On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:Given x87 is deprecated and has been recommended against since 2003 at the latest it's hard to understand why this could be seen as a good idea.If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }AFAIK dmd use real for floating point operations instead of double
Oct 29 2019
On 2019-10-29 17:43:47 +0000, H. S. Teoh said:On Tue, Oct 29, 2019 at 04:54:23PM +0000, ixid via Digitalmars-d-learn wrote:Why should the real type be a wrong decision? Maybe the code generation should be optimized if all terms are double to avoid x87 but overall more precision is better for some use-cases. I'm very happpy it exists, and x87 too because our app really needs this extended precision. I'm not sure what we would do if we only had doubles. I'm not aware of any 128 bit real implementations done using SIMD instructions which get good speed. Anyone? -- Robert M. Münch http://www.saphirion.com smarter | better | fasterOn Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:Walter talked about this recently as one of the "misses" in D (one of the things he predicted wrongly when he designed it).On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote: AFAIK dmd use real for floating point operations instead of doubleGiven x87 is deprecated and has been recommended against since 2003 at the latest it's hard to understand why this could be seen as a good idea.
Oct 30 2019
On Wed, Oct 30, 2019 at 09:03:49AM +0100, Robert M. Münch via Digitalmars-d-learn wrote:On 2019-10-29 17:43:47 +0000, H. S. Teoh said:It wasn't a wrong *decision* per se, but a wrong *prediction* of where the industry would be headed. Walter was expecting that people would move towards higher precision, but what with SSE2 and other such trends, and the general neglect of x87 in hardware developments, it appears that people have been moving towards 64-bit doubles rather than 80-bit extended. Though TBH, my opinion is that it's not so much neglecting higher precision, but a general sentiment of the recent years towards standardization, i.e., to be IEEE-compliant (64-bit floating point) rather than work with a non-standard format (80-bit x87 reals). I also would prefer to have higher precision, but it would be nicer if that higher precision was a standard format with guaranteed semantics that isn't dependent upon a single vendor or implementation.On Tue, Oct 29, 2019 at 04:54:23PM +0000, ixid via Digitalmars-d-learn wrote:Why should the real type be a wrong decision? Maybe the code generation should be optimized if all terms are double to avoid x87 but overall more precision is better for some use-cases.On Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:Walter talked about this recently as one of the "misses" in D (one of the things he predicted wrongly when he designed it).On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote: AFAIK dmd use real for floating point operations instead of doubleGiven x87 is deprecated and has been recommended against since 2003 at the latest it's hard to understand why this could be seen as a good idea.I'm very happpy it exists, and x87 too because our app really needs this extended precision. I'm not sure what we would do if we only had doubles. I'm not aware of any 128 bit real implementations done using SIMD instructions which get good speed. Anyone?[...] Do you mean *simulated* 128-bit reals (e.g. with a pair of 64-bit doubles), or do you mean actual IEEE 128-bit reals? 'cos the two are different, semantically. I'm still longing for 128-bit reals (i.e., actual IEEE 128-bit format) to show up in x86, but I'm not holding my breath. In the meantime, I've been looking into arbitrary-precision float libraries like libgmp instead. It's software-simulated, and therefore slower, but for certain applications where I want very high precision, it's the currently the only option. T -- If Java had true garbage collection, most programs would delete themselves upon execution. -- Robert Sewell
Oct 30 2019
On 2019-10-30 15:12:29 +0000, H. S. Teoh said:It wasn't a wrong *decision* per se, but a wrong *prediction* of where the industry would be headed.Fair point...Walter was expecting that people would move towards higher precision, but what with SSE2 and other such trends, and the general neglect of x87 in hardware developments, it appears that people have been moving towards 64-bit doubles rather than 80-bit extended.Yes, which is wondering me as well... but all the AI stuff seems to dominate the game and follow the hype is still a frequently used management strategy.Though TBH, my opinion is that it's not so much neglecting higher precision, but a general sentiment of the recent years towards standardization, i.e., to be IEEE-compliant (64-bit floating point) rather than work with a non-standard format (80-bit x87 reals).I see it more of a "let's sell what people want". The CPU vendors don't seem able to market higher precision. Better implement a highly-specific and exploding command-set...Do you mean *simulated* 128-bit reals (e.g. with a pair of 64-bit doubles), or do you mean actual IEEE 128-bit reals?Simulated, because HW support is lacking on X86. And PPC is not that mainstream. I exect Apple to move to ARM, but never heard about 128-Bit support for ARM.I'm still longing for 128-bit reals (i.e., actual IEEE 128-bit format) to show up in x86, but I'm not holding my breath.Me too.In the meantime, I've been looking into arbitrary-precision float libraries like libgmp instead. It's software-simulated, and therefore slower, but for certain applications where I want very high precision, it's the currently the only option.Yes, but it's way too slow for our product. Maybe one day we need to deliver an FPGA based co-processor PCI card that can run 128-Bit based calculations... but that will be a pretty hard way to go. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Oct 31 2019
On Thu, Oct 31, 2019 at 09:52:08AM +0100, Robert M. Münch via Digitalmars-d-learn wrote:On 2019-10-30 15:12:29 +0000, H. S. Teoh said:[...]Maybe you might be interested in this: https://stackoverflow.com/questions/6769881/emulate-double-using-2-floats It's mostly talking about simulating 64-bit floats where the hardware only supports 32-bit floats, but the same principles apply for simulating 128-bit floats with 64-bit hardware. [...]Do you mean *simulated* 128-bit reals (e.g. with a pair of 64-bit doubles), or do you mean actual IEEE 128-bit reals?Simulated, because HW support is lacking on X86. And PPC is not that mainstream. I exect Apple to move to ARM, but never heard about 128-Bit support for ARM.Fair point. In my case I'm mainly working with batch-oriented processing, so a slight slowdown is an acceptable tradeoff for higher accuracy.In the meantime, I've been looking into arbitrary-precision float libraries like libgmp instead. It's software-simulated, and therefore slower, but for certain applications where I want very high precision, it's the currently the only option.Yes, but it's way too slow for our product.Maybe one day we need to deliver an FPGA based co-processor PCI card that can run 128-Bit based calculations... but that will be a pretty hard way to go.[...] Maybe switch to PPC? :-D T -- If you want to solve a problem, you need to address its root cause, not just its symptoms. Otherwise it's like treating cancer with Tylenol...
Oct 31 2019
On 2019-10-31 16:07:07 +0000, H. S. Teoh said:Maybe you might be interested in this: https://stackoverflow.com/questions/6769881/emulate-double-using-2-floatsThanks, I know the 2nd mentioned paper.Maybe switch to PPC? :-DWell, our customers don't use PPC Laptops ;-) otherwise that would be cool. -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Oct 31 2019
On Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:Thanks for the clarification. It appears then that because of dmd's real calculations, it produces more accurate results, but maybe slower. (Calculating the result with the high precision calculator at https://keisan.casio.com/calculator agrees with dmd.)If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }AFAIK dmd use real for floating point operations instead of double
Oct 29 2019
On Tue, Oct 29, 2019 at 07:10:08PM +0000, Twilight via Digitalmars-d-learn wrote:On Tuesday, 29 October 2019 at 16:11:45 UTC, Daniel Kozak wrote:Yes, it will be somewhat more accurate, depending on the exact calculation you're performing. But it depends on the x87 coprocessor, which hasn't been improved for many years now, and not much attention has been paid to it, so it would appear that 64-bit double arithmetic using SIMD or MMX instructions would probably run faster. (I'd profile it just to be sure, though. Sometimes performance predictions can be very wrong.) So roughly speaking, if you want accuracy, use real, if you want speed, use float or double.On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:Thanks for the clarification. It appears then that because of dmd's real calculations, it produces more accurate results, but maybe slower.If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }AFAIK dmd use real for floating point operations instead of double(Calculating the result with the high precision calculator at https://keisan.casio.com/calculator agrees with dmd.)To verify accuracy, it's usually safer to use an arbitrary-precision calculator instead of assuming that the most common answer is the right one (it may be far off, depending on what exactly is being computed and how, e.g., due to catastrophic cancellation and things like that). Like `bc -l` if you're running *nix, e.g. the input: scale=32 l(1 - 0.9999) / l(1 - (1 - 0.6)^20) gives: 837675572.37859373067028812966306043501772 So it appears that the C++ answer is less accurate. Note that not all of the digits above are trustworthy; running the same calculation with scale=100 gives: 837675572.3785937306702880546932327627909527172023597021486261165664\ 994508853029795054669322261827298817174322 which shows a divergence in digits after the 15th decimal, meaning that the subsequent digits are probably garbage values. This is probably because the logarithm function near 0 is poorly-conditioned, so you could potentially be getting complete garbage from your floating-point operations if you're not careful. Floating-point is a bear. Every programmer should learn to tame it lest they get mauled: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html :-D T -- May you live all the days of your life. -- Jonathan Swift
Oct 29 2019
On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:On Tue, Oct 29, 2019 at 4:45 PM Twilight via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }D calculation: mport std.stdio;My fault, for ldc and gdc you will get same result as C++ only when you use pow not ^^(operator) and use doubles: import std.stdio; import std.math; void main() { writefln("%12.3F",log(1-0.9999)/log((1-pow(1-0.6,20)))); }writefln("%12.2F",log(1-0.9999)/log(1-(1-0.6)^^20)); 837675572.38 C++ calculation: cout<<setprecision(12)<< (log(1-0.9999)/log(1-pow(1-0.6,20))) <<'\n'; 837675573.587 As a second data point, changing 0.9999 to 0.75 yields 126082736.96 (Dlang) vs 126082737.142 (C++). The discrepancy stood out as I was ultimately taking the ceil of the results and noticed an off by one anomaly. Testing with octave, www.desmos.com/scientific, and libreoffice(calc) gave results consistent with the C++ result. Is the dlang calculation within the error bound of what double precision should yield?If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }
Oct 29 2019
On Tuesday, 29 October 2019 at 16:20:21 UTC, Daniel Kozak wrote:On Tue, Oct 29, 2019 at 5:09 PM Daniel Kozak <kozzi11 gmail.com> wrote:The real issue here IMO is that there's still only a `real` version of std.math.log. If there were proper double and float overloads, like for other std.math functions, the OP would get the expected result with his double inputs, and we wouldn't be having this discussion. For LDC, it would only mean uncommenting 2 one-liners forwarding to the LLVM intrinsic; they're commented because otherwise you'd get different results with LDC compared to DMD, and other forum threads/bugzillas/GitHub issues would pop up. Note that there's at least one bugzilla for these float/double math overloads already. For a start, one could simply wrap the corresponding C functions.On Tue, Oct 29, 2019 at 4:45 PM Twilight via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }D calculation: mport std.stdio;My fault, for ldc and gdc you will get same result as C++ only when you use pow not ^^(operator) and use doubles: import std.stdio; import std.math; void main() { writefln("%12.3F",log(1-0.9999)/log((1-pow(1-0.6,20)))); }writefln("%12.2F",log(1-0.9999)/log(1-(1-0.6)^^20)); 837675572.38 C++ calculation: cout<<setprecision(12)<< (log(1-0.9999)/log(1-pow(1-0.6,20))) <<'\n'; 837675573.587 As a second data point, changing 0.9999 to 0.75 yields 126082736.96 (Dlang) vs 126082737.142 (C++). The discrepancy stood out as I was ultimately taking the ceil of the results and noticed an off by one anomaly. Testing with octave, www.desmos.com/scientific, and libreoffice(calc) gave results consistent with the C++ result. Is the dlang calculation within the error bound of what double precision should yield?If you use gdc or ldc you will get same results as c++, or you can use C log directly: import std.stdio; import std.math : pow; import core.stdc.math; void main() { writefln("%12.3F",log(1-0.9999)/log(1-(1-0.6)^^20)); }
Oct 29 2019
On Tuesday, 29 October 2019 at 20:15:13 UTC, kinke wrote:Note that there's at least one bugzilla for these float/double math overloads already. For a start, one could simply wrap the corresponding C functions.I guess, that this issue: https://issues.dlang.org/show_bug.cgi?id=20206 boils down to the same problem. I allready found out, that it's high likely that the bug is not to be found inside std.complex but probably the log function.
Oct 30 2019