## digitalmars.D - double trouble

- "John C" <johnch_atms hotmail.com> Feb 25 2006
- "John C" <johnch_atms hotmail.com> Feb 25 2006
- "Lionello Lunesu" <lio remove.lunesu.com> Mar 01 2006
- xs0 <xs0 xs0.com> Mar 01 2006
- Don Clugston <dac nospam.com.au> Mar 01 2006
- "John C" <johnch_atms hotmail.com> Mar 01 2006

I'm writing routines for numeric formatting and this involves round-tripping of floating-point numbers. I'm storing the integral and fractional parts in a string buffer, and in a separate variable storing the decimal point position, and these are converted back into a double when needed. Unfortunately the algorithm I'm using is too lossy and as a result a number like 2.78 (stored in the buffer as "278", with the decimal pos being 1) becomes 2.7799999999998. I believe the problem is with my scaling code. Any ideas to increase the accuracy? // v is a numeric representation of the digits, eg 278. // exponent is either the exponent for scientific numbers, or the number of fractional digits, eg 2. double p10 = 10.0; int n = exponent; if (n < 0) n = -n; while (n) { if (n & 1) { if (exponent < 0) v /= p10; else v *= p10; } n >>= 1; p10 *= p10; }

Feb 25 2006

I believe the problem is with my scaling code. Any ideas to increase the accuracy?

Scratch that. It must be the way I'm converting the digits to a double that introduces the inaccuracy. // pDigits is a pointer into a zero-terminated buffer (with the digits "278"). double v = 0.0; while (*pDigits) { v = v * 10 + (*pDigits - '0'); pDigits++; } v is now 277999999999999.999990. Then it's scaled using the code previously posted to get 2.7799999999999998. How would I round this up to the original 2.78?

Feb 25 2006

I would get rid of the while, or at least the /= and *= therein, these will only add to the inaccuracy. Better keep track of integers (++n, --n) and use pow when you need the 10^n or so. Remember that the 10.0 is already inaccurate (imagine it's 9.999 or so), using it multiple times will definately add to the inaccuracy. L.

Mar 01 2006

Lionello Lunesu wrote:I would get rid of the while, or at least the /= and *= therein, these will only add to the inaccuracy. Better keep track of integers (++n, --n) and use pow when you need the 10^n or so. Remember that the 10.0 is already inaccurate (imagine it's 9.999 or so), using it multiple times will definately add to the inaccuracy. L.

Actually, 10.0 is accurately representable, but 0.1 isn't.. Perhaps the most accurate way to calculate decimal digits would be digit/pow(10.0, -exp) instead of digit*pow(0.1, exp), avoiding exponentiating an inaccurate number.. Well, I went ahead and tested it (in Java, but the results should match D's double). Using digit*constant (0.000...0001) and digit/pow(10, exp) produce about the same results, while repeatedly multiplying with 0.1 and exponentiation of 0.1 also produce about the same results, except the latter two are slightly more off. http://www.xs0.com/prec/ xs0

Mar 01 2006

John C wrote:I'm writing routines for numeric formatting and this involves round-tripping of floating-point numbers. I'm storing the integral and fractional parts in a string buffer, and in a separate variable storing the decimal point position, and these are converted back into a double when needed. Unfortunately the algorithm I'm using is too lossy and as a result a number like 2.78 (stored in the buffer as "278", with the decimal pos being 1) becomes 2.7799999999998.

The trick to maximal efficiency in these conversions is to make sure that you only do ONE division (because that's the point at which the rounding error occurs). Don't divide by 10 every time, and definitely don't use pow. Instead, keep track of a denominator, and every time you'd do a v/=10, do denominator*=10 instead. Then, right at the end, divide v by denominator. The reason this works is that integers can be exactly represented in reals, so the multiplies aren't introducing any error. But every time you divide by something that isn't a multiple of 2, a tiny roundoff error creeps in. You'll also reduce the error if you use real v=0.0; instead of double. Even if you ultimately want a double.I believe the problem is with my scaling code. Any ideas to increase the accuracy? // v is a numeric representation of the digits, eg 278. // exponent is either the exponent for scientific numbers, or the number of fractional digits, eg 2. double p10 = 10.0; int n = exponent; if (n < 0) n = -n; while (n) { if (n & 1) { if (exponent < 0) v /= p10; else v *= p10; } n >>= 1; p10 *= p10; }

Mar 01 2006

"Don Clugston" <dac nospam.com.au> wrote in message news:du48kq$i5i$1 digitaldaemon.com...The trick to maximal efficiency in these conversions is to make sure that you only do ONE division (because that's the point at which the rounding error occurs). Don't divide by 10 every time, and definitely don't use pow. Instead, keep track of a denominator, and every time you'd do a v/=10, do denominator*=10 instead. Then, right at the end, divide v by denominator. The reason this works is that integers can be exactly represented in reals, so the multiplies aren't introducing any error. But every time you divide by something that isn't a multiple of 2, a tiny roundoff error creeps in. You'll also reduce the error if you use real v=0.0; instead of double. Even if you ultimately want a double.

Thanks. Working on the raw bits (actually a 64-bit integer) eventually proved easier (relatively speaking).

Mar 01 2006