## digitalmars.D - double trouble

• John C (26/26) Feb 25 2006 I'm writing routines for numeric formatting and this involves round-trip...
• John C (12/14) Feb 25 2006 Scratch that. It must be the way I'm converting the digits to a double t...
• Lionello Lunesu (6/6) Mar 01 2006 I would get rid of the while, or at least the /= and *= therein, these w...
• xs0 (13/22) Mar 01 2006 Actually, 10.0 is accurately representable, but 0.1 isn't..
• Don Clugston (14/44) Mar 01 2006 The trick to maximal efficiency in these conversions is to make sure
• John C (4/17) Mar 01 2006 Thanks. Working on the raw bits (actually a 64-bit integer) eventually
"John C" <johnch_atms hotmail.com> writes:
```I'm writing routines for numeric formatting and this involves round-tripping
of floating-point numbers. I'm storing the integral and fractional parts in
a string buffer, and in a separate variable storing the decimal point
position, and these are converted back into a double when needed.
Unfortunately the algorithm I'm using is too lossy and as a result a number
like 2.78 (stored in the buffer as "278", with the decimal pos being 1)
becomes 2.7799999999998.

I believe the problem is with my scaling code. Any ideas to increase the
accuracy?

// v is a numeric representation of the digits, eg 278.
// exponent is either the exponent for scientific numbers, or the number
of fractional digits, eg 2.
double p10 = 10.0;
int n = exponent;
if (n < 0)
n = -n;
while (n) {
if (n & 1) {
if (exponent < 0)
v /= p10;
else
v *= p10;
}
n >>= 1;
p10 *= p10;
}
```
Feb 25 2006
"John C" <johnch_atms hotmail.com> writes:
``` I believe the problem is with my scaling code. Any ideas to increase the
accuracy?

Scratch that. It must be the way I'm converting the digits to a double that
introduces the inaccuracy.

// pDigits is a pointer into a zero-terminated buffer (with the digits
"278").
double v = 0.0;
while (*pDigits) {
v = v * 10 + (*pDigits - '0');
pDigits++;
}

v is now 277999999999999.999990. Then it's scaled using the code previously
posted to get 2.7799999999999998. How would I round this up to the original
2.78?
```
Feb 25 2006
"Lionello Lunesu" <lio remove.lunesu.com> writes:
```I would get rid of the while, or at least the /= and *= therein, these will
only add to the inaccuracy. Better keep track of integers (++n, --n) and use
pow when you need the 10^n or so. Remember that the 10.0 is already
inaccurate (imagine it's 9.999 or so), using it multiple times will
definately add to the inaccuracy.

L.
```
Mar 01 2006
xs0 <xs0 xs0.com> writes:
```Lionello Lunesu wrote:
I would get rid of the while, or at least the /= and *= therein, these will
only add to the inaccuracy. Better keep track of integers (++n, --n) and use
pow when you need the 10^n or so. Remember that the 10.0 is already
inaccurate (imagine it's 9.999 or so), using it multiple times will
definately add to the inaccuracy.

L.

Actually, 10.0 is accurately representable, but 0.1 isn't..
Perhaps the most accurate way to calculate decimal digits would be

digit/pow(10.0, -exp)

instead of digit*pow(0.1, exp), avoiding exponentiating an inaccurate
number..

Well, I went ahead and tested it (in Java, but the results should match
D's double). Using digit*constant (0.000...0001) and digit/pow(10, exp)
produce about the same results, while repeatedly multiplying with 0.1
and exponentiation of 0.1 also produce about the same results, except
the latter two are slightly more off.

http://www.xs0.com/prec/

xs0
```
Mar 01 2006
Don Clugston <dac nospam.com.au> writes:
```John C wrote:
I'm writing routines for numeric formatting and this involves round-tripping
of floating-point numbers. I'm storing the integral and fractional parts in
a string buffer, and in a separate variable storing the decimal point
position, and these are converted back into a double when needed.
Unfortunately the algorithm I'm using is too lossy and as a result a number
like 2.78 (stored in the buffer as "278", with the decimal pos being 1)
becomes 2.7799999999998.

The trick to maximal efficiency in these conversions is to make sure
that you only do ONE division (because that's the point at which the
rounding error occurs). Don't divide by 10 every time, and definitely
don't use pow. Instead, keep track of a denominator, and every time
you'd do a v/=10, do denominator*=10 instead. Then, right at the end,
divide v by denominator.
The reason this works is that integers can be exactly represented in
reals, so the multiplies aren't introducing any error. But every time
you divide by something that isn't a multiple of 2, a tiny roundoff
error creeps in.
You'll also reduce the error if you use
real v=0.0;
instead of double. Even if you ultimately want a double.

I believe the problem is with my scaling code. Any ideas to increase the
accuracy?

// v is a numeric representation of the digits, eg 278.
// exponent is either the exponent for scientific numbers, or the number
of fractional digits, eg 2.
double p10 = 10.0;
int n = exponent;
if (n < 0)
n = -n;
while (n) {
if (n & 1) {
if (exponent < 0)
v /= p10;
else
v *= p10;
}
n >>= 1;
p10 *= p10;
}

```
Mar 01 2006
"John C" <johnch_atms hotmail.com> writes:
```"Don Clugston" <dac nospam.com.au> wrote in message
news:du48kq\$i5i\$1 digitaldaemon.com...
The trick to maximal efficiency in these conversions is to make sure that
you only do ONE division (because that's the point at which the rounding
error occurs). Don't divide by 10 every time, and definitely don't use
pow. Instead, keep track of a denominator, and every time you'd do a
v/=10, do denominator*=10 instead. Then, right at the end, divide v by
denominator.
The reason this works is that integers can be exactly represented in
reals, so the multiplies aren't introducing any error. But every time you
divide by something that isn't a multiple of 2, a tiny roundoff error
creeps in.
You'll also reduce the error if you use
real v=0.0;
instead of double. Even if you ultimately want a double.

Thanks. Working on the raw bits (actually a 64-bit integer) eventually
proved easier (relatively speaking).
```
Mar 01 2006