www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - wrong rounding

reply "Oleg B" <code.viator gmail.com> writes:
Hello. I found unexpected (for me) behavior of rounding double 
values at casting to ulong:

$ cat roundtest.d
import std.stdio;

void main()
{
     double a = 10, b = 0.01;

     writeln( "int:   ", cast(int)(a/b) );
     writeln( "uint:  ", cast(uint)(a/b) );
     writeln( "long:  ", cast(long)(a/b) );
     writeln( "ulong: ", cast(ulong)(a/b) );
}

$ rdmd roundtest.d
int:   1000
uint:  1000
long:  1000
ulong: 999   <----- WTF?? -------

$ dmd --version
DMD64 D Compiler v2.067.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright

$ ldc2 roundtest.d && ./roundtest
int:   1000
uint:  1000
long:  1000
ulong: 1000  <------- Ok --------

$ ldc2 --version
LDC - the LLVM D compiler (0.15.2-beta1):
   based on DMD v2.066.1 and LLVM 3.6.0
   Default target: x86_64-unknown-linux-gnu
   Host CPU: core-avx-i
   http://dlang.org - http://wiki.dlang.org/LDC

   Registered Targets:
     x86    - 32-bit X86: Pentium-Pro and above
     x86-64 - 64-bit X86: EM64T and AMD64

$ uname -a

2015 x86_64 x86_64 x86_64 GNU/Linux

have it behavior explain? can I use casting in dmd for rounding 
values or it not safe?
Jun 01 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 5:29 PM, Oleg B wrote:
 Hello. I found unexpected (for me) behavior of rounding double values at
 casting to ulong:

 $ cat roundtest.d
 import std.stdio;

 void main()
 {
      double a = 10, b = 0.01;

      writeln( "int:   ", cast(int)(a/b) );
      writeln( "uint:  ", cast(uint)(a/b) );
      writeln( "long:  ", cast(long)(a/b) );
      writeln( "ulong: ", cast(ulong)(a/b) );
 }

 $ rdmd roundtest.d
 int:   1000
 uint:  1000
 long:  1000
 ulong: 999   <----- WTF?? -------
These are NOT roundings. They are truncations. Note that for floating point 0.01 is not representable exactly. This is the reason you get the error. On other systems, you may not get errors for this one case, but you could get errors for other. Please read about floating point error: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems -Steve
Jun 01 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 1 June 2015 at 21:44:57 UTC, Steven Schveighoffer 
wrote:
 On 6/1/15 5:29 PM, Oleg B wrote:
 Hello. I found unexpected (for me) behavior of rounding double 
 values at
 casting to ulong:

 $ cat roundtest.d
 import std.stdio;

 void main()
 {
     double a = 10, b = 0.01;

     writeln( "int:   ", cast(int)(a/b) );
     writeln( "uint:  ", cast(uint)(a/b) );
     writeln( "long:  ", cast(long)(a/b) );
     writeln( "ulong: ", cast(ulong)(a/b) );
 }

 $ rdmd roundtest.d
 int:   1000
 uint:  1000
 long:  1000
 ulong: 999   <----- WTF?? -------
These are NOT roundings. They are truncations. Note that for floating point 0.01 is not representable exactly. This is the reason you get the error. On other systems, you may not get errors for this one case, but you could get errors for other. Please read about floating point error: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems -Steve
Nonetheless, surely in this case the values should be the same regardless of the integer target type, no?
Jun 01 2015
next sibling parent Manfred Nowak <svv1999 hotmail.com> writes:
John Colvin wrote:

 regardless of the integer target type
on win64 dmd 2.067.1 gives correct results int: 999 uint: 999 long: 999 ulong: 999 -manfred
Jun 01 2015
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/1/15 6:24 PM, John Colvin wrote:
 Nonetheless, surely in this case the values should be the same
 regardless of the integer target type, no?
Hm... yeah probably :) I didn't grok the example thoroughly. Makes me curious actually, and that does seem like a bug. Doing some testing, if you assign a/b to a double and cast, it works. If you use a function to truncate taking real as a parameter, it returns 999 in both ulong and long cases. The assembly is too much over my head to figure out where the difference is. I think you should file a codegen bug. -Steve
Jun 01 2015
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:
 Nonetheless, surely in this case the values should be the same 
 regardless of the integer target type, no?
Not sure... the spec says that the compiler is allowed to do intermediate computations at higher precision, but it doesn't say it is required to. So, strictly speaking, its at the compiler's discretion to use higher precision when rounding to `int` than when rounding to `ulong`...
Jun 02 2015
next sibling parent "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
Interestingly dmd computes the a/b with SSE instructions if the 
result is cast to int, uint or long, but uses x87 instructions 
for the division if the result is cast to ulong.
Jun 02 2015
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/2/15 5:53 AM, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" wrote:
 On Monday, 1 June 2015 at 22:24:31 UTC, John Colvin wrote:
 Nonetheless, surely in this case the values should be the same
 regardless of the integer target type, no?
Not sure... the spec says that the compiler is allowed to do intermediate computations at higher precision, but it doesn't say it is required to. So, strictly speaking, its at the compiler's discretion to use higher precision when rounding to `int` than when rounding to `ulong`...
Right, but there is a difference between ulong and long. These should be the same exact code. Something definitely isn't right. -Steve
Jun 02 2015