www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - cast problem

reply LiuXuHong <LiuXuHong_member pathlink.com> writes:
void main(){
double fVal = 66.6f;
int rate = 10;

int intValue = cast(int)(fVal*rate);
printf("Value = %d\n", intValue);	
}

// Value = 665
// What is the matter?
Nov 17 2004
parent reply "Simon Buchan" <currently no.where> writes:
On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong  
<LiuXuHong_member pathlink.com> wrote:

 void main(){
 double fVal = 66.6f;
 int rate = 10;

 int intValue = cast(int)(fVal*rate);
 printf("Value = %d\n", intValue);	
 }

 // Value = 665
 // What is the matter?

This is what is most likely happing: fVal : 66.6 x10 : ~666.0 (ie 665.93853...) (int): 665 [ .93853 cut off] This is due to the fact that floating points are represented in binary, and therefore only values like xxx/1024 can be represented exactly, and when a floating point number is cast to an int, everything after the decemal place is simply truncated. If you must round the number properly, add .5 to it before casting. -- Using Opera's revolutionary e-mail client: http://www.opera.com/m2/
Nov 17 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Thu, 18 Nov 2004 01:09:13 +1300, Simon Buchan <currently no.where> 
wrote:
 On Wed, 17 Nov 2004 09:35:52 +0000 (UTC), LiuXuHong  
 <LiuXuHong_member pathlink.com> wrote:

 void main(){
 double fVal = 66.6f;
 int rate = 10;

 int intValue = cast(int)(fVal*rate);
 printf("Value = %d\n", intValue);	
 }

 // Value = 665
 // What is the matter?

This is what is most likely happing: fVal : 66.6 x10 : ~666.0 (ie 665.93853...) (int): 665 [ .93853 cut off] This is due to the fact that floating points are represented in binary, and therefore only values like xxx/1024 can be represented exactly, and when a floating point number is cast to an int, everything after the decemal place is simply truncated. If you must round the number properly, add .5 to it before casting.

Depending on the result you're after you can also move the cast i.e. to get 666: intValue = (cast(int)fVal*rate); (this truncates the 66.6 to 66, then does x10) to get 670: intValue = (cast(int)(fVal+0.5)*rate); (this does what Simon mentioned, then truncates, then does x10) Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Nov 17 2004
parent reply "Simon Buchan" <currently no.where> writes:
On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan netwin.co.nz> wrote:

  <snip>
 Depending on the result you're after you can also move the cast i.e.

 to get 666: intValue = (cast(int)fVal*rate);
 (this truncates the 66.6 to 66, then does x10)

 to get 670: intValue = (cast(int)(fVal+0.5)*rate);
 (this does what Simon mentioned, then truncates, then does x10)

 Regan

Actually the top one will give you 660 (since when does 66 x 10 give 666 :D), and the bottom will give you 671. (67.1 x 10) You only multiply by .5 just before the cast, of course. (cast(int)(fVal*rate + 0.5)) This gives you your 666. -- Using Opera's revolutionary e-mail client: http://www.opera.com/m2/
Nov 17 2004
parent Regan Heath <regan netwin.co.nz> writes:
On Thu, 18 Nov 2004 15:13:59 +1300, Simon Buchan <currently no.where> 
wrote:
 On Thu, 18 Nov 2004 11:39:19 +1300, Regan Heath <regan netwin.co.nz> 
 wrote:

   <snip>
 Depending on the result you're after you can also move the cast i.e.

 to get 666: intValue = (cast(int)fVal*rate);
 (this truncates the 66.6 to 66, then does x10)

 to get 670: intValue = (cast(int)(fVal+0.5)*rate);
 (this does what Simon mentioned, then truncates, then does x10)

 Regan

Actually the top one will give you 660 (since when does 66 x 10 give 666 :D),

Doh! I mean't to type that ;o)
 and the bottom will give you 671. (67.1 x 10)

No, it gives 670. The cast to int truncates 67.1 to 67 before the x10 occurs. (cast must have higher precedence than *?)
 You only multiply by .5 just before the cast, of course.  
 (cast(int)(fVal*rate + 0.5))
 This gives you your 666.

Ahh, so that is what you meant. Multiply, add 0.5 then cast to int truncating 666.x to 666. Regan -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Nov 18 2004