www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - cast(int) getting an unexpected number

reply Joel Christensen <joelcnz gmail.com> writes:
Input:
import std.stdio;
void main() {
	float f=0.01;
	writefln("%0.2f->%d",f,cast(int)(f*100f));
}

Output:
0.01->0

Comment:
What!?
Nov 04 2009
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz gmail.com>  
wrote:

 Input:
 import std.stdio;
 void main() {
 	float f=0.01;
 	writefln("%0.2f->%d",f,cast(int)(f*100f));
 }

 Output:
 0.01->0

 Comment:
 What!?
hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal. If you imagined that the computer stored things in decimal, see how the 1/3 example would work a = 1.0/3; // truncated to 0.3333333 a *= 3; // 0.9999999 auto i = cast(int)a; // 0 This is analagous to what you are asking the compiler to do. To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -Steve
Nov 04 2009
next sibling parent reply rmcguire <rjmcguire gmail.com> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:
 
 On Wed, 04 Nov 2009 06:14:36 -0500, Joel Christensen <joelcnz gmail.com>  
 wrote:
 
 Input:
 import std.stdio;
 void main() {
  float f=0.01;
  writefln("%0.2f->%d",f,cast(int)(f*100f));
 }

 Output:
 0.01->0

 Comment:
 What!?
hehe. .01 is not exactly represented by float, because it's stored as a binary value, not a decimal value. Think about how there is no way to represent 1/3 in decimal. If you imagined that the computer stored things in decimal, see how the 1/3 example would work a = 1.0/3; // truncated to 0.3333333 a *= 3; // 0.9999999 auto i = cast(int)a; // 0 This is analagous to what you are asking the compiler to do. To be safe, whenever converting to int, always add a small epsilon. I think you can use float.epsilon, but I don't have any experience with whether that is always reasonable. -Steve
why is this not a compiler bug? because: import std.stdio; void main() { float f=0.01; writefln("%0.2f->%d",f,cast(int)(f*100f)); writefln("%0.2f->%d",f,cast(int)(.01*100f)); writefln("%0.2f->%f",f,(f*100f)); } results in: 0.01->0 0.01->1 0.01->1.000000 I would say something is dodgy. -Rory
Nov 04 2009
next sibling parent reply Michal Minich <michal minich.sk> writes:
Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.
 
 -Rory
 
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time. http://www.digitalmars.com/d/2.0/function.html#interpretation It seems that .01*100f is computed at compile time. try declarting float f as const or invariant and see what happens (I'm not sure..)
Nov 04 2009
parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Michal Minich wrote:
 Hello rmcguire,
 
 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Nov 04 2009
parent reply Charles Hixson <charleshixsn earthlink.net> writes:
Lars T. Kyllingstad wrote:
 Michal Minich wrote:
 Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
Nov 04 2009
parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Charles Hixson wrote:
 Lars T. Kyllingstad wrote:
 Michal Minich wrote:
 Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -Lars
Nov 04 2009
parent reply rmcguire <rjmcguire gmail.com> writes:
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:
 
 Charles Hixson wrote:
 Lars T. Kyllingstad wrote:
 Michal Minich wrote:
 Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -Lars
Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator. -Rory
Nov 04 2009
parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
rmcguire wrote:
 "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:
  
 Charles Hixson wrote:
 Lars T. Kyllingstad wrote:
 Michal Minich wrote:
 Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -Lars
Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator.
You don't, really. You just stay away from casts. :) IMHO, casts are only good for hammering a value of one type into a different type, no worries about the result. I (almost) never use cast(int) to round floating-point numbers. There are two reasons for this: 1. It always rounds towards zero, which is usually not what I want. 2. It doesn't perform any checks, and can easily overflow. If the rounding mode is not important, or if for some reason I want the same result as cast(int), I use std.conv.to(). This function checks that the converted number fits in the target type and then performs an ordinary cast. (Note: D2 only!) import std.conv; void main() { int i = to!int(1.23); // OK assert (i == 1); int j = to!int(real.max); // ConvOverflowError } If "natural" rounding, i.e. towards nearest integer, is what I want, then I use std.math.lround() or std.math.round(). There are a lot of rounding functions in std.math that do different things and that are worth checking out. import std.math; void main() { long i = lround(1.23); assert (i == 1); long j = lround(0.5); assert (j == 1); } -Lars
Nov 04 2009
parent rmcguire <rjmcguire gmail.com> writes:
"Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:
 
 rmcguire wrote:
 "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> wrote:
  
 Charles Hixson wrote:
 Lars T. Kyllingstad wrote:
 Michal Minich wrote:
 Hello rmcguire,

 why is this not a compiler bug?
 because:
 import std.stdio;
 void main() {
 float f=0.01;
 writefln("%0.2f->%d",f,cast(int)(f*100f));
 writefln("%0.2f->%d",f,cast(int)(.01*100f));
 writefln("%0.2f->%f",f,(f*100f));
 }
 results in:
 0.01->0
 0.01->1
 0.01->1.000000
 I would say something is dodgy.

 -Rory
I think this may be case of: At comple time floating point computations may be done at a higher precision than run time.
Yes, if you do this: float f = 0.01; float g = f * 100f; real r = f * 100f; writeln("%s, %s, %s", f, cast(int) g, cast(int) r); you get: 0.01, 0, 1 I believe just writing cast(int)(f*100f) is more or less the same as the 'real' case above. -Lars
Can that *really* be the explanation?? I know that float doesn't have all that much precision, but I thought it was more than 5 or 6 places...and this is, essentially, two places.
Yes it does, but that's not what matters. Say that for floats, the representable number closest to 0.01 is 0.01000000000001. (This is just an example, I don't know the true number.) Then you have a lot more precision than the two digits you mention, and multiplying with 100 gives 1.000000000001. Round this towards zero (which is what cast(int) does) and you get 1. This is the 'float g = f*100f;' case in my example. Now, say that for reals, which (I think) is what the compiler uses internally, the number closest to 0.01 is 0.0099999999999999999999999999999999999999. Again, just an example, the point is that the precision is higher than the above, but the closest number is now smaller than 0.01. Multiply this by 100, and you get 0.99999999999999999999999999999999999999. This number will be cast to the integer 0, which happens in the OP's case. I admit I'm no expert in these things, but I suspect this is how it goes. By the way, I recommend Don's excellent article on floating-point numbers. It has really cleared things up for me: http://www.digitalmars.com/d/2.0/d-floating-point.html -Lars
Hmm, thanks. pity its not consistent. Would be nice if you didn't have to learn everything about floating point just to make a calculator.
You don't, really. You just stay away from casts. :) IMHO, casts are only good for hammering a value of one type into a different type, no worries about the result. I (almost) never use cast(int) to round floating-point numbers. There are two reasons for this: 1. It always rounds towards zero, which is usually not what I want. 2. It doesn't perform any checks, and can easily overflow. If the rounding mode is not important, or if for some reason I want the same result as cast(int), I use std.conv.to(). This function checks that the converted number fits in the target type and then performs an ordinary cast. (Note: D2 only!) import std.conv; void main() { int i = to!int(1.23); // OK assert (i == 1); int j = to!int(real.max); // ConvOverflowError } If "natural" rounding, i.e. towards nearest integer, is what I want, then I use std.math.lround() or std.math.round(). There are a lot of rounding functions in std.math that do different things and that are worth checking out. import std.math; void main() { long i = lround(1.23); assert (i == 1); long j = lround(0.5); assert (j == 1); } -Lars
err, right. good point, hehe.
Nov 05 2009
prev sibling next sibling parent Ali Cehreli <acehreli yahoo.com> writes:
rmcguire Wrote:

 why is this not a compiler bug?
 because:
 import std.stdio;
 
 void main() {
         float f=0.01;
 results in:
 0.01->0
0.01 is double, there is type conversion there.
         writefln("%0.2f->%d",f,cast(int)(f*100f));
 results in:
 0.01->1
f*100f is represented to be less than 0.
         writefln("%0.2f->%d",f,cast(int)(.01*100f));
.01 * 100f is double and apparently represented as something more than 1.
         writefln("%0.2f->%f",f,(f*100f));
 results in:
 0.01->1.000000
Someting that is less than 0 is being "printed" as 1 at that precision. Printing with a higher precision should print something less than 0. Ali
Nov 04 2009
prev sibling parent Ali Cehreli <acehreli yahoo.com> writes:
Ok, I messed up my previous comment while moving rmcguire's lines around.
Trying again... Also, I am not sure about the last bit now. :)

rmcguire Wrote:

 why is this not a compiler bug?
 because:
 import std.stdio;
 
 void main() {
         float f=0.01;
Just an information: 0.01 is double, there is type conversion there.
         writefln("%0.2f->%d",f,cast(int)(f*100f));
 results in:
 0.01->0
f*100f is represented to be less than 0.
         writefln("%0.2f->%d",f,cast(int)(.01*100f));
 results in:
 0.01->1
.01 * 100f is double and apparently represented as something more than 1.
         writefln("%0.2f->%f",f,(f*100f));
 results in:
 0.01->1.000000
If it's like in C, floating point values must be passed as double arguments to functions. So there is float-to-double conversion before the function call, effectively representing the argument as double. Ali
Nov 04 2009
prev sibling parent reply Joel Christensen <joelcnz gmail.com> writes:
 To be safe, whenever converting to int, always add a small epsilon.  I 
 think you can use float.epsilon, but I don't have any experience with 
 whether that is always reasonable.
 
 -Steve
Thanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Nov 04 2009
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
Joel Christensen wrote:
 To be safe, whenever converting to int, always add a small epsilon.  I 
 think you can use float.epsilon, but I don't have any experience with 
 whether that is always reasonable.

 -Steve
Thanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
If the values you are working with are monetary, I don't think it makes sense to convert to int by casting, since casting always rounds towards zero. Instead you should use the std.math.lround() function, which rounds in a more natural way (towards the nearest integer, away from zero if the fractional part is exactly 0.5). To answer your question, this is how you would get and use the epsilon value in your example: cast(int)(f*100f + float.epsilon); -Lars
Nov 04 2009
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Nov 2009 19:38:45 -0500, Joel Christensen <joelcnz gmail.com>  
wrote:

 To be safe, whenever converting to int, always add a small epsilon.  I  
 think you can use float.epsilon, but I don't have any experience with  
 whether that is always reasonable.
  -Steve
Thanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
Might I suggest using integers instead of floating point. My experience with floating point is limited, but I've been burned enough to avoid using it wherever possible. To be fair, there are certainly many cases where floating point is useful, like in statistics, but when I can use integers, I do. Translating money into integers is as easy as using the lowest denomination as 1. For example, if you always store your money in cents, then $1 is simply stored as 100. If you want to add tax (say 5%), you do x * 5 / 100. If you want to do rounding, then it's a little trickier, but not too bad: (x * 5 + 50) / 100. The trick is to add half the value of the denominator before dividing to do exact rounding. I've written lots of code that uses decimal numbers without ever using floating point. It's also fun to experiment with fraction types, which are more exact when you are dealing with rational numbers. I once had a very comprehensive C++ fraction type that used a templated base type for the numerator/denominator. I even ported it to Java using BigInt as the numerator/denominator types. It could do very exact calculations with pretty useable speed. This was only in algorithmic coding competitions, I've never had a real world use for them (though I know they exist). There are types in other languages such as C#'s decimal that do a lot of the work for you. It would be a nice addition to D. Hope this helps. -Steve
Nov 05 2009
prev sibling parent Charles Hixson <charleshixsn earthlink.net> writes:
Joel Christensen wrote:
 To be safe, whenever converting to int, always add a small epsilon.  I
 think you can use float.epsilon, but I don't have any experience with
 whether that is always reasonable.

 -Steve
Thanks every one for the replies. I still have problems. How do I use epsilon? 'real' helps in my example program but not in my money program. I think there's a function out there that does dollars and cents (eg. .89 -> 89c and 1.00 -> $1), but I'm interested how to solve this problem.
If you're working with an accounting function, you should use bucket rounding. I don't think there's any standard routine for it, but basically first you round, and then you accumulate the error created by rounding in a separate variable. Then when you've processed the "column" or "batch" of numbers, you take the accumulated error and redistribute it over the numbers in a way that makes sense to you. (Perhaps proportional to the size of the number, or some such.) The exact procedure for redistributing the error is case specific, which is probably why there aren't any standard functions to handle this, but uniform distribution rarely works, because the goal is that the totals precisely equal the value. (Note that for this you don't want to use floating point values, but rather fixed point values...which can be emulated with integers that are converted to floats when it's time to export them.) Remember, accounting routines aren't allowed to lose or generate any extra pennies. If you want to say "That's a silly way to thing about numbers", I'll agree with you. But the accountants who wrote the rules back in the 1800's (or earlier?) didn't use computers. They used abacuses, and pen and ink. And preventing cheating in various forms was one of their primary goals.
Nov 05 2009
prev sibling parent Joel Christensen <joelcnz gmail.com> writes:
Seems random. I've got it working though, changed real to float. I seem 
to get different results with my little test program though.

Here's from my full program. (I think there's a function in phobos to do 
this any way):

char[] cashToString(dub money,char[] paddingForCents="".dup) {
   char[] amount;
   if ( money<1f ) {
     float cents=money*100f;
     amount=format("%s%d",paddingForCents,cast(int)cents);
   }
   else
     amount=format("%0.2f",money);
   return
     format("%s%s%s", money>=1f ? "$" : "", amount, money<1f ? "c" : "");
}
Nov 05 2009