www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - / operator for float values

reply "Thomas Kuehne" <eisvogel users.sourceforge.net> writes:
code:
# float f = 1.2e-3;
# float a = 12.0;
# float b = 10000.0;
# float c = a/b;
# assert(f == c);
# assert(f == a/b);

The first assertion passes but the second fails.

test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d
Oct 13 2004
parent reply Burton Radons <burton-radons smocky.com> writes:
Thomas Kuehne wrote:

 code:
 # float f = 1.2e-3;
 # float a = 12.0;
 # float b = 10000.0;
 # float c = a/b;
 # assert(f == c);
 # assert(f == a/b);
 
 The first assertion passes but the second fails.
 
 test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d

This is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion. The real bug is that this doesn't work properly: assert(f == cast(float) (a/b)); This should cause the value to be truncated, but it doesn't. You can cause your code to succeed if you eliminate the rounding error. Divide by a power of 2 instead: float f = 0.01171875; float a = 12.0; float b = 1024.0; This will then work. That could probably be considered a correct test of the language, as D depends on IEEE being present.
Oct 13 2004
next sibling parent "Thomas Kuehne" <eisvogel users.sourceforge.net> writes:
Burton Radons schrieb:
 code:
 # float f = 1.2e-3;
 # float a = 12.0;
 # float b = 10000.0;
 # float c = a/b;
 # assert(f == c);
 # assert(f == a/b);

 The first assertion passes but the second fails.

 test case: svn://svn.kuehne.cn/dstress/run/float_literal_dec_02.d

This is because the first comparison uses float while the second uses real; the float truncates some rounding error in the division that is preserved for the comparison in the second assert. This is a natural effect of type promotion. The real bug is that this doesn't work properly: assert(f == cast(float) (a/b)); This should cause the value to be truncated, but it doesn't.

Thanks for the fix - guess I have to read a little about IEEE :) Thomas
Oct 13 2004
prev sibling parent "Lynn Allan" <l_d_allan adelphia.net> writes:
Thanks for the time and effort the two of you are putting into making
'D' an even better tool.

"Burton Radons" <burton-radons smocky.com> wrote in message
news:ckj8ps$2te8$1 digitaldaemon.com...
 Thomas Kuehne wrote:

 code:
 # float f = 1.2e-3;


Oct 13 2004