www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Weird floating point rounding - Bug or how to control it correctly

reply An Pham <home home.com> writes:
import std.stdio;

     void main()
     {
         float f = 6394763.345f;
     	
         import std.format : sformat;

         char[80] vBuffer = void;
         writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", f));
     	
     }

Output
6394763.345 = 6394763.5000
Sep 13 2023
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, September 13, 2023 9:23:48 PM MDT An Pham via Digitalmars-d-
learn wrote:
 import std.stdio;

      void main()
      {
          float f = 6394763.345f;

          import std.format : sformat;

          char[80] vBuffer = void;
          writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", f));

      }

 Output
 6394763.345 = 6394763.5000
The nature of floating point numbers is such that there a bunch of values that they can't actually represent, and they get rounded pretty easily depending on the exact numbers involved. So, in general, you shouldn't expect floating point numbers to be exact. You will generally do better with increased precision though, and in this case, it looks like your number will stay the same if you use double or real instead of float. I would suggest that you watch this video from dconf 2016 which discusses floating point values: https://www.youtube.com/watch?v=YEUAUnamQiA - Jonathan M Davis
Sep 13 2023
prev sibling parent Basile B. <b2.temp gmx.com> writes:
On Thursday, 14 September 2023 at 03:23:48 UTC, An Pham wrote:
 import std.stdio;

     void main()
     {
         float f = 6394763.345f;
     	
         import std.format : sformat;

         char[80] vBuffer = void;
         writeln("6394763.345 = ", sformat(vBuffer[], "%.4f", 
 f));
     	
     }

 Output
 6394763.345 = 6394763.5000
Classic question. The float literal `6394763.345f` is not representable as IEEE-754 floating point number. Try https://www.h-schmidt.net/FloatConverter/IEEE754.html for a short introduction to the issue.
Sep 13 2023