www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Double value is rounded to unexpected value: 2.5 -> 2 instead of 3

reply alex_ca <heirloomalexis gmail.com> writes:
Hi,

I'm having trouble understanding why in some cases a double value 
will be rounded up and other times down, for the same code. 
Here's a snippet with code I tried to debug:

   int getNumberOfStitchesForRowLength(double rowLength)
   {
     writeln("input ", rowLength, " ", currentGauge.stitch_gauge, 
" ", currentGauge.gauge_length);

     writeln("stitches: ", (rowLength * currentGauge.stitch_gauge) 
/ currentGauge.gauge_length,  " -> " , ((rowLength * 
currentGauge.stitch_gauge) / 
currentGauge.gauge_length).roundTo!int);

     double end = 2.5;
     double start = 0;
     writeln("I expect: ", ((abs(end-start)*10)/10).roundTo!int);

     return ((rowLength * currentGauge.stitch_gauge) / 
currentGauge.gauge_length).roundTo!int;

   }

And here's some output from that:
input 2.5 10 10
stitches: 2.5 -> 2
I expect: 3

OR, similarly, I get

input 3.5 10 10
stitches: 3.5 -> 3
I expect: 4

However, it works as I would expect for one value:

input 1.5 10 10
stitches: 1.5 -> 2
I expect: 2

I would appreciate some ideas for why I see this seeming 
inconsistency. Or if someone can share how I can further debug 
this.

Thanks!

I'm using DMD32 D Compiler v2.074.1 on Windows
Jul 07
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 07/07/2017 11:29 AM, alex_ca wrote:

 input 2.5 10 10
 stitches: 2.5 -> 2
 I expect: 3
That's because what is printed as 2.5 is actually a little less than that. (Try printing with a format string like "%.20f".) The common way of dealing with this issue is to add 0.5 before the conversion: import std.stdio; import std.conv; void main() { for (double d = 1; d < 2; d += 0.1) { writefln("%s (%.20f): %s", d, d, (d + 0.5).to!int); } } 1 (1.00000000000000000000): 1 1.1 (1.10000000000000008882): 1 1.2 (1.20000000000000017764): 1 1.3 (1.30000000000000026645): 1 1.4 (1.40000000000000035527): 1 1.5 (1.50000000000000044409): 2 1.6 (1.60000000000000053291): 2 1.7 (1.70000000000000062172): 2 1.8 (1.80000000000000071054): 2 1.9 (1.90000000000000079936): 2 Ali
Jul 07
parent alex_ca <heirloomalexis gmail.com> writes:
Thanks, that was what was happening.
Jul 08
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 07/07/2017 08:29 PM, alex_ca wrote:
 I'm having trouble understanding why in some cases a double value will 
 be rounded up and other times down, for the same code. Here's a snippet 
 with code I tried to debug:
 
    int getNumberOfStitchesForRowLength(double rowLength)
    {
      writeln("input ", rowLength, " ", currentGauge.stitch_gauge, " ", 
 currentGauge.gauge_length);
 
      writeln("stitches: ", (rowLength * currentGauge.stitch_gauge) / 
 currentGauge.gauge_length,  " -> " , ((rowLength * 
 currentGauge.stitch_gauge) / currentGauge.gauge_length).roundTo!int);
 
      double end = 2.5;
      double start = 0;
      writeln("I expect: ", ((abs(end-start)*10)/10).roundTo!int);
 
      return ((rowLength * currentGauge.stitch_gauge) / 
 currentGauge.gauge_length).roundTo!int;
 
    }
 
 And here's some output from that:
 input 2.5 10 10
 stitches: 2.5 -> 2
 I expect: 3
 
 OR, similarly, I get
 
 input 3.5 10 10
 stitches: 3.5 -> 3
 I expect: 4
 
 However, it works as I would expect for one value:
 
 input 1.5 10 10
 stitches: 1.5 -> 2
 I expect: 2
 
 I would appreciate some ideas for why I see this seeming inconsistency. 
 Or if someone can share how I can further debug this.
Works for me when I plug in the exact values: ---- import std.stdio; import std.math: abs; import std.conv: roundTo; import std.math; struct Gauge { double stitch_gauge; double gauge_length; } Gauge currentGauge = Gauge(10, 10); int getNumberOfStitchesForRowLength(double rowLength) { writeln("input ", rowLength, " ", currentGauge.stitch_gauge, " ", currentGauge.gauge_length); writeln("stitches: ", (rowLength * currentGauge.stitch_gauge) / currentGauge.gauge_length, " -> " , ((rowLength * currentGauge.stitch_gauge) / currentGauge.gauge_length).roundTo!int); return ((rowLength * currentGauge.stitch_gauge) / currentGauge.gauge_length).roundTo!int; } void main() { foreach (x; [1.5, 2.5, 3.5]) { getNumberOfStitchesForRowLength(x); } } ---- Prints: ---- input 1.5 10 10 stitches: 1.5 -> 2 input 2.5 10 10 stitches: 2.5 -> 3 input 3.5 10 10 stitches: 3.5 -> 4 ---- As expected, right? Could it be that the inputs aren't exactly the numbers you think they are? For example, when you change one the 10s to 9.9999999, it still prints as "10", but the results are different.
Jul 07