Fixed Point Rounding

[BebeSparkelSparkel]

A test for printing a floating point number of 12.345 with two decimal places checks for the result of "12.34". This matches with the result of showFFloat but @clyring is suggesting this is an error.

What should the result of formatDouble (standard places) be when it is the same distance to round up or down?

[clyring]

What should the result of formatDouble (standard places) be when it is the same distance to round up or down?

Rounding to even is the usual expected behavior when it really is the same distance to round up or down. But my point is that:

• There is no value of type Double which exactly represents 12.345.
• 12.345 :: Double therefore means the closest representable Double to the real number 12.345, which happens to be 0x1.8b0a3d70a3d71p3 and represents the rational number 6949617174986097 % 562949953421312.
• This rational number is greater than 12.345, by approximately 6.4e-16, and is thus slightly closer to 12.35 than it is to 12.34.

Both base and bytestring have the same bug, and first round 0x1.8b0a3d70a3d71p3 to the minimum number of decimal digits needed to disambiguate it from its neighboring Doubles and only then round that intermediate result to two decimal digits, when the user has asked 0x1.8b0a3d70a3d71p3 to be rounded directly to two decimal digits.

[BebeSparkelSparkel]

Seems like the choice is to have compatibility with base or be more correct.