Float rounded is inexact
By Nicolas Cellier - Posted on June 6th, 2009
Tagged: 3.1
| Project: | GNU Smalltalk |
| Component: | Base classes |
| Category: | bug |
| Priority: | minor |
| Assigned: | Unassigned |
| Status: | fixed |
Description
5000000000000001.0 asExactFraction -> 5000000000000001
This Float is thus represented exactly.
Unfortunately, trying to round it introduces some avoidable inexactness:
5000000000000001.0 rounded -> 5000000000000002
This is http://bugs.squeak.org/view.php?id=7134
I presume the patch can be ported to gst easily.
Updates
#1 submitted by Paolo Bonzini on Mon, 06/08/2009 - 08:05
| Status: | active | » fixed |
Indeed. I added a test to the testsuite too, using
p := (1 bitShift: self precision - 1) + (self coerce: 1.0). self assert: p rounded = p asExactFraction
I'll be committing the fix soonish.