LargePositiveInteger asFloat: does not round correctly
By Nicolas Cellier - Posted on June 6th, 2009
Tagged: 3.1
| Project: | GNU Smalltalk |
| Component: | Base classes |
| Category: | bug |
| Priority: | normal |
| Assigned: | bonzinip |
| Status: | fixed |
Description
Paolo, the simplification you made are great!
But you just swapped the lowBit test...
Example:
self assert: ((1 bitShift: 53+100) + (1 bitShift: 100) + 1) asFloatD truncated lowBit = 102.
Number of truncated bits is 101.
Carry is 1.
Truncated mantissa candidate is even (1 bitShift: 52).
But there are other trailing bits behind the carry (self lowBit < nTruncatedBits), so the truncated part > 0.5 ulp, so the mantissa must be rounded upper.
Please correct.
Cheers
Updates
#1 submitted by Paolo Bonzini on Mon, 06/08/2009 - 07:47
| Assigned to: | Unassigned | » bonzinip |
Also, #asFloatD/#asFloatE/#asFloatQ primitives should fail if they cannot do rounding.
#2 submitted by Paolo Bonzini on Mon, 06/08/2009 - 08:07
| Status: | active | » fixed |
I'll commit the patch this evening.
