LargePositiveInteger asFloat: does not round correctly

Tagged:
Project:GNU Smalltalk
Component:Base classes
Category:bug
Priority:normal
Assigned:bonzinip
Status:fixed
Description

Paolo, the simplification you made are great!
But you just swapped the lowBit test...

Example:

self assert: ((1 bitShift: 53+100) + (1 bitShift: 100) + 1) asFloatD truncated lowBit = 102.

Number of truncated bits is 101.
Carry is 1.
Truncated mantissa candidate is even (1 bitShift: 52).
But there are other trailing bits behind the carry (self lowBit < nTruncatedBits), so the truncated part > 0.5 ulp, so the mantissa must be rounded upper.

Please correct.
Cheers

Updates

#1 submitted by Paolo Bonzini on Mon, 06/08/2009 - 07:47
Assigned to:Unassigned» bonzinip

Also, #asFloatD/#asFloatE/#asFloatQ primitives should fail if they cannot do rounding.

#2 submitted by Paolo Bonzini on Mon, 06/08/2009 - 08:07
Status:active» fixed

I'll commit the patch this evening.

User login