[R] 0.1 + 0.2 != 0.3 revisited

[Peter Dalgaard]

Christian Hoffmann <christian.hoffmann at wsl.ch> writes:

> > 0.1 would be stored as (1 + 0.6)*2^(-4) and 0.2 would be stored as (1
> > + 0.6)*2^(-3), whereas 0.3 would be stored as (1 + 0.2)*2^(-2).  You
> > should expect 56 decimal (binary?) place accuracy on 0.1, 55 place
> > accuracy on 0.2, and 54 place accuracy on 0.3.  It's not surprising
> > weird things happen!
> 
> I don *not* think so: all mantissas here have *52 binary* places!

(53, since the leading 1 is not stored...)

Actually, in the x86 FPUs numbers get extended to 64 bits during
evaluation, so after alignment for addition, the smaller numbers may
have bits beyond the truncation point of the larger ones. These bits
disappear when the result is stored, but rounding may be affected
giving those unit-in-the-last-place differences.

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907