[R] Precision in R

[Duncan Murdoch]

On 1/15/2009 1:32 PM, David Winsemius wrote:
> On Jan 15, 2009, at 12:25 PM, Charles C. Berry wrote:
>>
>>
>> This is what I get on windows XP:
>>
>>> tcp1-tcp2
>>              [,1]          [,2]          [,3]
>> [1,] -2.861023e-06 -4.768372e-07 -4.768372e-07
>> [2,] -4.768372e-07 -3.814697e-06  2.622604e-06
>> [3,] -4.768372e-07  2.622604e-06 -5.960464e-08
>>>
>>
>> but on my Gentoo Linux Intel Core 2 Duo:
>>
>> print(tcp1-tcp2,digits=20)
>>     [,1] [,2] [,3]
>> [1,]    0    0    0
>> [2,]    0    0    0
>> [3,]    0    0    0
>>>
>>
>> So, it would seem to be that the floating point calcs on my Windows
>> box, David's Mac, and whatever system you are using are not as
>> accurate as they might be.
> 
> For the record; on an Intel Mac (Leopard) with Urbanek's compilation  
> of R 2.8.1
> 
>  >  print(tcp1-tcp2,digits=20)
>       [,1] [,2] [,3]
> [1,]    0    0    0
> [2,]    0    0    0
> [3,]    0    0    0
> 

I haven't debugged these, but I would guess this is because we have the 
Windows libraries set to use extended precision (64 bit mantissas) on 
intermediate results, whereas once things get stored to RAM, they are 
rounded to double precision (53 bit mantissas).  This has the benefit of 
giving a more accurate answer in many circumstances, but the 
disadvantage that the final results are more dependent on the order of 
calculations.  I think the other platforms never do the full extended 
precision calculations, so their results are consistent (but probably 
less accurate sometimes).

Duncan Murdoch