# [R] rounding down with as.integer

Duncan Murdoch murdoch.duncan at gmail.com
Fri Jan 2 10:52:50 CET 2015

```On 01/01/2015 10:05 PM, Mike Miller wrote:
> On Thu, 1 Jan 2015, Duncan Murdoch wrote:
>
>> On 01/01/2015 1:21 PM, Mike Miller wrote:
>>
>>> I understand that it's all about the problem of representing digital
>>> numbers in binary, but I still find some of the results a little
>>> surprising, like that list of numbers from the table() output.  For
>>> another example:
>>>
>>>> 1000+3 - 1000*(1+3/1000)
>>> [1] 1.136868e-13
>>>
>>>> 3 - 1000*(0+3/1000)
>>> [1] 0
>>>
>>>> 2000+3 - 1000*(2+3/1000)
>>> [1] 0
>>>
>>> See what I mean?  So there is something special about the numbers
>>> around 1000.
>>
>> I think it's really that there is something special about the numbers
>> near 1, and you're multiplying that by 1000.
>>
>> Numbers from 1 to just below 2 are stored as their fractional part, with
>> 52 bit precision.  Some intermediate calculations will store them with
>> 64 bit precision.  52 bits gives about 15 or 16 decimal places.
>
>
> This is how big those errors are:
>
>> 512*.Machine\$double.eps
> [1] 1.136868e-13
>
> Under other conditions you also were seeing errors of twice that, or
> 1024*.Machine\$double.eps.  It might not be a coincidence that the largest
> number giving me an error was 1023.
>
>> 2^-43
> [1] 1.136868e-13
>
>> .Machine\$double.eps
> [1] 2.220446e-16
>
>> 2^-52
> [1] 2.220446e-16
>
> I guess the 52 comes from the IEEE floating point spec...
>
> http://en.wikipedia.org/wiki/Double-precision_floating-point_format#IEEE_754_double-precision_binary_floating-point_format:_binary64
>
> ...but why are we seeing errors so much bigger than the machine precision?

You are multiplying by 1000.  That magnifies the error.

> Why does it change at 2?

Because (most) floating point numbers are stored as (1 + x) * 2^y, where
x is a number between 0 and 1, and y is an integer value between -1023
and 1023. The value of y changes at 2, and this means errors in x become
twice as big.  (The exceptions are 0, Inf, NaN, etc., as well as
"denormals", where y is -1024 and the format changes to x * 2^(y+1).)

Duncan Murdoch

> It doesn't really matter to my work, but it is a curious thing, so I would
> be interested to learn about it.
>
> Mike
>

```