[Rd] significant digits (PR#9682)

[Simon Urbanek]

On Jun 3, 2008, at 5:12 PM, Duncan Murdoch wrote:

> On 6/3/2008 4:36 PM, Simon Urbanek wrote:
>> On Jun 3, 2008, at 2:48 PM, Duncan Murdoch wrote:
>>> On 6/3/2008 11:43 AM, Patrick Carr wrote:
>>>> On 6/3/08, Duncan Murdoch <murdoch at stats.uwo.ca> wrote:
>>>>>
>>>>> because signif(0.90, digits=2) == 0.9.  Those two objects are   
>>>>> identical.
>>>> My text above that is poorly worded. They're identical internally,
>>>> yes. But in terms of the number of significant digits, 0.9 and 0.90
>>>> are different. And that matters when the number is printed, say  
>>>> as an
>>>> annotation on a graph. Passing it through sprintf() or format()  
>>>> later
>>>> requires you to specify the number of digits after the decimal,  
>>>> which
>>>> is different than the number of significant digits, and requires  
>>>> case
>>>> testing for numbers of different orders of magnitude.
>>>> The original complainant (and I) expected this behavior from   
>>>> signif(),
>>>> not merely rounding. As I said before, I wrote my own workaround so
>>>> this is somewhat academic, but I don't think we're alone.
>>>>> As far as I know, rounding is fine in Windows:
>>>>>
>>>>> > round(1:10 + 0.5)
>>>>> [1]  2  2  4  4  6  6  8  8 10 10
>>>>>
>>>> It might not be the rounding, then. (windows xp sp3)
>>>>  > signif(12345,digits=4)
>>>>  [1] 12340
>>>>  > signif(0.12345,digits=4)
>>>>  [1] 0.1235
>>>
>>> It's easy to make mistakes in this, but a little outside-of-R   
>>> experimentation suggests those are the right answers.  The number   
>>> 12345 is exactly representable, so it is exactly half-way between   
>>> 12340 and 12350, so 12340 is the right answer by the unbiased  
>>> round- to-even rule.  The number 0.12345 is not exactly  
>>> representable, but  (I think) it is represented by something  
>>> slightly closer to 0.1235  than to 0.1234.  So it looks as though  
>>> Windows gets it right.
>>>
>>>
>>>> OS X (10.5.2/intel) does not have that problem.
>>>
>>> Which would seem to imply OS X gets it wrong.
>> This has nothing to do with OS X, you get that same answer on  
>> pretty  much all other platforms (Intel/Linux, MIPS/IRIX, Sparc/ 
>> Sun, ...).  Windows is the only one delivering the incorrect result  
>> here.
>>> Both are supposed to be using the 64 bit floating point standard,   
>>> so they should both give the same answer:
>> Should, yes, but Windows doesn't. In fact 10000.0 is exactly   
>> representable and so is 1234.5 which is the correct result that  
>> all  except Windows get.
>
> I think you skipped a step.

I didn't - I was just pointing out that what you are trying to show is  
irrelevant. We are dealing with FP arithmetics here, so although your  
reasoning is valid algebraically, it's not in FP world. You missed the  
fact that FP operations are used to actually get the result (*10000.0,  
round and divide again) and thus those operation will influence it as  
well.


>  The correct answer is either 0.1234 or 0.1235, not something 10000  
> times bigger.  The first important question is whether 0.12345 is  
> exactly representable, and the answer is no.  The second question is  
> whether it is represented by a number bigger or smaller than the  
> real number 0.12345.  If it is bigger, the answer should be 0.1235,  
> and if it is smaller, the answer is 0.1234.

No. That was what I was trying to point out. You can see clearly from  
my post that 0.12345 is not exactly representable and that the  
representation is slightly bigger. This is, however, irrelevant,  
because the next step is to multiply that number by 10000 (see fprec  
source) and this is where your reasoning breaks down - the result is  
exact representation of 1234.5, because the imprecision gets lost in  
the operation on all platforms but Windows. The result is that Windows  
is inconsistent with others, whether that is a bug or feature I don't  
care. All I really wanted to say is that this has nothing to do with  
OS X - if anything then it's a Windows issue.


>  My experiments suggest it is bigger.

I was not claiming otherwise.


>  Yours don't look relevant.

Vice versa as it turns out.

Cheers,
Simon


> It certainly isn't exactly equal to 1234.5/10000, because that  
> number is not representable.  It's equal to x/2^y, for some x and y,  
> and it's a pain to figure out exactly what they are.
>
> However, I am pretty sure R is representing it (at least on Windows)  
> as the binary expansion
>
> 0.000111111001101001101011010100001011000011110010011111
>
> while the true binary expansion (using exact rational arithmetic)  
> starts out
>
> 0.00011111100110100110101101010000101100001111001001111011101...
>
> If you line those up, you'll see that the first number is bigger  
> than the second.  (Ugly code to derive these is down below.)
>
> Clearly the top representation is the correct one to that number of  
> binary digits, so I think Windows got it right, and all those other  
> systems didn't.  This is probably because R on Windows is using  
> extended precision (64 bit mantissas) for intermediate results, and  
> those other systems stick with 53 bit mantissas.
>

However, this means that Windows doesn't conform

> Duncan Murdoch
>
> # Convert number to binary expansion; add the decimal point manually
>
> x <- 0.12345
> while (x != 0) {
>  cat(trunc(x))
>  x <- x - trunc(x)
>  x <- x * 2
> }
>
> # Do the same thing in exact rational arithmetic
>
> num <- 12345
> denom <- 100000
> for (i in 1:60) {
>  cat(ifelse(num > 100000, "1", "0"))
>  num <- num %% 100000
>  num <- 2*num
> }
>
> # Manually cut and paste the results to get these:
>
> "0.000111111001101001101011010100001011000011110010011111"
> "0.00011111100110100110101101010000101100001111001001111011101"
>
>