[Rd] NaN and linear algebra

[Simon Urbanek]

Bill,

On Mar 21, 2005, at 10:31 PM, Bill Northcott wrote:

> As I see it, the MacOS X behaviour is not IEEE-754 compliant.
>
> I had a quick look at the IEEE web site and it seems quite clear  
> that NaNs should not cause errors, but propagate through  
> calculations to be tested at some appropriate (not too frequent)  
> point.

This is not quite correct and in fact irrelevant to the problem you  
describe. NaNs may or may not signal, depending on how they are used.  
Certain operations on NaN must signal by the IEEE-754 standard. The  
error you get is not a trap, and it's not a result of a signal check,  
either. The whole problem is that depending on which algorithm is  
used, the NaNs will be used different ways and thus may or may not  
use signaling operations.

I don't consider the `solve' error a bug - in fact I would rather get  
an error telling me that something is wrong (although I agree that  
the error is misleading - the error given in Linux is a bit more  
helpful) than getting a wrong result.

If I would mark something in your example as a bug that would be det 
(m)=0, because it should return NaN (remember, NaN==NaN is FALSE;  
furthermore if det was calculated inefficiently using Laplace  
expansion, the result would be NaN according to IEEE rules). det=0 is  
consistent with the error given, though. Should we check this in R  
before calling Lapack - if the vector contains NaNs, det/determinant  
should return NaN right away?

Many functions in R will actually bark at NaN inputs (e.g. qr,  
eigen, ...) - maybe you're saying that we should check for NaNs in  
solve before proceeding and raising an error?


Cheers,
Simon