[R] floating point question
Simon Wood
simon at stats.gla.ac.uk
Sat Feb 1 16:51:05 CET 2003
> Incidentally, this may well be a reason for the irritating habit of DLLs
> in setting the floating point precision to 53 bits (well, the irritating
> bit is not changing it back). This makes the results of computations
> more nearly independent of where the numbers end up being stored.
- I suspect that's right, I still find it a bit alarming that
a=b; if (a==b) printf("hooray") else printf("rats");
can occasionally print `rats' since `a' got stored in memory, `b' was left
in an fpu register and hence, when it comes to comparison, they are not
equal. Simon
>
> -thomas
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> http://www.stat.math.ethz.ch/mailman/listinfo/r-help
>
More information about the R-help
mailing list