[R] question about precision, floor, and powers of two.
Berton Gunter
gunter.berton at gene.com
Fri Nov 4 17:22:15 CET 2005
> In this particular case, it is slightly odd that we can't get an exact
> answer for operations that could in principle be carried out using
> integer arithmetic, but we're actually calculating log(8)/log(2).
>
> (Curiously, the same effect is not seen on Linux or Solaris until
>
> > log2(2^29)-29
> [1] 3.552714e-15
>
>
But it's a nice example of the risks of using tests for exact equality (via
==, say)with floating point arithmetic -- even when it **seems** that we
have integers. I'll bet a bunch of my code would flop because of this.
-- Bert Gunter
Genentech Non-Clinical Statistics
South San Francisco, CA
"The business of the statistician is to catalyze the scientific learning
process." - George E. P. Box
More information about the R-help
mailing list