[R-SIG-Mac] NA handling in 2.7.0 versus 2.6.2 for SD

[Prof Brian Ripley]

On Tue, 22 Apr 2008, William Revelle wrote:

> Hi,
> When comparing how R2.7.0 handles missing values  I discovered that
> it differs from 2.6.2 (at least for the sd function).   In
> particular, if a column of a matrix is all NA, sd in 2.7.0 throws an
> error while in 2.6.2 it just returns NA for that column.  Is this
> intentional?

I presume you mean sd(na.rm=TRUE): the default behaviour is unchanged.

This is intentional and documented in the NEWS file as a BUG FIX.

     o   co[rv](use = "complete.obs") now always gives an error if there
         are no complete cases: they used to give NA if
         method = "pearson" but an error for the other two methods.
         (Note that this is pretty arbitrary, but zero-length vectors
         always give an error so it is at least consistent.)

sd() calls var = cov.  Why would anyone expect

> sd(numeric(0))
Error in var(x, na.rm = na.rm) : 'x' is empty
> sd(NA_real_, na.rm=TRUE)
[1] NA
> sd(na.omit(NA_real_))
Error in var(as.vector(x), na.rm = na.rm) : 'x' is empty

(from 2.6.2)?

>
>>  x <- matrix(1:50,ncol=5)
>>  sd(x)
> [1] 3.027650 3.027650 3.027650 3.027650 3.027650
>>   x[,1] <- NA
>>  sd(x,na.rm=TRUE)
> Error in var(x, na.rm = na.rm) : no complete element pairs
>
>
>
> R version 2.7.0 RC (2008-04-21 r45421)
> i386-apple-darwin8.10.1
>
> locale:
> en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
>
> compare this to version 2.6.2
>>  x <- matrix(1:50,ncol=5)
>>  sd(x)
> [1] 3.027650 3.027650 3.027650 3.027650 3.027650
>
>>   x[,1] <- NA
>>  sd(x,na.rm=TRUE)
> [1]       NA 3.027650 3.027650 3.027650 3.027650
>
>>  sessionInfo()
> R version 2.6.2 (2008-02-08)
> i386-apple-darwin8.10.1
>
> locale:
> en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>
> Thanks for all the great work.  Sorry not to have detected this sooner.
>
> Bill
>
>
> -- 
> William Revelle		http://personality-project.org/revelle.html
> Professor			http://personality-project.org/personality.html
> Department of Psychology             http://www.wcas.northwestern.edu/psych/
> Northwestern University	http://www.northwestern.edu/
> Use R for statistics:                          http://personality-project.org/r
>
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595