[Rd] FW: [R] The Quality & Accuracy of R

[David M Smith]

On Sun, Jan 25, 2009 at 4:20 PM, Peter Dalgaard
<p.dalgaard at biostat.ku.dk> wrote:
>>> - a good reason to want post-install validation is that validity can depend on other part of the system outside developer control (e.g. an overzealous BLAS optimization, sacrificing accuracy and/or standards compliance for speed, can cause trouble). This is also a reason for not making too far-reaching statements about validity.

I wanted to echo Peter's point here.  It's the main reason why we
don't claim our distribution of R is validated: *no* software can be
considered validated outside of the environment where it is installed
and used.  (We do however claim Revolution R is ready for a validation
*process*, a small but significant part of which is coming on-site to
run tests and verify the results.) We've come across a number of
environmental issues (locales, random number generators, shared
libraries, path settings, many others) that may affect the validation
process.  My main point here is that R can only be validated in situ,
and the process isn't practical to automate.  With the right build
tools in place, many of the *tests* can be automated, but that leaves
out validation on how the results are stored, used, and accessed in
practice.

> Muenchen, Robert A (Bob) wrote:
>>> Asking to add a superfluous step to an installation may seem like a
> waste of time, and technically it is. But psychologically this testing
> will have a important impact that will silence many critics.

Nonetheless, Bob has an excellent point here -- even short of a
complete validation process, *perception* can prevent the validation
ball from getting stuck in the first place.  Giving the user some
degree of easily-digestible feedback that the installed R has run and
passed a battery of tests could help for that, and is something we'll
look at for the Revolution R distribution.

# David Smith

P.S. For those who subscribe to r-devel but not r-help, some further
discussion of validation for R is here:
http://blog.revolution-computing.com/2009/01/analyzing-clinical-trial-data-with-r.html

--
David M Smith <david at revolution-computing.com>
Director of Community, REvolution Computing www.revolution-computing.com
Tel: +1 (206) 577-4778 x3203 (Seattle, USA)