[R] The Quality & Accuracy of R

[Muenchen, Robert A (Bob)]

That's a great idea. I know of no commercial vendors who provide such
detailed info.

Bob

-----Original Message-----
From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com] 
Sent: Monday, January 26, 2009 7:52 PM
To: Muenchen, Robert A (Bob)
Cc: R-help at r-project.org
Subject: Re: [R] The Quality & Accuracy of R

It would be possible to develop tools to develop code coverage
statistics quantifying the percent of the code that the tests
exercise.

On Fri, Jan 23, 2009 at 10:04 AM, Muenchen, Robert A (Bob)
<muenchen at utk.edu> wrote:
> Hi All,
>
>
>
> We have all had to face skeptical colleagues asking if software made
by
> volunteers could match the quality and accuracy of commercially
written
> software. Thanks to the prompting of a recent R-help thread, I read,
"R:
> Regulatory Compliance and Validation Issues, A Guidance Document for
the
> Use of R in Regulated Clinical Trial Environments
> (http://www.r-project.org/doc/R-FDA.pdf). This is an important
document,
> of interest to the general R community. The question of R's accuracy
is
> such a frequent one, it would be beneficial to increase the visibility
> of the non-clinical  information it contains. A document aimed at a
> general audience, entitled something like, "R: Controlling Quality and
> Assuring Accuracy" could be compiled from the these sections:
>
>
>
> 1.      What is R? (section 4)
>
> 2.      The R Foundation for Statistical Computing (section  3)
>
> 3.      The Scope of this Guidance Document (section 2)
>
> 4.      Software Development Life Cycle (section 6)
>
>
>
> Marc Schwartz, Frank Harrell, Anthony Rossini, Ian Francis and others
> did such a great job that very few words would need to change. The
only
> addition I suggest is to mention how well R did in, Keeling & Parvur's
> "A comparative study of the reliability to nine statistical software
> packages, May 1, 2007 Computational Statistics & Data Analysis,
Vol.51,
> pp 3811-3831.
>
>
>
> Given the importance of this issue, I would like to see such a
document
> added to the PDF manuals in R's Help.
>
>
>
> The document mentions (Sect. 6.3) that a set of validation tests, data
> and known results are available. It would be useful to have an option
to
> run that test suite in every R installation, providing clear progress,
> "Validating accuracy of t-tests...Validating accuracy of linear
> regression...." Whether or not people chose to run the tests, they
would
> at least know that such tests are available. Back in my mainframe
> installation days, this step was part of many software installations
and
> it certainly gave the impression that those were the companies that
took
> accuracy seriously. Of course the other companies probably just ran
> their validation suite before shipping, but seeing it happen had a
> tremendous impact.  I don't know how much this would add to download,
> but if it was too much, perhaps it could be implemented as a separate
> download.
>
>
>
> I hope these suggestions can help mitigate the concerns so many non-R
> users have.
>
>
>
> Cheers,
>
> Bob
>
>
>
> =========================================================
>
> Bob Muenchen (pronounced Min'-chen),
>
> Manager, Research Computing Support
>
> U of TN Office of Information Technology
>
> Stokely Management Center, Suite 200
>
> 916 Volunteer Blvd., Knoxville, TN 37996-0520
>
> Voice: (865) 974-5230
>
> FAX: (865) 974-4810
>
> Email: muenchen at utk.edu
>
> Web: http://oit.utk.edu/research <http://oit.utk.edu/scc>
>
> Map to Office: http://www.utk.edu/maps
>
> Newsletter: http://listserv.utk.edu/archives/rcnews.html
> <http://listserv.utk.edu/archives/statnews.html>
>
> =========================================================
>
>
>
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>