[R] Validation of R

[Liaw, Andy]

Perception, perception, perception...

This reminds me of what I heard from Frank Harrell: "The different between S
and SAS is five years".  S has been around much longer than Insightful.
Doesn't that count?

R also comes with its own set suite ("make check", "make fullcheck", etc.).
Doesn't that count?

I guess one can try to validate NLMIXED by comparing the output with that of
nlme()...

Andy

> -----Original Message-----
> From: Paul, David A [mailto:paulda at BATTELLE.ORG]
> Sent: Thursday, April 17, 2003 4:18 PM
> To: r-help at stat.math.ethz.ch
> Subject: RE: [R] Validation of R
> 
> 
> At Battelle, the QA/QC folks have the philosophy that the
> FDA will likely hold us responsible for whatever internal
> standards we set for ourselves, assuming that such standards
> are "reasonable".
> 
> For software, our internal standards basically say that
> 
> (1) COTS (Com'l Off The Shelf software) developed by a
> company having both a long history of selling high-quality
> products and good QA doesn't need extensive from-scratch
> validation, only validation of simpler routines like the
> computation of means, variances, linear regression models,
> &etc.  (After all, how would anyone really validate what, 
> say, PROC NLMIXED yields in a complex growth-curve application?)
> 
> (2) Anything free needs to be extensively validated by 
> comparing it with something that fits into (1)
> 
> This leaves R completely out of our GLP studies, and favors
> SAS since Insightful hasn't been around as long as the SAS 
> Institute.  Like it or not, the perception is that using SAS
> won't get you into trouble with the FDA or other regulatory
> agencies.
> 
> 
> -David Paul
> 
> 
> 
> -----Original Message-----
> From: partha_bagchi at hgsi.com [mailto:partha_bagchi at hgsi.com] 
> Sent: Thursday, April 17, 2003 3:32 PM
> To: Frank E Harrell Jr
> Cc: k.benjamin at retroscreen.com; r-help at stat.math.ethz.ch;
> a.mann at retroscreen.com; s.balasingam at retroscreen.com;
> r.lambkin at retroscreen.com; v_bill_pikounis at merck.com;
> s.bossuyt at retroscreen.com
> Subject: Re: [R] Validation of R
> 
> 
> I agree with your points and if you notice I share your philosophical 
> view. I was commenting more on what you call "mind" share. It 
> is still 
> real. 
> 
> However, also a minor point  - there is mention in the regs 
> regarding COTS 
> software (which I believe stands for Commercial of the Shelf 
> software) ..
> 
> 
> 
> 
> 
> 
> Frank E Harrell Jr <fharrell at virginia.edu>
> 04/17/2003 12:10 PM
> 
>  
>         To:     partha_bagchi at hgsi.com
>         cc:     v_bill_pikounis at merck.com, 
> k.benjamin at retroscreen.com, 
> r-help at stat.math.ethz.ch, a.mann at retroscreen.com, 
> s.balasingam at retroscreen.com, r.lambkin at retroscreen.com, 
> s.bossuyt at retroscreen.com
>         Subject:        Re: [R] Validation of R
> 
> 
> On Thu, 17 Apr 2003 10:38:06 -0400
> partha_bagchi at hgsi.com wrote:
> 
> > However, the perception out there is the "SAS is the accepted 
> > software" especially for regulatory submission and 
> especially in the 
> > US. Thus, I think validation usually means "Yeah, but did 
> you use SAS 
> > to get the answer" , no matter how irrelevant the question 
> is. For a 
> > non-statistician, or a person doing validation certain 
> software do not 
> > need validation (Microsoft Word, SAS etc.) certain other , perhaps 
> > more
> so
> > for open source, validation is essential.
> 
> SAS is NOT the accepted software for FDA, because FDA does 
> not accept ANY 
> brand of software.  This is really a "mind share" issue at pharma 
> companies.  SAS is not validated in every sense; there is a 
> huge list of 
> current SAS bugs.
> 
> Validation is best done on a per-project basis as you can't 
> anticipate all 
> aspects of a particular dataset.  The validation can be done by 
> independent calculations of pivotal findings.  For R there is an 
> especially good opportunity because if you are using the base 
> packages you 
> can run essentially the same code in S-Plus to get an independent 
> validation of the underlying calculations (but not of your S 
> code).  The 
> base code in R is independent of that in S-Plus (this is not 
> true of most 
> add-on packages by users).  There is no other "SAS" you can run.
> 
> ---
> Frank E Harrell Jr              Prof. of Biostatistics & Statistics
> Div. of Biostatistics & Epidem. Dept. of Health Evaluation Sciences U.
> Virginia School of Medicine  http://hesweb1.med.virginia.edu/biostat
> 
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
> 
> 
> 
> 	[[alternate HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>