[R] "R is not a validated software package.."

[Marc Schwartz]

On Fri, 2007-06-08 at 16:02 +0200, Giovanni Parrinello wrote:
> Dear All,
> discussing with a statistician of a pharmaceutical company I received 
> this answer about the statistical package that I have planned to use:
> 
> As R is not a validated software package, we would like to ask if it 
> would rather be possible for you to use SAS, SPSS or another approved 
> statistical software system.
> 
> Could someone suggest me a 'polite' answer?
> TIA
> Giovanni
> 

The polite answer is that there is no such thing as 'FDA approved'
software for conducting clinical trials. The FDA does not approve,
validate or otherwise endorse software.

If the pharma company in question has developed their own list of
acceptable software applications that you must comply with, that is
different, but is independent of any FDA requirements.

As the saying used to be several decades ago, "Nobody ever got fired for
buying IBM".  In the clinical trials realm today, the same could be said
for SAS or Oracle Clinical. 

That is a political, and perhaps a corporate legal counsel driven "risk
aversion" based issue, not a scientific one.  It is also a human
behavioral issue, as Bert noted, relative to fighting inertia, training
or re-training issues and the pre-existing investment in internal
processes and infrastructure.  This will change over time as more
statisticians, who have been trained in the use of R during their
academic years, enter into industry positions.

As others have noted, there is a PERCEPTION that somehow SAS is endorsed
by the FDA or that it constitutes a 'gold standard' of sorts. This is a
perception and not reality.

That being said:

There are a variety of relevant Guidance and Guideline documents that
the FDA has put forth to address these issues. Most recently, the FDA
approved final guidance for the use of computerized systems in clinical
investigations (May 2007):

http://www.fda.gov/OHRMS/DOCKETS/98fr/04d-0440-gdl0002.pdf

In addition, there is a General Principles of Software Validation
document:

http://www.fda.gov/cdrh/comp/guidance/938.html

The majority of the 21 CFR Part 11 requirements (audit trails,
electronic signatures, etc.) are relevant to systems that manage "source
medical records". These would typically be database applications and
medical devices, not statistical applications. In our shop for example,
our Oracle 10g server has been implemented in accordance with these
requirements.

There is a 21 CFR Part 11 guidance document here:

http://www.fda.gov/ohrms/dockets/98fr/5667fnl.pdf

There are also all of the so-called FDA and ICH GxP (Good x Practice)
documents:

   http://www.fda.gov/oc/gcp/guidance.html
   http://www.ich.org/cache/compo/475-272-1.html

that provide a framework for operations in a regulated environment and
for relevant statistical practice guidance. The 'x' above is replaced by
words such as "Clinical", "Manufacturing", "Laboratory", etc.

There is even a draft guidance document on the use of Bayesian
techniques for medical device trials:

http://www.fda.gov/cdrh/osb/guidance/1601.html


Some of the references in other posts have to do with software embedded
in medical devices, which could be anything such as bedside ECG
monitoring stations, diagnostic imaging systems, radiation therapy
instrumentation and pacemakers. These are generally not relevant to this
discussion.

The bottom line, is that while there is a burden on the part of the
'software publisher' to utilize and document reasonable manufacturing,
version control, software maintenance and quality processes, the
overwhelming burden is on the END USER to determine that their
statistical package is suitable for the application intended and to have
written SOPs (Standard Operating Procedures) to define how they will
validate their installation and use of the statistical software. 

This goes to some of the comments that Cody had relative to IQ/OQ/PQ
documentation, which refers to Installation Qualification, Operational
Qualification and Performance Qualification.

For example, in the context of R, the use of "make check-all" and the
retention of the output subsequent to compiling R from source code can
be part of that documentation process. Bert referred to this in his
comments.

Beyond that, the details of such documentation will be driven by a
variety of characteristics that are relevant to the nature of the
environment (academic, commercial, clinical, pre-clinical, etc.) in
which one is operating and related considerations.

As Frank noted, there will be a session at useR!2007:

  http://user2007.org/

entitled "The Use of R in Clinical Trials and Industry-Sponsored Medical
Research".  This session will take place on Friday, August 10 and I
would invite any interested parties to attend the meetings. I think that
you will find the subject matter quite enlightening.

One closing comment:  There is increasing use of R within the FDA itself
and this will only further help to assuage the fears of prospective
users over time.

Best regards,

Marc Schwartz