[R] SAS or R software
henric.nilsson at statisticon.se
Sat Dec 18 14:10:40 CET 2004
Marc Schwartz said the following on 2004-12-18 01:19:
> As you are likely aware, other statistically relevant issues are
> contained in various ICH guidance documents regarding GCP considerations
> and principles for clinical trials:
ICH E9 states that (p. 27):
"The computer software used for data management and statistical analysis
should be reliable, and documentation of appropriate software testing
procedures should be available."
Some commercial software vendors (SAS, Insightful, and StatSoft) offer
white papers stating that their software can work within an 21 CFR Part
11 compliant system.
Some commercial vendors (SAS and Insightful) also offers tools for
validation of the installation and operation of the software. SAS has
and S-PLUS has validate().
As a statistical consultant working within the pharamceutical industry,
I think that our clients find the white papers being some kind of
quality seal. It signals that someone has actually thought about the
issues involved, written a document about it, and even stated that it
can be done. Of course, there's a lot of FUD going on here. But if our
lives can be made simpler by producing similar white papers and QA
tools, why not?
(But for some people, only SAS will do:
Last week we were audited on behalf of a client. One of the specific
issues discussed were validation and the Part 11 compliance of S-PLUS.
In this specific trial, data are to be transferred from Oracle Clinical
-> SAS -> SPLUS, and they auditors were really worried about the first
and last link of that chain. Finally, they suggested using only SAS...
And in this particular case, Part 11 is really a non-issue since
physical records exists (i.e. case report forms) and all final S-PLUS
output and code will also be stored physically (i.e. print-outs) -- no
need for electronic signatures here!)
> There is also a general guidance document for computer systems used in
> clinical trials here:
> Though it is to be superseded by a draft document here:
From the introduction (p. 2):
"This document provides guidance about computerized systems that are
used to create, modify, maintain, archive, retrieve, or transmit
clinical data required to be maintained and/or submitted to the Food and
Drug Administration (FDA)"
The `retrieve' part is certainly applicable. If we regard R as
off-the-shelf software, the guidance says (p. 11):
"For most off-the-shelf software, the design level validation will have
already been done by the company that wrote the software. Given the
importance of ensuring valid clinical trial data, FDA suggests that the
sponsor or contract research organization (CRO) have documentation
(either original validation documents or on-site vendor audit documents)
of this design level validation by the vendor and would itself have
performed functional testing (e.g., by use of test data sets) and
researched known software limitations, problems, and defect corrections.
Detailed documentation of any additional validation efforts performed by
the sponsor or CRO will preserve the findings of these efforts.
In the special case of database and spreadsheet software that is: (1)
purchased off-the-shelf, (2) designed for and widely used for general
purposes, (3) unmodified, and (4) not being used for direct entry of
data, the sponsor or contract research organization may not have
documentation of design level validation. FDA suggests that the sponsor
or contract research organization perform functional testing (e.g., by
use of test data sets) and research known software limitations,
problems, and defect corrections.
In the case of off-the-shelf software, we recommend that the following
be available to the Agency on request:
* A written design specification that describes what the software is
intended to do and how it is intended to do it;
* A written test plan based on the design specification, including both
structural and functional analysis; and
* Test results and an evaluation of how these results demonstrate that
the predetermined design specification has been met."
I think the guidance is quite clear here. We must prove to the FDA, at
their wish, that the software used is working properly. In order to do
this, we seem to need documents describing the development process and
the QA tools used by R Core. An idea of what we'll need may be found in
the `Computer Systems Validation in Clinical Research - A Practical
Guide (Edition 1)' at
Especially section 2.4, 5 + subsections, 8 + subsections, and 9.7 +
subsections seem relevant. (I've ordered the 2nd edition, but it hasn't
More information about the R-help