[R] OT Futility Analysis

[Kevin E. Thorpe]

Thank you Spencer and Steve for your helpful comments.  If I may, I
would like to elaborate on some of the points you raise.

Stephen A Roberts wrote:
> I would take the line that if they hadn't pre-specified any stopping
> rules, the only reason to stop is safety or new external data. I
> would be very suspicious of requests from the steering committee to
> stop for futility - they should be blinded so why are they thinking
> futility unless results have leaked? I would argue that they are
> obliged to finish the trial once they start.

In general I agree with this.  In this case the request for a futility
analysis came from the sponsor (a drug company).  It is a classic case
of company B buys company A and wnats to stop R&D on company A's drugs.
Therefore the company was looking for a reason to stop.  Now that they
will stop producing the drug used in the trial, recruitment will end
before reaching its target.  Now the Steering Committee's point of
view is that if there is any reasonable hope, they would find some
other way to continue recruitment.  I am confident that results have
not leaked.  I am well aquainted with the data management and blinding
procedures in place for the trial.

> This is an example of the need to sort out these things in advance -
> look up the stuff from the UK DAMOCLES project. The recent book
> edited by DeMets et al (Data Monitoring in Clinical Trials: A Case
> Studies Approach) is a good read on these sorts of issues and I think
> there is a more statistical book from the same group of authors.

Thanks for the reference.  My library has it, so will give it a look.

> As far as software is concerned, futility analysis and conditional
> power are simply standard analyses with made up data and more-or-less
> justifiable assumptions.

I am also interested if there are good alternatives to conditional
power for this type of scenario.

> Steve.
> 
> 
> 
>> 
>> What does this particular Steering Committee think a "futility 
>> analysis" is?  Do they have any particular reference(s)?  What do
>> you find in your own literature review?
>> 
>> If it were my problem, I think I'd start with questions like that. 
>> Your comments suggested to me a confounding of technical and
>> political problems.  The politics suggests the language you need to
>> use in your response.  Beyond that, I've never heard before of a
>> "futility analysis", but I think I could do one by just trying to
>> be clear about the options the Steering Committee might consider
>> plausible and then comparing them with appropriate simulations --
>> summarized as confidence intervals, as you suggest.

I did ask REPEATEDLY for guidelines from the steering committee, but
none came or are likely to come.  In fact, they wanted me to come up
with the recommendation, which I find entirely inappropriate, but here
I am.  So, I don't think I'm confounded between techincal and political.

Basically, they want to stop if there is a low chance of rejecting the
null hypothesis.  This is often referred to as conditional power or
stochastic curtailment.  I recently saw a paper by Scott Emerson
pointing out some problems (interpretation, relation to unconditional
power).

As far as references, I have used a book by Jennisen and Turnbull in
the past, but, as I recall, with the exception of stochastic
curtailment, it assumes the trial was designed with group sequential
methods.  I have also just found a 1988 Biometrics paper by Lan and
Wittes on the B-value which I will read.

>> And I hope that someone else will enlighten us both if there are 
>> better options available.
>> 
>> Best Wishes, spencer graves p.s.  For any attorneys who may read
>> these comments, the suggestions are obviously warranteed up to the
>> amount you paid for it, which is nothing. If you follow them and
>> they turn out to be inappropriate, you will pay the price.  I
>> encourage you to share the problems with me, so I can learn from
>> the experience.  However, the limits of my liability are as already
>> stated.
>> 
>> Kevin E. Thorpe wrote:
>> 
>> 
>>> I beg your pardon if this is too off topic.  I am posting here 
>>> since I hope to find an R solution to my problem.  Please indulge
>>>  me while I give a little background about what I'm trying to do.
>>> 
>>> 
>>> I'm on a DSMB for a clinical trial.  The Steering Committee for
>>> the trial has asked us to perform a futility analysis on their
>>> primary outcome which is a time-to-event endpoint.  The trial was
>>> not designed with group sequential methods, nor was any futility
>>> analysis spelled out in the protocol.  Another thing which may be
>>> relevant is that due to circumstances beyond the investigators'
>>> control, the trial will stop recruitment prematurely unless there
>>> is some compelling reason for them to find a way to continue the
>>> trial.  Lastly, the trial has accrued not quite half of the
>>> planned sample size.
>>> 
>>> Admittedly, I don't have a vast amount of experience implementing
>>>  stopping rules.  In other protocols I have seen where futility 
>>> analyses have been planned but a group sequential design has not 
>>> otherwise been employed, conditional power has been used for the 
>>> futility rule.  So naturally, that was my first thought (although
>>>  I may well be wrong) in this case.  I have done RSiteSearch()
>>> with the following terms (three different searches):
>>> 
>>> futility analysis conditional power stochastic curtailment
>>> 
>>> Nothing that looked relevant to my problem jumped out at me.
>>> 
>>> I have read, somewhat recently, that there are problems with
>>> conditional power, although I don't remember the details at the
>>> moment.  This has prompted me to consider other approaches to the
>>> problem.
>>> 
>>> One simple thing that has occurred to me, although I don't know 
>>> what the implications are is to simply look at a confidence 
>>> interval around the hazard ratio for the treatment effect.  In 
>>> the event that the CI includes 1 and excludes any clinically 
>>> important difference, I would take that as an indication of 
>>> futility.
>>> 
>>> I would appreciate your comments on this and to learn of any more
>>>  formal methods, particularly of implementations in R.
>>> 
>>> Thank you for reading.
>>> 
>>> Kevin


-- 
Kevin E. Thorpe
Biostatistician/Trialist, Knowledge Translation Program
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto
email: kevin.thorpe at utoronto.ca  Tel: 416.946.8081  Fax: 416.946.3297