[R] AFT-model with time-dependent covariates

[Ravi Varadhan]

Goran,

You had stated that:

"I have only coded the log likelihood function, and I use 'optim' (in R code) and 'BFGS', without
gradient, and I ask for a numerically differentiated Hessian, which I use for calculating standard errors and p-values. Tests show that this works surprisingly well, but for huge data sets it is very slow. If it was possible to ask for a hessian in the C version of BFGS, things would improve a lot."

Here is a suggestion that could potentially work well for large data sets.  There is a function called `bobyqa' in a package called "minqa" that is currently on R-forge (soon to be released; Kate Mullen is the author).  

http://r-forge.r-project.org/R/?group_id=395

This does not use derivatives, and might be faster on large-data sets.  You could then use the "numDeriv" package to calculate the hessian of the log-likelihood at the converged parameter value.  

Best,
Ravi.

-----------------------------------------------------------------------------------

Ravi Varadhan, Ph.D.

Assistant Professor, The Center on Aging and Health

Division of Geriatric Medicine and Gerontology 

Johns Hopkins University

Ph: (410) 502-2619

Fax: (410) 614-9625

Email: rvaradhan at jhmi.edu

Webpage:  http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.html



------------------------------------------------------------------------------------

-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of Göran Broström
Sent: Tuesday, October 20, 2009 10:07 AM
To: spencerg
Cc: r-help at r-project.org; Terry Therneau
Subject: Re: [R] AFT-model with time-dependent covariates

Sorry for being late in responding to this thread, but I was made
aware of it only two weeks ago. In my package 'eha' there is a
function 'aftreg', which performs what is asked for, given that the
time-varying covariates are step functions of time and that the
observation period for each individual is an interval. Left truncation
works if it can be assumed that the covariate values during the first
non-observable interval are the same as at the beginning of the first
interval under observation.

As Terry wrote, there is a lot of book-keeping in writing down the
likelihood function, and even more for scores and the hessian, so I
adopted a lazy way in the programming: I have only coded the log
likelihood function, and I use 'optim' (in R code) and 'BFGS', without
gradient, and I ask for a numerically differentiated Hessian, which I
use for calculating standard errors and p-values. Tests show that this
works surprisingly well, but for huge data sets it is very slow. If it
was possible to ask for a hessian in the C version of BFGS, things
would improve a lot.

Also note that you need the latest version (1.2-12) of eha for this to
work. aftreg in arlier versions only works (correctly) for
time-constant covariates. And, this is not well tested, so care is
needed. And I appreciate performance and bug reports, of course.

Göran

On Wed, May 13, 2009 at 9:34 PM, spencerg <spencer.graves at prodsyse.com> wrote:
> To see what's available in other packages, try the following:
>
> library(RSiteSearch)
> AFT <- RSiteSearch.function('AFT model')
> summary(AFT) # 24 help files found in 8 different packages
> HTML(AFT) # opens a table with 24 rows in a web browser.
>     There may be nothing here that will help you, but this provides a quick
> overview of what's available.  If this doesn't find what you want, it either
> has not been contributed or its help page does not use the phrase "AFT
> model".
>
>     Hope this helps.     Spencer Graves
>
> Terry Therneau wrote:
>>
>>  The coding for an AFT model with time-dependent covariates will be very
>> hard, and I don't know of anyone who has done it.  (But I don't keep watch
>> of other survival packages, so something might be there).
>>    In a Cox model, a subject's risk depends only on the current value of
>> his/her covariates; in an AFT model the risk depends on the entire covariate
>> history.  (My 'accelerated age' is the sum of all the extra years I have
>> ever gained).  Coding this is not theoretically complex, but would be a
>> pain-in-the-rear amount of bookkeeping.
>>          Terry Therneau
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Göran Broström

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.