[R] Non-finite finite-difference value" error in eha's, aftreg

[John C Nash]

This kind of error seems to surprise R users. It surprises me that it doesn't happen much
more frequently. The "BFGS" method of optim() from the 1990 Pascal version of my book was
called the Variable Metric method as per Fletcher's 1970 paper it was drawn from. It
really works much better with analytic gradients, and the Rvmmin package which is an all-R
version that adds bounds and masks is set up to generate a warning if they are not
available. Even with bounds, the finite different derivative code can step over a cliff
edge with   del <- (f(x+h) - f(x))/h  i.e., bounds may not be checked within the numerical
derivative functions. And "BFGS" is not set up with bounds. "L-BFGS-B" which has them is
actually a rather different method.

If you get such error messages, why not capture the parameter vector and check the
function computation at those parameters and nearby? Yes, a bit tedious, but rarely have I
found it a waste of time. For information, there should be a small function available
shortly on R-forge (project optimizer, likely in the optfntools package) to do an axial
search around a set of parameters and generate some information about the functional
surface. I still have to prepare documentation and examples, but if anxious, contact me
off-list.

JN



> Message: 21
> Date: Wed, 16 Nov 2011 15:06:00 +0100
> From: Milan Bouchet-Valat <nalimilan at club.fr>
> To: r-help <r-help at r-project.org>
> Subject: [R] "Non-finite finite-difference value" error in eha's
> 	aftreg
> Message-ID: <1321452360.13624.2.camel at milan>
> Content-Type: text/plain; charset="UTF-8"
> 
> Hi list!
> 
> 
> I'm getting an error message when trying to fit an accelerated failure
> time parametric model using the aftreg() function from package eha:
>> > Error in optim(beta, Fmin, method = "BFGS", control = list(trace =
>> >   as.integer(printlevel)),  : 
>> > non-finite finite-difference value [2]
> This only happens when adding four specific covariates at the same time
> in the model (see below). I understand that kind of problem can come
> from a too high correlations between my covariates, but is there
> anything I can do to avoid it? Does something need to be improved in
> aftreg.fit?
> 
> My data set is constituted of 34,505 observations (years) of 2,717
> individuals, which seems reasonable to me to fit a complex model like
> that (covariates are all factors with less than 10 levels). I can send
> it by private mail if somebody wants to help debugging this.
> 
> The details of the model and errors follow, but feel free to ask for
> more testing. I'm using R 2.13.1 (x86_64-redhat-linux-gnu), eha 2.0-5
> and survival 2.36-9.
> 
> 
> Thanks for your help!
> 
> 
>> > m <-aftreg(Surv(start, end, event) ~ homo1 + sexego + dipref1
> +                                    + t.since.school.q,
> +            data=ms, dist="loglogistic", id=ident)
> Error in optim(beta, Fmin, method = "BFGS", control = list(trace =
> as.integer(printlevel)),  : 
>   non-finite finite-difference value [2]
> Calls: aftreg -> aftreg.fit -> aftp0 -> optim
> 
>> > traceback()
> 4: optim(beta, Fmin, method = "BFGS", control = list(trace =
> as.integer(printlevel)), 
>        hessian = TRUE)
> 3: aftp0(printlevel, ns, nn, id, strata, Y, X, offset, dis, means)
> 2: aftreg.fit(X, Y, dist, strats, offset, init, shape, id, control, 
>        center)
> 1: aftreg(Surv(start, end, event) ~ homo1 + sexego + dipref1 + 
>        t.since.school.q, data = ms, dist = "loglogistic", id = ident)
>