[R] nlm's Hessian update method

[John C Nash]

BFGS is actually Fletcher's (1970) variable metric code that I modified with him in
January 1976 in Dundee and then modified very, very slightly (to insist that termination
only occurred on a steepest descent search -- this messes up the approximated inverse
Hessian, but I have an experimental Rvmminx in the optimization and solver project on
R-forge for those interested).

I haven't looked at the internals of nlm to judge the precise update used. However, the
code is there in R-2.13.1 under src/appl/uncmin.c, where at first glance a simple finite
difference approximation to the Hessian seems to be used. (numDeriv would likely do
better, even without the analytic gradient, because it uses an extrapolation process to
improve the approximation.)

Given that you do not have  a huge problem, it is likely safer to use numDeriv to compute
the Hessian when you need it, preferably using the Jacobian function on the (analytic)
gradient if the latter is available. That is what we do in optimx. Currently I am doing a
"back to basics" refactoring of optimx. The updated version is working, but I expect it
will be a few more weeks before I have a sufficiently comprehensive set of tests in place
and run. However, if someone is eager, I can provide access to the code earlier. Contact
me off-list. The existing version on CRAN works reasonably well, but the code was getting
too heavily patched.

John Nash



Date: Thu, 22 Sep 2011 12:15:41 +1000 From: Amy Willis <amy.willis at anu.edu.au> To:
r-help at r-project.org Subject: [R] nlm's Hessian update method Message-ID:
<343E6871-4564-4D9D-90F0-F2C9B30EA484 at anu.edu.au> Content-Type: text/plain;
charset=us-ascii Hi R-help! I'm trying to understand how R's nlm function updates its
estimate of the Hessian matrix. The Dennis/Schnabel book cited in the references presents
a number of different ways to do this, and seems to conclude that the positive-definite
secant method (BFGS) works best in practice (p201). However, when I run my code through
the optim function with the method as "BFGS", slightly different estimates are produced to
that of nlm:
> > optim(strt,jointll2,method="BFGS",hessian=T)$par
   -0.4016808     0.6057144     0.3744790    -7.1819734     3.0230386     0.4446641

> > nlm(jointll2,strt,hessian=T)$estimate
[1] -0.4016825  0.6057159  0.3744765 -7.1819737  3.0230386  0.4446623

Can anyone tell me if nlm employs the BFGS method for updating its estimates? Or does it
use another algorithm?


Thank you!



------------------------------


On 09/22/2011 06:00 AM, r-help-request at r-project.org wrote:
> Date: Thu, 22 Sep 2011 12:15:41 +1000 From: Amy Willis <amy.willis at anu.edu.au> To:
> r-help at r-project.org Subject: [R] nlm's Hessian update method Message-ID:
> <343E6871-4564-4D9D-90F0-F2C9B30EA484 at anu.edu.au> Content-Type: text/plain;
> charset=us-ascii Hi R-help! I'm trying to understand how R's nlm function updates its
> estimate of the Hessian matrix. The Dennis/Schnabel book cited in the references presents
> a number of different ways to do this, and seems to conclude that the positive-definite
> secant method (BFGS) works best in practice (p201). However, when I run my code through
> the optim function with the method as "BFGS", slightly different estimates are produced to
> that of nlm:
>> > optim(strt,jointll2,method="BFGS",hessian=T)$par
>    -0.4016808     0.6057144     0.3744790    -7.1819734     3.0230386     0.4446641 
> 
>> > nlm(jointll2,strt,hessian=T)$estimate
> [1] -0.4016825  0.6057159  0.3744765 -7.1819737  3.0230386  0.4446623
> 
> Can anyone tell me if nlm employs the BFGS method for updating its estimates? Or does it use another algorithm?
> 
> 
> Thank you!
> 
> 
> 
> ------------------------------