[R] Levenberg-Marquardt algorithm

[Prof Brian Ripley]

On Wed, 10 Jan 2001, Dermot MacSweeney wrote:

> Hi All,
>
> Is the Levenberg-Marquardt algorithm available in R. This method combines the
> steepest descent algorithm and Newton's method.

My books have it as a modified Gauss-Newton method for non-linear least
squares problems, and although its some years since I read the papers,
I am pretty sure that is what the author's tackled.  Another way to look
at it is that it uses ridge regression on the local linearizations
to find the step.

1) R does not specialised software for non-linear least squares: nls is
Gauss-Newton inside, I believe.  (I could not find that documented, except
for the S version.)  But NLS is a specialized problem and not that common
in my experience.

2) Levenberg-Marquardt is hardly competitive these days. It probably
was not by the time of Marquardt(1963) in statistical problems, which are
normally in optimization parlance `large residual problems'.  That is, the
best fit is not nearly exact.

For such problems general optimization algorithms (e.g. those in optim)
are often at least as good as specialized NLS methods.  So I did not
think it worth implementing the current crop of specialized NLS methods.

The Nocedal and Wright (1999) reference in ?optim is an excellent
overview: I had page 267 open as I wrote this.

`Performance of the Gauss-Newton and Levenberg-Marquardt algorithms is
usual poor in the large-residual case.'

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._