[R] Levenberg-Marquardt algorithm

[Hiroyuki Kawakatsu]

On Wed, 10 Jan 2001, Prof Brian Ripley wrote:

> 1) R does not specialised software for non-linear least squares: nls
> is Gauss-Newton inside, I believe. (I could not find that
> documented, except for the S version.) But NLS is a specialized
> problem and not that common in my experience.

Maybe in statistics, but not quite in econometrics. Most econometrics
textbooks discuss non-linear least squares in quite details. See, for
example, my favorite textbook

Davidson and MacKinnon (1993) Estimation and Inference in
Econometrics, Oxford University Press.

> For such problems general optimization algorithms (e.g. those in
> optim) are often at least as good as specialized NLS methods. So I
> did not think it worth implementing the current crop of specialized
> NLS methods.

There is an issue of what is meant by "good." Most NLS models
estimated in econometrics are small dimension "dense" models and for
these problems there is evidence that specialized algorithms produce
"better" results in terms of accuracy. There is a recent article^(1)
that compares the numerical accuracy of NLS estimates in canned
statistical packages using the NLS problems posted at NIST's
statistical reference datasets page at

http://www.nist.gov/itl/div898/strd/nls/nls_main.shtml

It is my experience that, at least for the dense NIST problems, the
specialized algorithms (for example, TOMS algorithm 573, NL2SOL
written by Dennis, Gay, and Welsch in 1981) return more accurate
estimates than general minimization routines that do not exploit the
least squares structure. I do not recall whether R was tested in the
article sited below, but it would be great if someone could volunteer
to test the accuracy of the R optimizer for the NIST problems.

(1) McCullough, BD. "Assessing the reliability of statistical
software"
Part  I: AMERICAN STATISTICIAN, 1998 NOV, V52 N4:358-366.
Part II: AMERICAN STATISTICIAN, 1999 MAY, V53 N2:149-159.

h.
--------------------------------------------------------------------
Time series regression studies give no sign of converging toward the
truth. (Phillip Cagan)

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._