[R] nonlinear least squares fitting Trust-Region"

[RAVI VARADHAN]

As suggested by Prof. Ripley, you should read a good book in the optimization area.  One that I would highly recommend is the book by Dennis and Schnabel (1983) - Numerical methods for unconstrained optimization, which does a great job of explaining both "line-search" and "trust-region" approaches for achieving globally-convergent versions of a fast numerical scheme such as Gauss-Newton.

Best,
Ravi.

----- Original Message -----
From: Prof Brian Ripley <ripley at stats.ox.ac.uk>
Date: Saturday, September 2, 2006 5:51 am
Subject: Re: [R] nonlinear least squares fitting Trust-Region"
To: Martin Ivanov <tramni at abv.bg>
Cc: r-help at stat.math.ethz.ch

> I believe people (including me) did not reply because you appeared 
> not to 
> have done your homework.  The help page for ?nls _does_ have a 
> reference 
> to the 'port' documentation, and RSiteSearch("trust region") is 
> informative and leads to an R package that does trust-region 
> optimization.  
> (So would looking in the R FAQ.)
> 
> You say:
> 
> > Since I am not an expert in the field of optimization, I am just 
> > conforming to what matlab documentation
> 
> Please note that some of the R developers are really expert in 
> that area, 
> and their advice (in the R documentation) should be taken as 
> seriously as 
> that in some commercial package that is merely commenting about 
> the very 
> sparse choice it offers.  Or if R is not in your personal trust 
> region, 
> just use 'matlab'.
> 
> Please
> 
> 1) do not shout at your helpers: using all caps is regarded as 
> shouting.
> 2) study and follow the posting guide.  People are much more 
> likely to 
> help you if you demonstrate you have made efforts to help yourself.
> 
> 3) read the literature.  The R FAQ leads to books that cover 
> fitting 
> non-linear models in S/R in considerable detail.
> 
> 
> On Sat, 2 Sep 2006, Martin Ivanov wrote:
> 
> > Dear Mr Graves,
> 
> > Thank you very much for your response. Nobody else from this 
> mailing 
> > list ventured to reply to me for the two weeks since I posted my 
> > question. "nlminb" and "optim" are just optimization procedures. 
> What I 
> > need is not just optimization, but a nonlinear CURVE FITTING 
> procedure.
> Which is just optimization: usually by least squares (although you 
> have 
> not actually specified that and there are better modern 
> statistical 
> ideas).
> 
> > If there is some way to perform nonlinear curve fitting with the 
> > "Trust-Region" algorithm using any of these functions, I would 
> me much 
> > obliged to you if you suggest to me how to achieve that. You 
> asked me 
> > why I do not want Gauss-Newton. Since I am not an expert in the 
> field of 
> > optimization, I am just conforming to what matlab documentation 
> > suggests, namely: "Algorithm used for the fitting procedure: 
> > Trust-Region -- This is the default algorithm and must be used 
> if you 
> > specify coefficient constraints. Levenberg-Marquardt -- If the 
> > trust-region algorithm does not produce a reasonable fit, and 
> you do not 
> > have coefficient constraints, you should try the Levenberg-
> Marquardt 
> > algorithm. Gauss-Newton --THIS ALGORITHM IS POTENTIALLY FASTER 
> THAN THE 
> > OTHER ALGORITHMS, BUT IT ASSUMES THAT THE RESIDUALS ARE CLOSE TO 
> ZERO. 
> > IT IS INCLUDED FOR PEDAGOGICAL REASONS AND SHOULD BE THE LAST 
> CHOICE FOR 
> > MOST MODELS AND DATA SETS. I browsed some literature about the 
> garchfit 
> > function, but I did not see the "Trust-Region" algorithm there 
> either: 
> > algorithm = c("sqp", "nlminb", "lbfgsb", "nlminb+nm", 
> "lbfgsb+nm"), 
> > control = list(), title = NULL, description = NULL, ...)
> > 
> > Thank you for your attention. I am looking forward to your reply.
> > Regards,
> > Martin
> > 
> > -----------------------------------------------------------------
> > vbox7.com - ??????? ????? ???????!
> > 
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-
> project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> > 
> 
> -- 
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.htmland provide commented, minimal, self-contained, 
> reproducible code.
>