[R] Cautioning optim() users about "Nelder-Mead" default - (originally) Optim instability

[lorenzo.isella at gmail.com]

Thanks a lot, Ravi.
Indeed you best understood the point of my email.
I am perfectly aware that most of the optimization algorithms find
local rather than global minima and therefore the choice of the
initial parameters plays (at least in principle) a role.
Nevertheless, my optimization problem is rather trivial and I did not
bother to look for anything beyond the most basic tool in R for
optimization.
What surprised me is that an algorithm different from the default one
in optim() is extremely robust to a partially deliberate bad choice
ofthe initial parameters, whereas the standard one is not.
You perfectly answered my question.
Regards

Lorenzo


On Sun, Nov 15, 2015 at 05:02:52PM +0000, Ravi Varadhan wrote:
>Hi,
>
>
>
>While I agree with the comments about paying attention to parameter scaling, a major issue here is that the default optimization algorithm, Nelder-Mead, is not very good.  It is unfortunate that the optim implementation chose this as the "default" algorithm.  I have several instances where people have come to me with poor results from using optim(), because they did not realize that the default algorithm is bad.  We (John Nash and I) have pointed this out before, but the R core has not addressed this issue due to backward compatibility reasons.
>
>
>
>There is a better implementation of Nelder-Mead in the "dfoptim" package.
>
>
>
>?require(dfoptim)
>
>mm_def1 <- nmk(par = par_ini1, min.perc_error, data = data)
>
>mm_def2 <- nmk(par = par_ini2, min.perc_error, data = data)
>
>mm_def3 <- nmk(par = par_ini3, min.perc_error, data = data)
>
>print(mm_def1$par)
>
>print(mm_def2$par)
>
>print(mm_def3$par)
>
>
>
>In general, better implementations of optimization algorithms are available in packages such as "optimx", "nloptr".  It is unfortunate that most naïve users of optimization in R do not recognize this.  Perhaps, there should be a "message" in the optim help file that points this out to the users.
>
>
>
>Hope this is helpful,
>
>Ravi
>