[R-sig-ME] optimizers for mixed models

[Ben Bolker]

Pantelis Hadjipantelis <kalakouentin at ...> writes:

> 
> I have personally used uobyqa() from minqa package (not bobyqa()) with
> success as a fast way to optimize the log-restricted-deviance without using
> derivatives. I had to move the optimization constraints within the function
> evaluation (ie. exponentiating everything before being used so I make sure
> they appear positive, taking cosines to bound parameters with [-1,1]). It
> appeared rather robust in cases that gradient-assisted BFGS seemed to
> converge to local minima's.

  Good to know.  Just to repeat: in many cases the best-fitting model
is singular, i.e. located on the constraint boundary of one or more of
the theta parameters -- in which case transforming to remove
constraints isn't a great idea -- but maybe "the gradient-assisted
BFGS seemed to converge to [a] local [minimum]" means precisely that
the solution was *not* on this boundary ... this ability to handle
singular cases is very convenient, and is in IMO one of lme4's
strengths.

  Ben Bolker
 
> >
> > On Sat, Mar 16, 2013 at 7:48 PM, Ben Bolker <bbolker at ...> wrote:
> >> >
> >> > On 3/14/2013 10:55 AM, Ross Boylan wrote:
> >> > > Optimization was originally via a custom optimizer using rho.  The
> >> > > custom optimizer did not incorporate bounds, and blew up when it got
> >> > > rho outside of [-1, 1].  So we switched to atanh(rho) as the target of
> >> > > optimization.  However, for some simulated datasets that failed to
> >> > > converge, as atanh(rho) marched slowly off toward infinity.  We
> >> > > switched to optim with bounds to cut that process off.
> >> > >
> >> > > So perhaps we should go back to rho, but using optim or the other
> >> > > bounded optimizers you suggested.
> >> > >
> >> > > So the fact that atanh(rho) is unbounded is a feature from some
> >> > > perspectives, but a bug from others.
> >> > I forgot to mention that we actually have analytic second (and first)
> >> > derivatives.  Switching to optim from the packages internal optimizer
> >> > meant we're no longer using the analytic 2nd derivative.
> >>

 [snip]