[R] MLE maximum number of parameters

[Albyn Jones]

I regularly optimize functions of over 1000 parameters for posterior
mode computations using a variant of newton-raphson.  I have some
favorable conditions: the prior is pretty good, the posterior is
smooth, and I can compute the gradient and hessian.

albyn


On Mon, Jun 19, 2006 at 06:53:00PM +0100, Patrick Burns wrote:
> Seagulls have a very different perspective to ballparks
> than ants.  Nonetheless, there is something that can be
> said.
> 
> There are several variables in addition to the number of
> parameters that are important.  These include:
> 
> * The complexity of the likelihood
> 
> * The number of observations in the dataset
> 
> * How close to the optimum is close enough
> 
> * Your patience
> 
> The latter is undoubtedly the most important of all.  It
> matters a lot whether you think a minute is a long time
> or only periods measured in weeks.
> 
> The optimization strategy can also have a big effect.  If
> you are using a derivative-based optimizer, then the number
> of parameters can have a big impact.  Typically one iteration
> in such algorithms requires p+1 function calls, where p is the
> number of parameters.  Since more iterations are generally
> required with more parameters, the speed can decrease
> rapidly as the number of parameters increases.
> 
> One strategy to deal with a large number of parameters is to
> start with something like a genetic algorithm.  Once the genetic
> algorithm has a pretty good solution, then switch to a derivative-
> based algorithm to finish.  The amount to run the initial
> algorithm before switching depends on the problem, the quality
> of the two optimizers, and probably other things.
> 
> With this switching strategy and at least a modicum of patience,
> problems with thousands of parameters may be feasible to solve.
> 
> Patrick Burns
> patrick at burns-stat.com
> +44 (0)20 8525 0696
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
> 
> Federico Calboli wrote:
> 
> >Hi All,
> >
> >I would like to know, is there a *ballpark* figure for how many  
> >parameters the minimisation routines can cope with?
> >
> >I'm asking because I was asked if I knew.
> >
> >Cheers,
> >
> >Federico
> >
> >--
> >Federico C. F. Calboli
> >Department of Epidemiology and Public Health
> >Imperial College, St. Mary's Campus
> >Norfolk Place, London W2 1PG
> >
> >Tel +44 (0)20 75941602   Fax +44 (0)20 75943193
> >
> >f.calboli [.a.t] imperial.ac.uk
> >f.calboli [.a.t] gmail.com
> >
> >______________________________________________
> >R-help at stat.math.ethz.ch mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> >
> >
> >  
> >
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html