[R] MLE maximum number of parameters

[Patrick Burns]

Seagulls have a very different perspective to ballparks
than ants.  Nonetheless, there is something that can be
said.

There are several variables in addition to the number of
parameters that are important.  These include:

* The complexity of the likelihood

* The number of observations in the dataset

* How close to the optimum is close enough

* Your patience

The latter is undoubtedly the most important of all.  It
matters a lot whether you think a minute is a long time
or only periods measured in weeks.

The optimization strategy can also have a big effect.  If
you are using a derivative-based optimizer, then the number
of parameters can have a big impact.  Typically one iteration
in such algorithms requires p+1 function calls, where p is the
number of parameters.  Since more iterations are generally
required with more parameters, the speed can decrease
rapidly as the number of parameters increases.

One strategy to deal with a large number of parameters is to
start with something like a genetic algorithm.  Once the genetic
algorithm has a pretty good solution, then switch to a derivative-
based algorithm to finish.  The amount to run the initial
algorithm before switching depends on the problem, the quality
of the two optimizers, and probably other things.

With this switching strategy and at least a modicum of patience,
problems with thousands of parameters may be feasible to solve.

Patrick Burns
patrick at burns-stat.com
+44 (0)20 8525 0696
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")

Federico Calboli wrote:

>Hi All,
>
>I would like to know, is there a *ballpark* figure for how many  
>parameters the minimisation routines can cope with?
>
>I'm asking because I was asked if I knew.
>
>Cheers,
>
>Federico
>
>--
>Federico C. F. Calboli
>Department of Epidemiology and Public Health
>Imperial College, St. Mary's Campus
>Norfolk Place, London W2 1PG
>
>Tel +44 (0)20 75941602   Fax +44 (0)20 75943193
>
>f.calboli [.a.t] imperial.ac.uk
>f.calboli [.a.t] gmail.com
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>
>
>  
>