[R] MLE maximum number of parameters

[Spencer Graves]

	  Applications with lots of parameters also tend to have parameters in 
a relatively small number of families, and each of these few families 
could be considered to have a distribution.  Splines, for example, have 
lots of parameters -- sometimes more parameters than observations (as do 
neural nets and other data mining techniques).  Spline estimation 
virtually always incorporates some kind of smoothness penalty.  The 
smoothness penalty is in essence an attempt to "eat the Bayesian omelet 
without breaking the Bayesian egg."(1)

	  In such situations, I believe it's wiser to embrace Bayes at least to 
the extent of using something like "lme", "nlme" or "lmer".  This 
redefines the problem in terms of estimating a small number of 
"hyperparameters" (using Frequentist methods with the 'nlme' and 'lme4' 
packages) and getting individual estimates of the larger number of 
"random" parameters conditional on the estimates of the hyperparameters.

	  Hope this helps.
	  Spencer Graves

(1) I don't know where I heard the phrase "eating the Bayesian omelet 
without breaking the Bayesian egg", but Google found it in the 
following: 
"http://philosophy.elte.hu/colloquium/2004/May-June/classicalstats.pdf".

Roger D. Peng wrote:
> It really depends on how well-behaved your objective function is, but I've been 
> able to fit a few models with 10--15 parameters.  But I felt like I was 
> stretching the limit there.
> 
> -roger
> 
> Federico Calboli wrote:
>> Hi All,
>>
>> I would like to know, is there a *ballpark* figure for how many  
>> parameters the minimisation routines can cope with?
>>
>> I'm asking because I was asked if I knew.
>>
>> Cheers,
>>
>> Federico
>>
>> --
>> Federico C. F. Calboli
>> Department of Epidemiology and Public Health
>> Imperial College, St. Mary's Campus
>> Norfolk Place, London W2 1PG
>>
>> Tel +44 (0)20 75941602   Fax +44 (0)20 75943193
>>
>> f.calboli [.a.t] imperial.ac.uk
>> f.calboli [.a.t] gmail.com
>>
>> ______________________________________________
>> R-help at stat.math.ethz.ch mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>>
>