[R-sig-ME] Suggestions on how to correct a misapprehension?

[Phillip Alday]

I believe I have a login somewhere for Wikipedia and would be happy to 
make an edit when I get a few minutes. This has bothered me for a while 
too, but I get busy and forget to do something about it ...

Skimming the Python code, I'm not sure that it's using EM, but there's a 
lot of indirection and it's a codebase I'm not familiar with. I'm also 
not convinced that SAS is using EM (and I recently read the PROC MIXED 
documentation).

Currently Wikipedia reads so:

> One method used to fit such mixed models is that of the 
> expectation–maximization algorithm 
> <https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm> 
> where the variance components are treated as unobserved nuisance 
> parameters <https://en.wikipedia.org/wiki/Nuisance_parameter> in the 
> joint likelihood.^[11] 
> <https://en.wikipedia.org/wiki/Mixed_model#cite_note-11> Currently, 
> this is the method implemented in major statistical software such as R 
> <https://en.wikipedia.org/wiki/R_(programming_language)> (lme4 
> package), Python 
> <https://en.wikipedia.org/wiki/Python_(programming_language)> 
> (statsmodels <https://en.wikipedia.org/wiki/Statsmodels> package), 
> Julia <https://en.wikipedia.org/wiki/Julia_(programming_language)> 
> (MixedModels.jl package), and SAS 
> <https://en.wikipedia.org/wiki/SAS_(software)> (proc mixed). The 
> solution to the mixed model equations is a maximum likelihood estimate 
> <https://en.wikipedia.org/wiki/Maximum_likelihood_estimate> when the 
> distribution of the errors is normal.

My proposed edit would be something along the lines of:

> There are several methods to fit mixed models, including 
> expectation-maximization, generalized least squares (used by R's 
> nlme), penalized least squares (used by R's lme4 and MixedModels.jl) 
> and direct optimization of the likelihood (used by e.g. R's glmmTMB). 
> Notably, while the canonical form proposed by Henderson is useful for 
> theory, many popular software packages use a different formulation for 
> numerical computation in order to take advantage of sparse matrix 
> methods (e.g. lme4 and MixedModels.jl).

Less technical "listeners" on this list should feel free to speak up now 
if this is unclear because Wikipedia should be for a fairly broad audience!

Phillip

On 12/13/22 11:39, Douglas Bates wrote:
> So my family is having to live through a "Someone is wrong on the
> internet",https://xkcd.com/386/, moment. In the past couple of days I have
> twice encountered the same mistaken characterization of how the parameter
> estimates in lmer and in the MixedModels.jl package are evaluated.
>
> As we documented in our 2015 paperhttp://dx.doi.org/10.18637/jss.v067.i01
> in lme4 the REML estimates or the ML estimates for the parameters of a
> linear mixed-effects model are evaluated by constrained optimization of a
> profiled log-likelihood or profiled log-restricted-likelihood.  The
> parameters directly being optimized are the elements of relative covariance
> factors.  The profiling involves solving a penalized least squares
> problem.  This PLS representation, and the use of sparse matrices, is what
> allows for fitting models with random effects associated with crossed or
> partially crossed grouping factors, such as "subject" and "item".  To many
> users this capability is one of the big selling points for lme4.
>
> In our paper we explain in great detail why this approach is, in our
> opinion, superior to earlier approaches.  And if someone doesn't believe
> us, both lme4 and MixedModels.jl are Open Source projects so anyone who
> wants to do so can just go read the code to find out what actually is done.
>
> So it came as a surprise when reading the Wikipedia entry on mixed models,
> https://en.wikipedia.org/wiki/Mixed_model, to learn that lme4 and
> MixedModels.jl use an EM algorithm that Mary Lindstrom and I described (
> https://doi.org/10.1080%2F01621459.1988.10478693) in 1988.  It is possible
> that in the early days of lme4 there was such an implementation, but not in
> the last 15 years, and there definitely has never been such an
> implementation in MixedModels.jl.  I noticed that the Python package
> statsmodels is described in the Wikipedia article and in their
> documentation,https://www.statsmodels.org/stable/mixed_linear.html, as
> using that EM algorithm. I didn't verify this in the code because reading
> code based on numpy and scipy causes me to start ranting and raving to the
> extent that family members need to take away my laptop and put me in a
> quiet room with the window shades drawn until I promise to behave myself.
>
> Anyway the Python statsmodels documentation claims that lme4 uses this
> method, which it doesn't.
>
> I have never gone through the process of proposing an edit in a Wikipedia
> article.  As I understand it I would need to create a login etc.  Would
> anyone who does have such a login be willing to propose an edit and save me
> the steps?
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org  mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
	[[alternative HTML version deleted]]