[R-sig-ME] Suggestions on how to correct a misapprehension?

[Jeff Newmiller]

Definitely a layperson in this field, but my reading of your proposed edit (matrix representation) doesn't feel equivalent to the objections raised by Doug, which appear to me to be deeper into what is being optimized than just the numerical efficiency of the calculations.

On December 13, 2022 9:55:27 AM PST, Phillip Alday <me using phillipalday.com> wrote:
>I believe I have a login somewhere for Wikipedia and would be happy to 
>make an edit when I get a few minutes. This has bothered me for a while 
>too, but I get busy and forget to do something about it ...
>
>Skimming the Python code, I'm not sure that it's using EM, but there's a 
>lot of indirection and it's a codebase I'm not familiar with. I'm also 
>not convinced that SAS is using EM (and I recently read the PROC MIXED 
>documentation).
>
>Currently Wikipedia reads so:
>
>> One method used to fit such mixed models is that of the 
>> expectation–maximization algorithm 
>> <https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm> 
>> where the variance components are treated as unobserved nuisance 
>> parameters <https://en.wikipedia.org/wiki/Nuisance_parameter> in the 
>> joint likelihood.^[11] 
>> <https://en.wikipedia.org/wiki/Mixed_model#cite_note-11> Currently, 
>> this is the method implemented in major statistical software such as R 
>> <https://en.wikipedia.org/wiki/R_(programming_language)> (lme4 
>> package), Python 
>> <https://en.wikipedia.org/wiki/Python_(programming_language)> 
>> (statsmodels <https://en.wikipedia.org/wiki/Statsmodels> package), 
>> Julia <https://en.wikipedia.org/wiki/Julia_(programming_language)> 
>> (MixedModels.jl package), and SAS 
>> <https://en.wikipedia.org/wiki/SAS_(software)> (proc mixed). The 
>> solution to the mixed model equations is a maximum likelihood estimate 
>> <https://en.wikipedia.org/wiki/Maximum_likelihood_estimate> when the 
>> distribution of the errors is normal.
>
>My proposed edit would be something along the lines of:
>
>> There are several methods to fit mixed models, including 
>> expectation-maximization, generalized least squares (used by R's 
>> nlme), penalized least squares (used by R's lme4 and MixedModels.jl) 
>> and direct optimization of the likelihood (used by e.g. R's glmmTMB). 
>> Notably, while the canonical form proposed by Henderson is useful for 
>> theory, many popular software packages use a different formulation for 
>> numerical computation in order to take advantage of sparse matrix 
>> methods (e.g. lme4 and MixedModels.jl).
>
>Less technical "listeners" on this list should feel free to speak up now 
>if this is unclear because Wikipedia should be for a fairly broad audience!
>
>Phillip
>
>On 12/13/22 11:39, Douglas Bates wrote:
>> So my family is having to live through a "Someone is wrong on the
>> internet",https://xkcd.com/386/, moment. In the past couple of days I have
>> twice encountered the same mistaken characterization of how the parameter
>> estimates in lmer and in the MixedModels.jl package are evaluated.
>>
>> As we documented in our 2015 paperhttp://dx.doi.org/10.18637/jss.v067.i01
>> in lme4 the REML estimates or the ML estimates for the parameters of a
>> linear mixed-effects model are evaluated by constrained optimization of a
>> profiled log-likelihood or profiled log-restricted-likelihood.  The
>> parameters directly being optimized are the elements of relative covariance
>> factors.  The profiling involves solving a penalized least squares
>> problem.  This PLS representation, and the use of sparse matrices, is what
>> allows for fitting models with random effects associated with crossed or
>> partially crossed grouping factors, such as "subject" and "item".  To many
>> users this capability is one of the big selling points for lme4.
>>
>> In our paper we explain in great detail why this approach is, in our
>> opinion, superior to earlier approaches.  And if someone doesn't believe
>> us, both lme4 and MixedModels.jl are Open Source projects so anyone who
>> wants to do so can just go read the code to find out what actually is done.
>>
>> So it came as a surprise when reading the Wikipedia entry on mixed models,
>> https://en.wikipedia.org/wiki/Mixed_model, to learn that lme4 and
>> MixedModels.jl use an EM algorithm that Mary Lindstrom and I described (
>> https://doi.org/10.1080%2F01621459.1988.10478693) in 1988.  It is possible
>> that in the early days of lme4 there was such an implementation, but not in
>> the last 15 years, and there definitely has never been such an
>> implementation in MixedModels.jl.  I noticed that the Python package
>> statsmodels is described in the Wikipedia article and in their
>> documentation,https://www.statsmodels.org/stable/mixed_linear.html, as
>> using that EM algorithm. I didn't verify this in the code because reading
>> code based on numpy and scipy causes me to start ranting and raving to the
>> extent that family members need to take away my laptop and put me in a
>> quiet room with the window shades drawn until I promise to behave myself.
>>
>> Anyway the Python statsmodels documentation claims that lme4 uses this
>> method, which it doesn't.
>>
>> I have never gone through the process of proposing an edit in a Wikipedia
>> article.  As I understand it I would need to create a login etc.  Would
>> anyone who does have such a login be willing to propose an edit and save me
>> the steps?
>>
>> 	[[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models using r-project.org  mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>	[[alternative HTML version deleted]]
>
>_______________________________________________
>R-sig-mixed-models using r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 
Sent from my phone. Please excuse my brevity.