[R-sig-ME] Question regarding updates or changes to lme4 package

[Ben Bolker]

   A few quick thoughts.

   If you gave an example with data someone who was knowledgeable and 
motivated could try this out with an older version of lme4 to see if the 
results changed (e.g. by checking out an old commit from the GitHub 
repository and installing it).

  The 'snapshot' package allows you to install old versions of packages 
(specifying a target date) from Microsoft's archival-mirror of CRAN 
(MRAN), this would be another (easier) way to check for differences 
depending on lme4 changes.

   This kind of problem is a good argument for  using frameworks like 
the 'packrat' or 'renv' packages to manage the package 
versions/dependencies used in your analyses.

   All of the changes that the maintainers think could affect results 
are listed in the NEWS file; *all* of the changes are listed in the 
commit history of the GitHub repository.  I think it's a very long shot, 
but there have been (1) minor changes to anova() and (2) possibly some 
changes to the environments in which some objects are evaluated 
[although IIRC these changes were actually reverted because they caused 
problems in some downstream packages].

   My *best* guess is that the differences might come from whether or 
not you had the lmerTest package loaded; the lmerTest package changes 
the behaviour of some functions in a generally useful but 
not-always-transparent way ...

    cheers
      Ben Bolker




On 6/4/21 11:57 PM, Allen, Frederick wrote:
> Hello,
> 
> I hope this email finds you well! I have a question with regard to the lme4 package (which I was told to redirect to this address). What, if any updates have been made to the package in the past six months to one year? Were any changes made to the package (or, if you know of any, to the imported packages) that would affect how p-values are calculated? (I looked at lme4 news but didn't find anything that I thought would explain my issue below.)
> 
> The reason I ask is this: about 9 months ago, I constructed linear effects models (plural, because we do this over a thousand times to analyze individual entities for -omics data) with and without a continuous independent (fixed) variable and with and without a categorical independent (fixed) variable to determine their contribution to a model for the dependent variable. At the time, we extracted coefficients from the model, as well as calculated p-values for contribution of each variable to the model.
> 
> When I repeat this analysis today with the same code and the same data, the coefficients and p-values for the categorical variable are exactly the same to the sixth decimal point (with occasional differences beyond the sixth decimal point). However, for the continuous variable, the coefficients are exactly the same, but the p-values are entirely different.
> 
> Of note, our workflow to extract p-values is as follows:
> 
> models.with <- dlply(data, "groups", function(df) lmer(Independent ~ categorical + continous + (1|random), data=df, REML = FALSE))
> models.without <- dlply(data, "groups", function(df) lmer(Independent ~ categorical + (1|random), data=df, REML = FALSE))
> object1 <- mapply(x=model.without, y=model.with, function(x,y) anova(x,y))
> 
> I would send an example with data, but I have no way of reproducing the old p-values I got.
> 
> Thank you for your help!
> -Fred
> 
> 
> Please consider the environment before printing this e-mail
> 
> Cleveland Clinic is currently ranked as the No. 2 hospital in the country by U.S. News & World Report (2017-2018). Visit us online at http://www.clevelandclinic.org for a complete listing of our services, staff and locations. Confidentiality Note: This message is intended for use only by the individual or entity to which it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. Thank you.
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>