[R-sig-ME] adjusted values

Lenth, Russell V russell-lenth at uiowa.edu
Thu Mar 22 22:36:02 CET 2018


The unadjusted P values in the model summary are appropriate for model selection, but post-hoc comparisons are another objective. I, personally, would report the single-step adjusted P values that you already have obtained. In another context such as testing the contributions of massive numbers of contributors to an effect, I’d use an FDR method, because there the goal is not to avoid all type-I errors, but rather to keep from making too many false identifications.

But I know there are people who would disagree with me. In fact, I was recently kind of flamed in another forum for expressing a similar view on such issues.


From: Cristiano Alessandro <cri.alessandro at gmail.com>
Sent: Thursday, March 22, 2018 4:19 PM
To: Lenth, Russell V <russell-lenth at uiowa.edu>
Cc: r-sig-mixed-models at r-project.org
Subject: Re: adjusted values

Thanks Russ,
point (1) was clear to me; point (2) a bit less, so thanks for the hint. I do only care about comparing with the -1 level. The question is whether I should do multiple comparison corrections with glht (as I did so far), or take the p-values of the estimated coefficients. Do you have suggestions here?

On Thu, Mar 22, 2018 at 4:05 PM, Lenth, Russell V <russell-lenth at uiowa.edu<mailto:russell-lenth at uiowa.edu>> wrote:

First, I want to make clear that you understand that (1) the intercept is really the estimate of the mean when des_days = -1, and that (2) the multiplicity correction used in the test is based on the multivariate t distribution (not Bonferroni). As such, it is less conservative than Bonferroni, and is "exact" in that if the underlying assumptions are exactly true, the probability of at least one type-I error is controlled at the desired level. (That said, the P values are actually computed using a simulation method, so they will vary a bit if you call glht again).

If you really only care about comparisons with the -1 level, I think what you have is a good solution. Some people want to use weaker control of the error rate. In that case, you can use

   summary(glht(...), test = adjusted("desired choice"))

(see the help file for summary.glht), which gives you other choices besides the single-step method that it defaults to. You could in fact specify adjusted("none") to get no adjustments, or adjusted("bonferroni"), adjusted("fdr"), etc. if you want to use one of the standard methods in stats::p.adjust.methods.

Often, people want to compare *all* pairs of treatments; and if that's the case, you can specify that using a call to mcp() in the linfct argument of glht.


Russell V. Lenth  -  Professor Emeritus
Department of Statistics and Actuarial Science
The University of Iowa  -  Iowa City, IA 52242  USA
Voice (319)335-0712<tel:%28319%29335-0712> (Dept. office)  -  FAX (319)335-3017<tel:%28319%29335-3017>

On 18-03-22 01:28 PM, Cristiano Alessandro wrote:
> Hi all,
> I am fitting a linear mixed model with lme4 in R. The model has a
> single factor (des_days) with 4 levels (-1,1,14,48), and I am using
> random intercept and slopes.
> Fixed effects: data ~ des_days
>                  Value   Std.Error  DF   t-value p-value
> (Intercept)  0.8274313 0.007937938 962 104.23757  0.0000
> des_days1   -0.0026322 0.007443294 962  -0.35363  0.7237
> des_days14  -0.0011319 0.006635512 962  -0.17058  0.8646
> des_days48   0.0112579 0.005452614 962   2.06469  0.0392
> I can clearly use the previous results to compare the estimations of
> each "des_day" to the intercept, using the provided t-statistics.
> Alternatively, I could use post-hoc tests (z-statistics):
>> ph_conditional <- c("des_days1  = 0",
>                       "des_days14  = 0",
>                       "des_days48 = 0");
>> lev.ph<http://lev.ph> <- glht(lev.lm, linfct = ph_conditional);
>> summary(lev.ph<http://lev.ph>)
> Simultaneous Tests for General Linear Hypotheses
> Fit: lme.formula(fixed = data ~ des_days, data = data_red_trf, random
> = ~des_days |
>     ratID, method = "ML", na.action = na.omit, control = lCtr)
> Linear Hypotheses:
>                  Estimate Std. Error z value Pr(>|z|)
> des_days1 == 0  -0.002632   0.007428  -0.354    0.971
> des_days14 == 0 -0.001132   0.006622  -0.171    0.996
> des_days48 == 0  0.011258   0.005441   2.069    0.101
> (Adjusted p values reported -- single-step method)
> The p-values of the coefficient estimates and those of the post-hoc
> tests differ because the latter are adjusted with Bonferroni
> correction. I wonder whether there is any form of correction in the
> coefficient estimated of the LMM, and which p-values are more appropriate to use.
> Thanks
> Cristiano

	[[alternative HTML version deleted]]

More information about the R-sig-mixed-models mailing list