[R-sig-ME] Your response to my R-sig-ME question

[Ben Bolker]

  [cc'ing r-sig-mixed-models]

  Honestly, it looks to me like you *do* need multiple-comparisons
corrections here. I can't give you detailed advice about how to do it;
emmeans does the pairwise comparisons, but it's not immediately obvious
how to do correction for *multiple* sets of pairwise comparisons.
(Perhaps you could get away with only doing the corrections at the level
of sets of pairwise comparisons.)  As I mentioned before, this is not a
particularly mixed-model-related question.  You could try CrossValidated
(https://stats.stackexchange.com).  The emmeans and multcomp packages
will probably be what you need in terms of machinery.

  sincerely
   Ben Bolker


On 2019-11-01 6:40 a.m., Francesco Romano wrote:
> Dear Ben,
> 
> 
> Apologies for emailing directly about this but there seems to be a
> problem with my subscription settings to the list at the moment. Had it
> not been for Ian Dworkin's message content, I never would have known you
> had also replied.
> 
> Fortunately, I managed to fetch your message from the r-sig-ME
> repository which I copy below:
> 
> "you could "just" use p.adjust(), something like this: library(lme4) gm1
> <- glmer(cbind(incidence, size - incidence) ~ period + (1 | herd), data
> = cbpp, family = binomial) cc <- coef(summary(gm1)) cc <-
> cbind(cc,adjust.p=p.adjust(cc[,"Pr(>|z|)"],"holm")) The general
> machinery in the multcomp package (especially the glht function) should
> work. This looks useful:
> https://thebiobucket.blogspot.com/2011/06/glmm-with-custom-multiple-comparisons.html
> The bottom line is that most standard multiple-comparisons or
> pairwise-comparisons machinery should "just work" with glmer fits.
> (There are some open questions about what you're doing: it's a bit
> unusual for people to apply multiple comparisons corrections on a set of
> "only" 6 parameters specified a priori: Tukey adjustments to post hoc
> pairwise comparisons are much more common ...)"
> 
> To fully understand your advice about using the multcomp package and/or
> tukey adjustments, I need to put this better into perspective for you. I
> have attached the table whose results the editor of the journal is
> requesting multiple comparison corrections for. All the pairwise
> comparisons had no a-priori hypotheses about them. It was simply a
> matter of ascertaining whether there were any significant differences
> between each pair of groups (factor 1) for a given level of complexity
> (factor 2) and vice versa, for each pair of levels of complexity at a
> given level of group. I did this by looking at simple effects and
> re-leveling.
> 
> Honestly, it is the first time someone asks me to apply corrections so I
> am not too sure either. What I would like, though, is to offer a cogent
> response as to why that should not be necessary.
> 
> Any help is much appreciated,
> 
> Frank Romano On 2019-10-31 10:36 a.m., Francesco Romano wrote: >/Dear
> all, />//>/A reviewer has asked me to apply a correction to multiple
> comparisons />/conducted for a logistic mixed effect regression with
> binary outcome. The />/model is: />//>/glmer(outcome ~ factor1 * factor2
> + (1|RE1) + (1|RE2), family =binomial, />/data = data) />//>/where
> factor 1 has two levels and factor 2 has three. Could you advise on
> />/how to run this and how to report the adjusted p-values in the same
> table? />/At the moment, my table has the following 6 headings:
> />//>/Reference level />//>/Contrasts />//>/Estimate />//>/SE />//>/Wald
> *z* />//>/*p* />//>//>/Many thanks in advance, />//>/Frank />//
> 
> 
> 
> Best,
> 
> Frank