[R-sig-ME] bootstrap confidence interval and p-value for treatment vs control comparisons

[Ben Bolker]

   I think the "trt.vs.ctrl" part will be relatively easy, because this 
is what you get from the default treatment contrasts in R anyway (that 
is, if `landuse` is a factor, then the beta (fixed-effect) parameters 
corresponding to `landuse` will be equal to the contrasts between each 
level of landuse and the reference level.

   So

bb <- bootMer(fitted_model,
    FUN = function(x) {
       ff <- fixef(f)
       ff[startsWith(names(ff), "landuse")]
    },
    nsim = 1000  ## or whatever
}

and confint(bb) will give you the confidence intervals.
bb$t will give you the ensemble of bootstrap values

You could get the two-tailed p-values something like this:

boot_pval <- function(x) { 2*min(mean(x<0), mean(x>0)) }
apply(bb$t, 2, boot_pval)

   However, I think a Dunnett adjustment will be harder.  Based on this 
<https://stats.stackexchange.com/questions/631129/multicomp-package-and-emmeans-package-produce-different-adjust-pvalues-for-dunne>, 
the Dunnett adjustment relies the t distribution (or multivariate t). In 
particular, if you look at 
<https://en.wikipedia.org/wiki/Dunnett%27s_test>, you can see that step 
5 includes knowing the residual degrees of freedom, which are hard to 
define for (G)LMMs ...

   You could choose one of the other methods implemented in base R's 
p.adjust() ...

   If you do want to apply Dunnett's correction on bootstrap replicates, 
you might need to ask on Cross Validated 
(https://stats.stackexchange.com), although I don't know if you'll get a 
satisfactory answer there either ...

   good luck,
    Ben Bolker


On 11/13/24 07:58, Lieke Moereels via R-sig-mixed-models wrote:
> Dear all
> I hope you�re doing well. I posted a similar question earlier but didn�t get a reply and because I�m under a bit of pressure to advance on this matter, I hope it�s fine I ask this modified, shorter question on the same topic.
> If I have a simple GLMM specified as: diversity ~ land use type + (1|location) where land use type has 4 levels (let�s say 1 control and 3 treatments) and location at least 8.
> How could I obtain parametric bootstrap confidence intervals and associated p-values for the difference in mean diversity between each of the treatments and the control, that are corrected for these multiple comparisons by a Dunnett correction?
> (So I'm basically looking for a parametric bootstrapping alternative to emmeans' "confint(contrast(emmeans(fitted_model, ~ landuse), "trt.vs.ctrl", ref = 1))")
> 
> Any help would be greatly appreciated!
> 
> Have a very nice day!
> Lieke
> 
> 	[[alternative HTML version deleted]]
> 
> 
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
* E-mail is sent at my convenience; I don't expect replies outside of 
working hours.