[R-sig-ME] Confidence interval around random effect variances in place of p-value

[Jack Solomon]

Thank you all very much. So, I can conclude that a likelihood ratio test
and/or a parametric bootstrapping can be used for random effect variance
component hypothesis testing.

But I also concluded that the idea of simply using a bootstrapped CI for a
random-effect variance component [e.g., in lme4;
confint(model,method="boot",oldNames=FALSE)  ] by definition can't be used
for significance testing, because it requires the possibility of seeing sd
= 0 which can't be "strictly" captured by such a CI from a multilevel model
(at least not easily so).

I hope my conclusions are correct,
Thank you all, Jack

On Fri, Apr 2, 2021 at 6:51 PM Ben Bolker <bbolker using gmail.com> wrote:

>   Sure. If all you want is p-values, I'd recommend parametric
> bootstrapping (implemented in the pbkrtest package) ... that will avoid
> these difficulties.  (I would also make sure that you know *why* you
> want p-values on the random effects ... they have all of the issues of
> regular p-values plus some extras:
>
> http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#testing-significance-of-random-effects
> )
>
> On 4/2/21 7:37 PM, Jack Solomon wrote:
> > Thanks. Just to make sure, to declare a statistically NON-significant
> > random effect variance component, the lower bound of the CI must be
> > EXACTLY "0", right?
> >
> > Tha is, for example, a CI like: [.0002, .14] is a
> > statistically significant random-effect variance component but one that
> > perhaps borders a p-value of relatively close to but smaller than .05,
> > right?
> >
> > On Fri, Apr 2, 2021 at 6:19 PM Ben Bolker <bbolker using gmail.com
> > <mailto:bbolker using gmail.com>> wrote:
> >
> >         This seems like a potential can of worms (as indeed are all
> >     hypothesis tests of null values on a boundary ...) However, in this
> >     case
> >     bootstrapping (provided you have resampled appropriately - you may
> need
> >     to do hierarchical bootstrapping ...) seems reasonable, because a
> null
> >     model would give you singular fits (i.e. estimated sd=0) half of the
> >     time ...
> >
> >         Happy to hear more informed opinions.
> >
> >     On 4/2/21 6:55 PM, Jack Solomon wrote:
> >      > Dear All,
> >      >
> >      > A colleague of mine suggested that I use the bootstrapped CIs
> >     around my
> >      > model's random effect variances in place of p-values for them.
> >      >
> >      > But random effect variances (or sds) start from "0". So, to
> declare a
> >      > statistically NON-significant random effect variance component,
> the
> >      > lower bound of the CI must be EXACTLY "0", right?
> >      >
> >      > Thank you very much,
> >      > Jack
> >      >
> >      >       [[alternative HTML version deleted]]
> >      >
> >      > _______________________________________________
> >      > R-sig-mixed-models using r-project.org
> >     <mailto:R-sig-mixed-models using r-project.org> mailing list
> >      > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >     <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
> >      >
> >
> >     _______________________________________________
> >     R-sig-mixed-models using r-project.org
> >     <mailto:R-sig-mixed-models using r-project.org> mailing list
> >     https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >     <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>
> >
>

	[[alternative HTML version deleted]]