[R-meta] sd of blups vs tau in RE model

[Reza Norouzian]

-- Yefeng,

Along the same lines, estimates of variance and covariance by the
model are obtained from the unconditional/marginal distribution of the
random effects independent of the observed effect sizes.

BLUPs, on the other hand, represent the mean of random effects
conditional on the observed effect sizes. Thus, if you hand-calculate
a statistic (sd etc.) from the BLUPs, it won't likely match that
estimated by the model.

-- Just curious, James,

In my field, sometimes there is an overlience on hand-calculating
certain statistics from BLUPs when those estimates can be more
accurately estimated by the model. I sometimes think it might be a bit
problematic.

For example, in the following paper (doi:10.1017/S027226312300027X),
authors calculated a split-half reliability by fitting two separate
models; each for one set of items and then correlated the relevant
BLUPs to arrive at their reliability estimate (~.84; relatively good
reliability).

If instead, they fit a single multivariate model, their estimate would
have been ~.47 (relatively low reliability).

Possibly, that could be an issue.


Reza

On Thu, Jun 29, 2023 at 9:04 AM James Pustejovsky via
R-sig-meta-analysis <r-sig-meta-analysis using r-project.org> wrote:
>
> Yes, a known property of empirical Bayes (BLUP) estimates is that their SD
> is not equal to the (estimated) SD of the random effects. See the following
> articles and references therein:
>
> Louis, T. A. (1984). Estimating a population of parameter values using
> Bayes and Empirical Bayes methods. Journal of the American Statistical
> Association, 79, 393–398.
>
> Howard S. Bloom, Stephen W. Raudenbush, Michael J. Weiss & Kristin Porter
> (2016): Using Multisite Experiments to Study Cross-Site Variation in
> Treatment Effects: A Hybrid Approach With Fixed Intercepts and a Random
> Treatment Coefficient, Journal of Research on Educational Effectiveness.
> https://doi.org/10.1080/19345747.2016.1264518
>
> Personally, I don't think this is a big problem because the SD of the BLUPs
> is not the same thing as the SD of the population distribution. If it's a
> concern, you could try putting a kernel density smooth over the BLUPs. The
> resulting density estimate will have larger variance than the BLUPs. (I've
> wondered for a while whether there's a way to choose the smoothing
> bandwidth to make it match the random effects SD, but haven't had time to
> look into it.)
>
> James
>
> On Thu, Jun 29, 2023 at 7:10 AM Yefeng Yang via R-sig-meta-analysis <
> r-sig-meta-analysis using r-project.org> wrote:
>
> > Dear experts,
> >
> >
> > I kindly request your guidance in resolving my matter.
> >
> > To simplify my query, I would like to focus solely on the random-effects
> > model for now. My question pertains to the conceptual equivalence between
> > the tau (standard deviation of the mean/overall effect) and the standard
> > deviation of the study-specific effects within the studies included in a
> > meta-analysis. Some researchers refer to these study-specific effects as
> > BLUPs.
> >
> > To explore this further, I conducted an analysis using a specific dataset
> > in metafor. However, my findings seem to suggest otherwise. I present a
> > reproducible example below for your reference:
> >
> > # calculate es and var
> > dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg,
> > data=dat.bcg)
> >
> > # RE model
> > res <- rma(yi, vi, data=dat)
> >
> > # calculate blups
> > blup_value <- blup(res)
> >
> > # test whether the overall effect from res is equal to the simple mean of
> > blup_value
> > res$beta[1] # -0.7145323
> > mean(blup_value$pred) # -0.7145323
> >
> > We see the two values are equal.
> >
> > # test whether tau from res is equal to the var of blup_value
> > sqrt(res$tau2) # 0.5596815
> > sd(blup_value$pred) # 0.4816293
> >
> > We see the two values are not equal and seem to have a big gap.
> > Any comments and insights?
> >
> > Best,
> > Yefeng
> >
> >
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis