[R-meta] sd of blups vs tau in RE model

[Yefeng Yang]

Hi James,

Very much thank you for providing your insights, two papers, and a potential solution. Before carefully looking at the papers you provided, I still have two related questions hope you can help address a bit:


  1.   Would you like to explain a bit "the SD of the BLUPs is not the same thing as the SD of the population distribution". Conceptually, I thought they are the same - both representing the marginal distribution after accounting for sampling error.
  2.  If I use some dispersion-relevant stats derived from the distribution of BLUPs to represent the heterogeneity, is it reasonable?

Best,
Yefeng
________________________________
From: James Pustejovsky <jepusto using gmail.com>
Sent: Friday, 30 June 2023 0:04
To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
Subject: Re: [R-meta] sd of blups vs tau in RE model

Yes, a known property of empirical Bayes (BLUP) estimates is that their SD is not equal to the (estimated) SD of the random effects. See the following articles and references therein:

Louis, T. A. (1984). Estimating a population of parameter values using Bayes and Empirical Bayes methods. Journal of the American Statistical Association, 79, 393–398.

Howard S. Bloom, Stephen W. Raudenbush, Michael J. Weiss & Kristin Porter (2016): Using Multisite Experiments to Study Cross-Site Variation in Treatment Effects: A Hybrid Approach With Fixed Intercepts and a Random Treatment Coefficient, Journal of Research on Educational Effectiveness. https://doi.org/10.1080/19345747.2016.1264518

Personally, I don't think this is a big problem because the SD of the BLUPs is not the same thing as the SD of the population distribution. If it's a concern, you could try putting a kernel density smooth over the BLUPs. The resulting density estimate will have larger variance than the BLUPs. (I've wondered for a while whether there's a way to choose the smoothing bandwidth to make it match the random effects SD, but haven't had time to look into it.)

James

On Thu, Jun 29, 2023 at 7:10 AM Yefeng Yang via R-sig-meta-analysis <r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>> wrote:
Dear experts,


I kindly request your guidance in resolving my matter.

To simplify my query, I would like to focus solely on the random-effects model for now. My question pertains to the conceptual equivalence between the tau (standard deviation of the mean/overall effect) and the standard deviation of the study-specific effects within the studies included in a meta-analysis. Some researchers refer to these study-specific effects as BLUPs.

To explore this further, I conducted an analysis using a specific dataset in metafor. However, my findings seem to suggest otherwise. I present a reproducible example below for your reference:

# calculate es and var
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)

# RE model
res <- rma(yi, vi, data=dat)

# calculate blups
blup_value <- blup(res)

# test whether the overall effect from res is equal to the simple mean of blup_value
res$beta[1] # -0.7145323
mean(blup_value$pred) # -0.7145323

We see the two values are equal.

# test whether tau from res is equal to the var of blup_value
sqrt(res$tau2) # 0.5596815
sd(blup_value$pred) # 0.4816293

We see the two values are not equal and seem to have a big gap.
Any comments and insights?

Best,
Yefeng



        [[alternative HTML version deleted]]

_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]