[R-meta] sd of blups vs tau in RE model

Yefeng Yang ye|eng@y@ng1 @end|ng |rom un@w@edu@@u
Fri Jun 30 13:17:47 CEST 2023

Dear Wolfgang,

Good points. I really appreciate your clarification and example.

From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Friday, 30 June 2023 20:20
To: Yefeng Yang <yefeng.yang1 using unsw.edu.au>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-project.org>
Subject: RE: [R-meta] sd of blups vs tau in RE model

Please see below for my responses.


>-----Original Message-----
>From: Yefeng Yang [mailto:yefeng.yang1 using unsw.edu.au]
>Sent: Friday, 30 June, 2023 10:42
>To: Viechtbauer, Wolfgang (NP); R Special Interest Group for Meta-Analysis; James
>Subject: Re: [R-meta] sd of blups vs tau in RE model
>Dear Wolfgang,
>Excellent!  Sorry that I did not realize this post was on the mailing list (the
>title of this post really hard let me relate it to my question - anyway sorry for
>repeating the question)
>I am arguing a bit. As pointed out by Wolfgang, the total variance of the
>population can be decomposed into two parts:
>tau^2 = var(u_i) = E(var(u_i|y_i)) + var(E(u_i|y_i))
>If k is larger enough, the expectation of the variance of the random effects
>should be zero or trivial, assuming the random effects are normally distributed.

But this is not true as demonstrated with the code I provided.

>So, the first part of the above formula E(var(u_i|y_i)) is decreasing to 0 as K -
>> infinite. This is my understanding. I might be silly.

Please try out what happens when you change k to different values. It has no systematic effect on mean(blups$se^2).

>A relevant question is do you think it is meaningful to use the distribution of
>BLUPs to calculate something like "the proportion of the true effects above a
>certain value" to represent the heterogeneity?
>Let's use your numerical example to show my point:
>Say we want to know the proportion of the true effects (which are denoted by
>BLUPs) above 0,  we get a proportion of 0.5046695. Can we say the data is
>moderately heterogeneous?
>BLUP = 0
>Z = (BLUP - res$beta[1]) / sqrt(res$tau2)
>1 - pnorm(Z)

I am a bit confused by your question, as you are not using the BLUPs to compute this; you are using the estimated coefficient and value of tau^2. Also, looking at the proportion larger than 0 isn't the best example because this will be ~50% eithre way. So let's compare:

pnorm(0.1, coef(res), sqrt(res$tau2), lower.tail=FALSE)
mean(blups$pred > 0.1)

As you will see, these are not the same. The latter yields a smaller proportion since the distribution of the BLUPs is not as wide as the estimated distribution of true effects based on the model estimates. We can seee this with:

hist(blups$pred, freq=FALSE, breaks=50)
curve(dnorm(x, coef(res), sqrt(res$tau2)), add=TRUE, lwd=3)

>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Friday, 30 June 2023 17:38
>To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>project.org>; James Pustejovsky <jepusto using gmail.com>
>Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
>Subject: RE: [R-meta] sd of blups vs tau in RE model
>Dear Yefeng,
>This was actually recently discussed on this mailing list:
>It is NOT true that the variance of the BLUPs will be equal to or approximate
>tau^2 when k is large. As I explain in that post, one can decompose tau^2 into
>two parts by the law of total variance. The variance of the BLUPs is only one
>part of this. To demonstrate:
>tau2 <- .02
>k <- 2500
>vi <- runif(k, .002, .05)
>yi <- rnorm(k, 0, sqrt(vi + tau2))
>res <- rma(yi, vi)
>blups <- ranef(res)
># variance of the BLUPs (way too small as this is essentially only
># by adding what is essentially E(var(u_i|y_i)), we get (approximately) tau^2
>var(blups$pred) + mean(blups$se^2)
>The larger tau^2 is relative to the sampling variances, the less relevant
>mean(blups$se^2) will be (try running the code above with tau2 <- .2). But still,
>the variance of the BLUPs will underestimate tau^2.
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>Behalf Of Yefeng Yang via R-sig-meta-analysis
>>Sent: Friday, 30 June, 2023 6:57
>>To: James Pustejovsky
>>Cc: Yefeng Yang; R Special Interest Group for Meta-Analysis
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>>Exactly. I was meant to the number of studies. I have no idea why I typed
>>study replicates. Sorry for the confusion.
>>From: James Pustejovsky <jepusto using gmail.com>
>>Sent: Friday, 30 June 2023 14:41
>>To: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
>>Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>>The approximations there are predicated on k (the number of studies) being large
>>enough that the estimated heterogeneity (tau-hat) converges to the true
>>heterogeneity parameter.
>>On Thu, Jun 29, 2023 at 11:29 PM Yefeng Yang
>><yefeng.yang1 using unsw.edu.au<mailto:yefeng.yang1 using unsw.edu.au>> wrote:
>>Hi both,
>>I happen to come across a paper, which can answer both of your comments.
>>Eq. 1 and the following Eqs. show the derivation of the equivalence mentioned by
>>my earlier email.
>>Wang C C, Lee W C. A simple method to estimate prediction intervals and
>>predictive distributions: summarizing meta‐analyses beyond means and confidence
>>intervals[J]. Research Synthesis Methods, 2019, 10(2): 255-266.
>>From: James Pustejovsky <jepusto using gmail.com<mailto:jepusto using gmail.com>>
>>Sent: Friday, 30 June 2023 13:08
>>To: Yefeng Yang <yefeng.yang1 using unsw.edu.au<mailto:yefeng.yang1 using unsw.edu.au>>
>>Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>>project.org<mailto:r-sig-meta-analysis using r-project.org>>
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>> Thanks for your clarification. Your explanations are very clear. Actually, the
>>SD of BLUPs and tau will converge when the within-study replicates are getting
>>Can you say more about this? Is this claim based on simulations or something? I
>>see the intuition, but it also seems like this property might depend not only on
>>the within-study replicates all being large, but also on their _relative_ sizes.

	[[alternative HTML version deleted]]

More information about the R-sig-meta-analysis mailing list