[R-meta] sd of blups vs tau in RE model

[Viechtbauer, Wolfgang (NP)]

Please see below for my responses.

Best,
Wolfgang

>-----Original Message-----
>From: Yefeng Yang [mailto:yefeng.yang1 using unsw.edu.au]
>Sent: Friday, 30 June, 2023 10:42
>To: Viechtbauer, Wolfgang (NP); R Special Interest Group for Meta-Analysis; James
>Pustejovsky
>Subject: Re: [R-meta] sd of blups vs tau in RE model
>
>Dear Wolfgang,
>
>Excellent!  Sorry that I did not realize this post was on the mailing list (the
>title of this post really hard let me relate it to my question - anyway sorry for
>repeating the question)
>
>I am arguing a bit. As pointed out by Wolfgang, the total variance of the
>population can be decomposed into two parts:
>
>tau^2 = var(u_i) = E(var(u_i|y_i)) + var(E(u_i|y_i))
>
>If k is larger enough, the expectation of the variance of the random effects
>should be zero or trivial, assuming the random effects are normally distributed.

But this is not true as demonstrated with the code I provided.

>So, the first part of the above formula E(var(u_i|y_i)) is decreasing to 0 as K -
>> infinite. This is my understanding. I might be silly.

Please try out what happens when you change k to different values. It has no systematic effect on mean(blups$se^2).

>A relevant question is do you think it is meaningful to use the distribution of
>BLUPs to calculate something like "the proportion of the true effects above a
>certain value" to represent the heterogeneity?
>
>Let's use your numerical example to show my point:
>Say we want to know the proportion of the true effects (which are denoted by
>BLUPs) above 0,  we get a proportion of 0.5046695. Can we say the data is
>moderately heterogeneous?
>BLUP = 0
>Z = (BLUP - res$beta[1]) / sqrt(res$tau2)
>1 - pnorm(Z)

I am a bit confused by your question, as you are not using the BLUPs to compute this; you are using the estimated coefficient and value of tau^2. Also, looking at the proportion larger than 0 isn't the best example because this will be ~50% eithre way. So let's compare:

pnorm(0.1, coef(res), sqrt(res$tau2), lower.tail=FALSE)
mean(blups$pred > 0.1)

As you will see, these are not the same. The latter yields a smaller proportion since the distribution of the BLUPs is not as wide as the estimated distribution of true effects based on the model estimates. We can seee this with:

hist(blups$pred, freq=FALSE, breaks=50)
curve(dnorm(x, coef(res), sqrt(res$tau2)), add=TRUE, lwd=3)

>Best,
>Yefeng
>
>________________________________________
>From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Friday, 30 June 2023 17:38
>To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>project.org>; James Pustejovsky <jepusto using gmail.com>
>Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
>Subject: RE: [R-meta] sd of blups vs tau in RE model
>
>Dear Yefeng,
>
>This was actually recently discussed on this mailing list:
>
>https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-May/004627.html
>
>It is NOT true that the variance of the BLUPs will be equal to or approximate
>tau^2 when k is large. As I explain in that post, one can decompose tau^2 into
>two parts by the law of total variance. The variance of the BLUPs is only one
>part of this. To demonstrate:
>
>library(metafor)
>
>tau2 <- .02
>k <- 2500
>vi <- runif(k, .002, .05)
>yi <- rnorm(k, 0, sqrt(vi + tau2))
>
>res <- rma(yi, vi)
>res$tau2
>blups <- ranef(res)
>
># variance of the BLUPs (way too small as this is essentially only
>var(E(u_i|y_i)))
>var(blups$pred)
>
># by adding what is essentially E(var(u_i|y_i)), we get (approximately) tau^2
>var(blups$pred) + mean(blups$se^2)
>
>The larger tau^2 is relative to the sampling variances, the less relevant
>mean(blups$se^2) will be (try running the code above with tau2 <- .2). But still,
>the variance of the BLUPs will underestimate tau^2.
>
>Best,
>Wolfgang
>
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>Behalf Of Yefeng Yang via R-sig-meta-analysis
>>Sent: Friday, 30 June, 2023 6:57
>>To: James Pustejovsky
>>Cc: Yefeng Yang; R Special Interest Group for Meta-Analysis
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>>
>>Exactly. I was meant to the number of studies. I have no idea why I typed
>within-
>>study replicates. Sorry for the confusion.
>>________________________________
>>From: James Pustejovsky <jepusto using gmail.com>
>>Sent: Friday, 30 June 2023 14:41
>>To: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
>>Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>>project.org>
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>>
>>The approximations there are predicated on k (the number of studies) being large
>>enough that the estimated heterogeneity (tau-hat) converges to the true
>>heterogeneity parameter.
>>
>>On Thu, Jun 29, 2023 at 11:29 PM Yefeng Yang
>><yefeng.yang1 using unsw.edu.au<mailto:yefeng.yang1 using unsw.edu.au>> wrote:
>>Hi both,
>>
>>I happen to come across a paper, which can answer both of your comments.
>>
>>Eq. 1 and the following Eqs. show the derivation of the equivalence mentioned by
>>my earlier email.
>>
>>Wang C C, Lee W C. A simple method to estimate prediction intervals and
>>predictive distributions: summarizing meta‐analyses beyond means and confidence
>>intervals[J]. Research Synthesis Methods, 2019, 10(2): 255-266.
>>
>>Best,
>>Yefeng
>>________________________________
>>From: James Pustejovsky <jepusto using gmail.com<mailto:jepusto using gmail.com>>
>>Sent: Friday, 30 June 2023 13:08
>>To: Yefeng Yang <yefeng.yang1 using unsw.edu.au<mailto:yefeng.yang1 using unsw.edu.au>>
>>Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
>>project.org<mailto:r-sig-meta-analysis using r-project.org>>
>>Subject: Re: [R-meta] sd of blups vs tau in RE model
>>
>> Thanks for your clarification. Your explanations are very clear. Actually, the
>>SD of BLUPs and tau will converge when the within-study replicates are getting
>>large.
>>
>>Can you say more about this? Is this claim based on simulations or something? I
>>see the intuition, but it also seems like this property might depend not only on
>>the within-study replicates all being large, but also on their _relative_ sizes.