[R-meta] [+externe Mail+] Re: forest plot and study-specific effect

Tue Jun 18 09:01:49 CEST 2024

Hi Yefeng,

note that the formulas in Sec. 2.8 relate to the *conditional*
distribution of the theta_j parameters (conditional on tau), which is a
normal distribution, characterized by mean and variance. In order to
arrive at the eventual marginal posterior, one still needs to integrate
over the heterogeneity (tau) posterior, yielding a non-normal posterior
distribution. In particular, there is no mean/median/mode plugged in.
If you did that, it may in some cases still be a reasonable
approximation; this is commonly done in an "empirical Bayes" type
analysis. (The "bayesmeta()" function will return the "proper" marginal
posterior for you.)

But your actual point relates to the original model specification (the
normal-normal hierarchical model) from Sec. 2.1 -- of course, one can
think of alternatives here, with more than a single heterogeneity
parameter. The simplest extension may e.g. be a model where you have
two groups of studies estimating the same parameter. Suppose that one
group reports unadjusted estimates, while the other group has adjusted
for some covariable. Then you might expect different amounts of
heterogeneity for the two groups. This could be implemented as special
case of a "location-scale model" as described by Viechtbauer and
Lopesz-Lopez (2022) here: https://doi.org/10.1002/jrsm.1562

This kind of model of course is only really helpful as long as you have
several studies in each group; with few studies per subgroup (or even
single studies in a group, or a separate heterogeneity parameter for
certain individual studies) it gets hard to estimate the heterogeneity,
as you are effectively trying to estimate a variance based on a single
(or few) observation(s).

If you still think the assumption of a single common normal variance
parameter is too restrictive, you may also consider using as model as
implemented in the "metaplus" package (
https://doi.org/10.32614/CRAN.package.metaplus), where the random
effect follows a non-normal distribution. For example, if you think
there are two groups of studies, one more and one less heterogeneous
(but you don't know which study belongs to which group), then using a
two-component normal mixture may do the trick. Or if you think the
(normal) heterogeneity variance may not be identical for all studies,
but may itself be somewhat heterogeneous, this may marginally lead to a
Student-t distributed random effect, where the degrees-of-freedom
parameter regulates the variability of the heterogeneity parameters. I
would however expect that you need quite a lot of data (i.e., studies)
to estimate these more complex models with reasonable precision.
However, if you think these models describe reality more accurately
this may be the way to go; and a Bayesian approach with informative
priors might help in case of fewer studies.

Cheers,

Christian

On Thu, 2024-06-06 at 08:14 +0000, Yefeng Yang wrote:
> Dear Christian,
> 
> I had a read of your introductive paper about bayesmeta.
> 
> A quick question to ask. Under the subsection 2.8. Shrinkage
> estimates of study-specific means, the formula for computing study-
> specific effects is presented. I am wondering whether it makes sense
> to plug-in tau^2_i (tau^2 for each study) rather than the
> mean/median/mode to the formula, given that each study might not have
> the same heterogeneity.
> 
> 
> All the best,
> Yefeng
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
> on behalf of Röver, Christian via R-sig-meta-analysis <
> r-sig-meta-analysis using r-project.org>
> Sent: 06 June 2024 16:25
> To: r-sig-meta-analysis using r-project.org <
> r-sig-meta-analysis using r-project.org>
> Cc: Röver, Christian <christian.roever using med.uni-goettingen.de>
> Subject: Re: [R-meta] [+externe Mail+] Re: forest plot and study-
> specific effect
>  
> Dear Yefeng,
> 
> the forest plots in the "bayesmeta" package by default show the
> shrinkage estimates / BLUPs along with the plain data; see e.g. here:
>   https://cran.r-project.org/package=bayesmeta
>   https://doi.org/10.18637/jss.v093.i06  (Fig.2)
> (bayesmeta's forest plots are based on the "forestplot" package).
> 
> I often find this helpful to illustrate the amount of borrowing-of-
> information / mutual support between studies, depending on the amount
> of heterogeneity. It may in particular also be an interesting option
> in
> cases where the "plain data estimates" may not be readily available,
> e.g., when you are looking at a binomial likelihood and some studies
> have zero event counts.
> 
> Cheers,
> 
> Christian
> 
> 
> On Wed, 2024-06-05 at 16:54 +0000, Dr. Gerta Rücker via R-sig-meta-
> analysis wrote:
> > Dear Yefeng,
> > 
> > You may like to have a look at the supplement " traceplot-R-code.R"
> > to the paper 
> > https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1693 (Röver,
> > Rindskopf, Friede) that shows how to obtain a joint forest plot
> with
> > study-specific estimates and BLUPs with R package bayesmeta.
> > 
> > Best,
> > Gerta
> > 
> > 
> > 
> > UNIVERSITÄTSKLINIKUM FREIBURG
> > Institute for Medical Biometry and Statistics
> > 
> > Dr. Gerta Rücker
> > Guest Scientist
> > 
> > Stefan-Meier-Straße 26 · 79104 Freiburg
> > gerta.ruecker using uniklinik-freiburg.de
> > 
> > 
> https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker
> > 
> > -----Ursprüngliche Nachricht-----
> > Von: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org
> >
> > Im Auftrag von Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
> > Gesendet: Mittwoch, 5. Juni 2024 14:19
> > An: R Special Interest Group for Meta-Analysis <
> > r-sig-meta-analysis using r-project.org>
> > Cc: Viechtbauer, Wolfgang (NP) <
> > wolfgang.viechtbauer using maastrichtuniversity.nl>
> > Betreff: Re: [R-meta] forest plot and study-specific effect
> > 
> > Dear Yefeng,
> > 
> > Plotting the BLUPs has been done before. See Figure 3 in:
> > 
> > van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002).
> Advanced
> > methods in meta-analysis: Multivariate approach and meta-
> regression.
> > Statistics in Medicine, 21(4), 589-624.
> > 
> > Actually, the forest plot shows the individual estimates, plus the
> > BLUPs.
> > 
> > You can find a recreation of this here:
> > 
> > 
> https://www.metafor-project.org/doku.php/analyses:vanhouwelingen2002
> > 
> > I think there are various reasons why the default is not to show
> the
> > BLUPs. For example, the estimates are simply what was found in each
> > of the invididual studies, while the BLUPs depend on what other
> > studies are included in the analysis and the values also depend on
> > the specifics of the modeling approach used. But showing both (as
> > above) is definitely interesting.
> > 
> > Best,
> > Wolfgang
> > 
> > > -----Original Message-----
> > > From: R-sig-meta-analysis <
> > > r-sig-meta-analysis-bounces using r-project.org> On Behalf
> > > Of Yefeng Yang via R-sig-meta-analysis
> > > Sent: Wednesday, June 5, 2024 08:14
> > > To: r-sig-meta-analysis using r-project.org
> > > Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> > > Subject: [R-meta] forest plot and study-specific effect
> > > 
> > > Dear community,
> > > 
> > > I have a small question about the forest plot used in the meta-
> > > analysis.
> > > 
> > > The forest plots and their varieties are often used in meta-
> > > analysis papers to
> > > show the overall/grand mean, individual effect size estimates,
> and
> > > other
> > > relevant info depending on the software making them. We know the
> > > effect size
> > > estimates from individual studies are usually noisy or not very
> > > precise. But,
> > > why do meta-analysts prefer to report individual effect size
> > > estimates rather
> > > than the study-specific effects (which benefit from the shrinkage
> > > or borrowing
> > > of strength). Or, put differently, the developers of software
> that
> > > can make
> > > forest plots do not seem to provide the option of showing study-
> > > specific effects
> > > (I did not check carefully; some software or packages might
> provide
> > > this
> > > functionality). Is there any specific reason? Or, it is just a
> > > convention.
> > > 
> > > Best,
> > > Yefeng
> > 
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ 
> R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> > 
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ 
> R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis