[R-meta] Question Re: Rma.mv Model

[Viechtbauer, Wolfgang (NP)]

Dear Tori,

I assume variable 'r_oriented' contains the mix of regular correlations, adjusted betas, partial r's, standardized regression coefficients, and unstandardized regression coefficients. If so, then what you are doing is not correct. By feeding this variable into escalc() with measure="COR", the calculations are based on the assumption that variable 'r_oriented' ONLY contains regular Pearson product-moment correlation coefficients, which is not the case here.

Also, unstandardized regression coefficients cannot be mixed at all with correlation coefficients, as they are on entirely different scales. Same goes for standardized regression coefficients (except for the case of simple regression where the standardized regression coefficient is equal to the Pearson correlation).

Best,
Wolfgang

> -----Original Message-----
> From: Tori Peña <Tori.Pena using stonybrook.edu>
> Sent: Tuesday, October 20, 2026 18:37
> To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Subject: Re: [R-meta] Question Re: Rma.mv Model
> 
> Thanks as always!
> 
> I wanted to confirm that I understood your instructions above re: converting
> multiple correlation types into Fisher's z.  I have a dataframe with adjusted
> beta, partial r, rho, beta, and unstandardized b.  Is this delta correction
> correct? Note: r_oriented is the effect size and N_Effect is the sample size --
> 
> meta_df_other <- meta_df %>% subset(Reported_Stat_Type != r (Pearson)")
> meta_df_other <- escalc(measure = "COR", ri=r_oriented, ni=N_Effect, data =
> meta_df_other)
> meta_df_other <- conv.delta(yi, vi, data=meta_df_other, transf=transf.rtoz,
> replace=TRUE)
> 
> On Fri, Sep 25, 2026 at 3:11 AM Viechtbauer, Wolfgang (NP)
> <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> 
> With respect to sensitivity analyses:
> 
> Many sensitivity analyses involve the typical kinds of choices one has to make
> when conducting a meta-analysis (or really any kind of analysis). Gelman and
> Loken used the metaphor of "the garden of forking paths" to describe this idea
> (not in the context of meta-analysis but more broadly):
> 
> Gelman, A., & Loken, E. (2014). The statistical crisis in science. American
> Scientist, 102(6), 460.
> 
> Simmons and colleagues call this "researcher degrees of freedom":
> 
> Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive
> psychology: Undisclosed flexibility in data collection and analysis allows
> presenting anything as significant. Psychological Science, 22(11), 1359-1366.
> 
> Of course, none of this is new. The idea that one has to make many decisions,
> judgments, and choices in collecting and analyzing a set of data is completely
> apparent to anybody who has ever been in this situation. In the meta-analytic
> context, this was discussed already early on; for example:
> 
> Light, R. J. (1980). Synthesis methods: Some judgment calls that must be made.
> Evaluation in Education, 4, 13-17.
> 
> Wanous, J. P., Sullivan, S. E., & Malinak, J. (1989). The role of judgment calls
> in meta-analysis. Journal of Applied Psychology, 74(2), 259-264.
> 
> And more related to the choices around the statistical methods:
> 
> Mengersen, K. L., Tweedie, R. L., & Biggerstaff, B. (1995). The impact of method
> choice on meta-analysis. Australian Journal of Statistics, 37(1), 19-44.
> 
> Describing all these kinds of choices and decision points would take too long
> and some of this is specific to a particular project. But typical choices and
> judgment calls include:
> 
> - eligibility: which studies, outcomes, time points, and populations to include.
> - effect-size definition: which effect measure to use and how to compute it;
> also which effect size to extract when there are multiple reasonable options
> (e.g., adjusted vs unadjusted odds ratio).
> - data handling: dealing with missing information, multiple outcomes, multiple
> groups, and dependent effects.
> - model choice: equal/common-effect vs. random-effects; potentially
> multilevel/multivariate models.
> - estimation: estimator for heterogeneity, inference method,
> confidence/prediction intervals.
> - heterogeneity: how to assess it and which moderators/subgroups to examine via
> meta-regression or subgrouping.
> - study quality: how risk of bias and/or study quality are evaluated and how
> they enter the analysis.
> - Outliers/influential studies: whether and how to identify and handle them.
> - Publication bias: which diagnostics and adjustment/sensitivity methods to use.
> 
> Hence, there isn't a sensitivity() function. Instead, you have to think about
> these choices, their potential impact, and which ones you think need to be
> examined (which is yet another choice - it's INCEPTION all over again). You do
> this by going down the alternative path in the garden to see what happens (e.g.,
> what happens when I DO remove the non-randomized studies? what happens when I
> assume a pre-post correlation of 0.7 or 0.1 instead of 0.4? what happens when I
> exclude this or that study?).
> 
> Best,
> Wolfgang
> 
> > -----Original Message-----
> > From: Tori Peña <mailto:Tori.Pena using stonybrook.edu>
> > Sent: Thursday, September 24, 2026 23:05
> > To: Viechtbauer, Wolfgang (NP)
> <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>
> > Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> > http://project.org>
> > Subject: Re: [R-meta] Question Re: Rma.mv Model
> >
> > This is incredibly helpful!
> >
> > I did not have specific sensitivity analyses in mind.  I'm open to any
> > suggestions based on your expertise.
> >
> > On Wed, Sep 23, 2026 at 11:17 AM Viechtbauer, Wolfgang (NP)
> > <mailto:mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >
> > Dear Tori,
> >
> > Please see my responses below.
> >
> > Best,
> > Wolfgang
> >
> > > -----Original Message-----
> > > From: R-sig-meta-analysis <mailto:mailto:r-sig-meta-analysis-bounces using r-
> project.org>
> > On Behalf
> > > Of Tori Peña via R-sig-meta-analysis
> > > Sent: Wednesday, September 23, 2026 16:38
> > > To: R Special Interest Group for Meta-Analysis
> > > Cc: Tori Peña <mailto:mailto:Tori.Pena using stonybrook.edu>
> > > Subject: [R-meta] Question Re: Rma.mv Model
> > >
> > > Hello everyone -
> > >
> > > I have a couple of questions related to a three-level model we plan to
> > > conduct using *metafor. *For context, we have compiled correlations
> > > indexing relationships between cognition and inflammation in the body. We
> > > plan to nest effect sizes (level 1) in sample cohorts (level 2) within
> > > studies (level 3). I wanted to check the following questions:
> > >
> > >    1. The literature has different types of correlations. Is it acceptable
> > >    to convert all of these correlations to Fisher's *z *for the models?
> > >    Or, should there way to handle these varying effect size types?
> >
> > You can use the standard r-to-z transformation on all kinds of correlation
> > coefficients. However, the sampling variance equation 1/(n-3) only applies
> when
> > converting Pearson product-moment correlations with the r-to-z transformation.
> > For other types of correlation coefficients (Spearman, (point-)biserial,
> > tetrachoric, (semi)partial, and so on), it would not be correct to use 1/(n-3)
> > as the variance.
> >
> > Using the delta method, one can compute the variance of any r-to-z transformed
> > correlation coefficient as long as one has an estimate of the variance of the
> > correlation coefficient to begin with. For example, say you have a study that
> > dichotomized both variables of interest and reports the results in terms of a
> > 2x2 table (how many people scored above/below the cutoff on the first variable
> > versus the second variable). Say this is the 2x2 table:
> >
> >        Above  Below
> > Above  15     5
> > Below   8     18
> >
> > Then we can get the tetrachoric correlation with:
> >
> > dat <- escalc(measure="RTET", ai=15, bi=5, ci=8, di=18)
> > dat
> >
> >       yi     vi
> > 1 0.6397 0.0258
> >
> > (this estimates the correlation between the underlying continuous variables).
> I
> > can now r-to-z transform this with:
> >
> > transf.rtoz(dat$yi)
> > [1] 0.757601
> >
> > But
> >
> > 1/((15+5+8+18)-3)
> > [1] 0.02325581
> >
> > is NOT the sampling variance of the transformed tetrachoric correlation. Using
> > the delta method, we can compute a correct (i.e., consistent) estimate of the
> > sampling variance:
> >
> > conv.delta(yi, vi, data=dat, transf=transf.rtoz,
> > replace=TRUE)
> >
> >       yi     vi
> > 1 0.7576 0.0740
> >
> > Note that the variance (0.0740) is more than 3 times larger than the incorrect
> > estimate (0.0233), because we lost information from the dichotomization of the
> > two variables and having to reconstruct the correlation from that information.
> > You can actually skip the whole conv.delta() thing since escalc() can directly
> > do all of this for you:
> >
> > dat <- escalc(measure="ZTET", ai=15, bi=5, ci=8,
> > di=18)
> > dat
> >
> >       yi     vi
> > 1 0.7576 0.0740
> >
> > The same principle applies to other correlation coefficients. Note that the r-
> > to-z transformation is only a variance stabilizing transformation for Pearson
> > product-moment correlations, not for (most) other types of correlation
> > coefficients. However, using the transformation can still be advantageous,
> since
> > raw correlations are bounded which can result in skewed sampling
> distributions.
> > The r-to-z transformation can then help to normalize the sampling
> distributions.
> >
> > >    2. I could not find information in the *metafor *documentation about
> > >    conducting sensitivity analyses for http://rma.mv objects. Is this
> possible? If
> > >    so, how?
> >
> > What type of sensitivity analyses are you thinking of?
> >
> > > Thanks in advance,
> > > Tori