[R-meta] Multivariate multi-level meta-analysis: adjusting control variables when modeling publication bias?

[Viechtbauer, Wolfgang (NP)]

Hi Daniel,

I don't think you ever got a response to this.

I don't have much to say about this anyway, except that *very rarely* does it ever make sense to run a meta-analysis on test statistics. In any case, the idea of formulating a model like in PET where we use the test statistics as the dependent variables just doesn't carry over to random-effects / multilevel models as I demonstrated in one of my previous posts. Instead, we should use the model formulation where the effect sizes estimates are the dependent variable.

Note that partial correlations are not test statistics.

Best,
Wolfgang

> -----Original Message-----
> From: Daniel Foster <daniel.foster using utoronto.ca>
> Sent: Tuesday, December 5, 2023 21:46
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Subject: Re: [R-meta] Multivariate multi-level meta-analysis: adjusting control
> variables when modeling publication bias?
>
> Hi Wolfgang,
>
> Thank you! Sorry, I am new to Stack Exchange and did not realize that you had
> already provided a response. I deeply appreciate your insights.
>
> I do have a follow up question, if that's okay. I am also carrying two other
> multilevel analyses - one uses the t-statistic as a dependent outcome, and
> another uses the partial correlation coefficient (the computation of which is
> derived from the t-statistic) as the dependent variable.
>
> You mention that the authors divide everything because they are working with
> test statistics, which makes me wonder whether the analyses I just mentioned
> require the same adjustment? However, you go on to say that random-effects
> models and models with a multilevel structure do not require this adjustment. Do
> you mind clarifying?
>
> Thank you again for your time! The mailing list archives are an incredible
> resource!!
>
> I will also post this response on stack exchange, so that it is available to
> others.
>
> Best,
> Daniel
>
> ________________________________________
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> on behalf
> of Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis <r-sig-meta-analysis using r-
> project.org>
> Sent: Tuesday, December 5, 2023 2:46 PM
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>
> Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Subject: Re: [R-meta] Multivariate multi-level meta-analysis: adjusting control
> variables when modeling publication bias?
>
> Hi Daniel,
>
> Didn't you ask the same question here?
>
> https://stats.stackexchange.com/q/633046/1934
>
> Please see my answer there.
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On
> Behalf
> > Of Daniel Foster via R-sig-meta-analysis
> > Sent: Tuesday, December 5, 2023 20:09
> > To: r-sig-meta-analysis using r-project.org
> > Cc: Daniel Foster <daniel.foster567 using gmail.com>
> > Subject: [R-meta] Multivariate multi-level meta-analysis: adjusting control
> > variables when modeling publication bias?
> >
> > Hello Wolfgang and all,
> >
> > I am carrying out multivariate multilevel meta-analysis using the
> > rma.mv function in the metafor package, and I have come across an
> > issue that has been giving me a lot of trouble. At this point I am at
> > a stand still and any insight would be greatly appreciated!!
> >
> > When testing for publication bias using the PET approach, Doucouliagos
> > & Stanley (2009), suggest using the following model in a weighted
> > least squares formula:
> >
> > ES= B1(1/SE) + SIGMA ak(Zj/SEi )+ e
> >
> > Where SE is the standard error of the effect estimate,  Z is a vector
> > of meta-independent variables reflecting differences across studies
> > for the jth study in literature, ak is the meta-regression coefficient
> > which reflects the effect of particular study characteristics.
> >
> > My confusion lies in the fact that they are suggesting that the
> > control variables (Z) included need to be divided by the standard
> > error of the effect estimate. My questions are this:
> >
> > Should I be dividing my control variables by the standard error of
> > the effect estimate when using the rma.mv function?  I have found some
> > multivariate multilevel meta-analyses that follow this method
> > (Klomp, 2009), but then others that don't (at least explicitly;
> > Akgunduz, 2018)
> >
> > If I do need to do this, it is not clear to me how a binary control
> > variable can be incorporated in my mra.mv model (i.e., 1, 0). To my
> > mind, it seems strange to divide a dichotomous variable by a
> > continuous variable. What steps do I need to execute to include these
> > variables in my mra.mv model?
> >
> > Thank you so much in advance for your insights!
> >
> > Daniel
> >
> > Akgunduz, Y. E., & Plantenga, J. (2018). doi: 10.1111/joes.12192
> > Doucouliagos, H., & Stanley, T. D. (2009). doi: 10.1111/j.1467-
> 8543.2009.00723.x
> > Klomp, J., & De Haan, J. (2010). doi: 10.1111/j.1467-6419.2009.00597.x