[R-meta] Choice of moderator for Egger's regression test in rma.mv

[Elizabeth Wade]

Dear Dr. Viechtbauer,

Thank you so much for your thorough explanation and quick reply.

Interestingly, I could not get the model to run properly (with results for
the test of moderation) until I created a separate variable representing
the inverse sample sizes (data <- data %>% mutate(inverse_ni = (1/ni)). I
also found I had to drop the "|" from the mods line, so my final model was:

model.egger <- rma.mv(ri.c, vi.c, mods = ~ inverse_ni, random = ~ 1 |
Study_ID/Effect_ID, data = data)

I imagine this is all expected behavior, but I thought I would include
these details here in case others are following or run into the same issue
in the future.

Thank you to you and this community for your support and guidance.

My best wishes,
Betsy Wade

Betsy Wade, MA
Clinical Psychology Doctoral Student
Department of Psychology
University of Pennsylvania


On Thu, May 21, 2020 at 2:23 PM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Betsy,
>
> Unless you know exactly what you are doing, I would not mess with the
> weights in the context of a multilevel model. The (default) weighting
> structure in such models is more complex than just assigning a particular
> weight to each estimate. Such a multilevel model also implies a certain
> degree of covariance between the underlying true effects within studies,
> which results in a weight matrix that is not just diagonal, but also has
> non-zero off-diagonal elements. When estimating the overall mean (or fixed
> effects in general), this ensures that multiple estimates coming from the
> same study are not treated as if they are independent.
>
> So, I would recommend using:
>
> model <- rma.mv(ri.c, vi.c, random = ~ 1 | Study_ID/Effect_ID, data =
> data)
>
> As for your actual question: Since you are meta-analyzing correlations, I
> would not use the sampling variances (or some function thereof) for an
> Egger-type test. There is an inherent correlation between correlations and
> their sampling variances, which can lead to false positives. I would use
> the inverse sample sizes as the predictor, that is:
>
> model <- rma.mv(ri.c, vi.c, mods = ~ I(1/ni), random = ~ 1 |
> Study_ID/Effect_ID, data = data)
>
> where 'ni' is the name of the variable containing the sample sizes.
>
> And with respect to 'mods = ~ <moderator>' vs 'mods = <moderator>': This
> is explained under help(rma) and help(rma.mv). But you can essentially
> ignore the latter and always use the formula way of specifying moderators.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org]
> >On Behalf Of Elizabeth Wade
> >Sent: Thursday, 21 May, 2020 18:34
> >To: r-sig-meta-analysis using r-project.org
> >Subject: [R-meta] Choice of moderator for Egger's regression test in
> rma.mv
> >
> >Dear all,
> >
> >I have fit a three-level model using rma.mv to meta-analyze correlation
> >coefficients corrected for measurement unreliability. Based on this
> >documentation (
> >http://www.metafor-project.org/doku.php/tips:hunter_schmidt_method), I
> have
> >weighted the model using the inverse of the corrected variance.
> >
> >My model is:
> >model <- rma.mv(ri.c, vi.c, W = 1/vi.c, random = ~ 1 |
> >Study_ID/Effect_ID, data = data, method = "REML")
> >
> >Now I am intending to perform Egger's regression test. Based on this
> >explanation (
> >
> https://stats.stackexchange.com/questions/155693/metafor-package-bias-and-
> >sensitivity-diagnostics),
> >I selected the inverse of the variance to use as a moderator in the
> >regression test.
> >
> >So I have:
> ><http://www.metafor-
> >project.org/doku.php/tips:hunter_schmidt_method>egger.model
> ><- rma.mv(ri.c, vi.c, W = 1/vi.c, mods = 1/vi.c, random = ~ 1 |
> >Study_ID/Effect_ID, data = data, method = "REML")
> >
> >As I do this, I wonder whether it is appropriate to use the inverse
> >variance to both weight the model and to perform Egger's test. Will this
> >not detect publication bias, given that I am examining potential bias
> using
> >the same variable with which I weighted my model? Do you recommend a
> >different approach?
> >
> >As an aside, I also wonder why some documentation uses the tilde before
> the
> >moderators are listed (mods = ~age) and some do not (mods = 1/vi.c).
> >
> >Thank you for reading,
> >Betsy Wade
> >
> >Betsy Wade, MA
> >Clinical Psychology Doctoral Student
> >Department of Psychology
> >University of Pennsylvania
>

	[[alternative HTML version deleted]]