[R-meta] Question on funnel plot interpretation
Gabriel Cotlier
g@b|k|m01 @end|ng |rom gm@||@com
Wed Jul 5 13:10:30 CEST 2023
Thanks a lot Yefeng
Regards,
Gabriel
On Wed, Jul 5, 2023 at 2:03 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au> wrote:
> If you are doing an Egger's test using rma.mv, use SE of Zr estimates as
> the predictor and look at the slope's estimate and corresponding test.
> Also, add other important predictors that might cause variations.
>
> Best,
> Yefeng
> ------------------------------
> *From:* Gabriel Cotlier <gabiklm01 using gmail.com>
> *Sent:* Wednesday, 5 July 2023 20:35
> *To:* Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> *Cc:* R Special Interest Group for Meta-Analysis <
> r-sig-meta-analysis using r-project.org>; Michael Dewey <lists using dewey.myzen.co.uk
> >
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>
> Dear Michael and Yefeng ,
>
> Thank you very much for the interesting observations and orientation
> provided.
>
> - Regarding the strangeness of primary studies.
> Yes indeed, it is something I noticed before that some studies have a very
> low value or almost no correlation (0.001) while others have a very high
> value almost maximum possible (close to 1).
> All correlations included (n = 149) are the result of Fisher's z-to-r
> transformation, and original values of r also in some cases present such
> extreme values.
> The primary dataset of correlations were in some cases given in the
> screened studies, but for the vast majority of the correlations were
> calculated by myself as Pearsons's product-moment correlation employing the
> values reported by the different studies. These correlations (r) were later
> transformed to z by means of Fisher's *r-to-z* transform.
> Studies are very diverse, coming from different geographies, with varied
> types of treatments and applying different methods.
> I do not know the reason for such variability of the range of the
> correlations, but it would be interesting to have a test or quantitative
> way to give account of such variation in the range.
>
> - Regarding the the subjectivity of the interpretation of the funnel plot
> and that I cannot use the function regtest()since is I am using rma.mv()
> object, I also run numerical test for publication bias employing
> different predictors:
>
> 1. sampling variance
> 2. inverse of sampling variance
> 3. standard error
>
> However since for each of the cases/predictors used (see below) I got a
> full model result, I assume --may be wrongly--that the value I should take
> as the numerical estimation of the publication bised is the "intercept" of
> the model, is this correct?
>
> Is there a given range that might serve as a proxy indicator of potential publication
> bias?
>
> Code and model's results below.
>
> Thanks a lot.
> Kind regards,
> Gabriel
>
>
> ## NUMERICAL TEST FOR PUBLICATION BIAS
>
> ## extending Egger's test to more complex models.
> ## "regression test for funnel plot asymmetry".
>
> ## 1. using : the sampling variance
> PubB<-rma.mv(yi = yi,
> V = vi,
> mods = vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8908 0.9438 72 no Article
> # sigma^2.2 2.1970 1.4822 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24617.3110, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4381, p-val = 0.2304
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.5620 0.2584 2.1743 0.0297 0.0554 1.0685 *
> # mods -6.7997 5.6701 -1.1992 0.2304 -17.9130 4.3135
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
> ## 2. using : the inverse of the sampling variance
> PubB_1<-rma.mv(yi = yi,
> V = vi,
> mods = 1/vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_1
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.9176 0.9579 72 no Article
> # sigma^2.2 2.1797 1.4764 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 23656.2997, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4469, p-val = 0.2290
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.0957 0.2595 0.3689 0.7122 -0.4129 0.6044
> # mods 0.0040 0.0034 1.2029 0.2290 -0.0025 0.0106
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> ## 3. using : standard errors (square-root of the sampling variances)
>
> PubB_2<-rma.mv(yi = yi,
> V = vi,
> mods = sqrt(vi),
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_2
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8991 0.9482 72 no Article
> # sigma^2.2 2.1952 1.4816 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24489.6313, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.2528, p-val = 0.2630
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.7801 0.4374 1.7834 0.0745 -0.0772 1.6375 .
> # mods -2.6473 2.3652 -1.1193 0.2630 -7.2829 1.9883
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
>
> On Wed, Jul 5, 2023 at 12:45 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> wrote:
>
> Dear Gabriel
>
> Apart from Michael observation on data checking before analyses (which is
> always a good practice), I add one.
> The funnel plot is just a visual check of publication bias. So the
> observations based on the funnel plots are inevitably subjective - I mean
> you think that is "randomly scattering", while others might think not. In
> contrast, Egger's test is a more objective way to test the asymmetry of a
> funnel plot. Regarding how to do it, you can try to find them in the
> archives associated with this mailing list.
>
> Please be noted that whether none funnel plot and Egger's test can
> indicate publication bias directly. But it is common to assume the
> asymmetry of a funnel plot is caused by publication bias (or more
> precisely. small study effects), after accounting for heterogeneity.
>
> Best,
> Yefeng
> ------------------------------
> *From:* R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
> on behalf of Michael Dewey via R-sig-meta-analysis <
> r-sig-meta-analysis using r-project.org>
> *Sent:* Wednesday, 5 July 2023 19:31
> *To:* R Special Interest Group for Meta-Analysis <
> r-sig-meta-analysis using r-project.org>
> *Cc:* Michael Dewey <lists using dewey.myzen.co.uk>; Gabriel Cotlier <
> gabiklm01 using gmail.com>
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>
> Dear Gabriel
>
> My interpretation looking at your plots is that you have a very strange
> set of primary studies. If the x-axis is really the z transformation of
> r then some of the r are .999 and some 0.001 which seems worthy of
> investigation before looking further.
>
> Michael
>
> On 05/07/2023 09:14, Gabriel Cotlier via R-sig-meta-analysis wrote:
> > Hello all,
> >
> > I have produced a funnel plot on the basis of an rma.mv
> > <http://rma.mv>() objectapplied to all the data set together ( not
> > subsetting using moderators ) as follows:
> >
> > image.png
> >
> >
> > When looking at the figure I tried to think that maybe one of the
> > following two interpretations could be the correct one:
> >
> > a. There is a kind of random scattering of the effect sizes, therefore
> > no symmetry is found and thus publication bised is observed.
> > b. Given the randomness of the effect sizes distribution covering the
> > plot's space unevenly there is not a clear pattern that can indicate
> > publication bias is observed.
> >
> > Is any of this interpretation the correct one?
> >
> > Thanks a lot.
> > Kind regards,
> > Gabriel
> >
> >
> > #### CODE. ######
> > funnel_all <- rma.mv <http://rma.mv>(yi,
> > vi,
> > random = ~ 1 | Article / Sample_ID,
> > data=dat)
> > png(file = "funnel.png",
> > width = 250,
> > height = 200,
> > res = 600,
> > units = "mm")
> > # par(mfrow = c(2, 1))
> >
> > # full data
> > f1 <- funnel(funnel_all,
> > yaxis = "seinv",
> > level = c(90, 95, 99),
> > ylim = c(1, 20),
> > shade = c("white", "gray55", "gray75"),
> > refline = 0,
> > legend = TRUE)
> > mtext("A", side = 3, line = 0, adj = -0.13, cex = 2)
> >
> > dev.off()
> >
> > <
> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> Virus-free.www.avg.com <
> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient
> >
> >
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list