[R-meta] Question on funnel plot interpretation
Gabriel Cotlier
g@b|k|m01 @end|ng |rom gm@||@com
Wed Jul 5 13:31:32 CEST 2023
Ok, so in my case would it be better to use for instance either inverse n
or alternatively variance, is this correct?
Regarding the high variability in the range of my Fisher's z transformed
values you noticed before.
This variability could be due to the fact that the values come
from different geographies, use of varied treatments and
different methods as well?
Is there a quantitative measure or way that can help to give account of
such variability or explain it?
Thanks a lot again for your advice and guidance.
Kind regards,
Gabriel
On Wed, Jul 5, 2023 at 2:20 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au> wrote:
> There is a misunderstanding. Fishers Zr's sampling variance (or error) is
> not correlated with Zr. Correlation coefficent r is correlated with is
> variance or error. So, if you transformed r to Zr, as you said in early
> email, variance should be a better measure of precision contrast to inverse
> n.
>
> Best,
> Yefeng
> ------------------------------
> *From:* Gabriel Cotlier <gabiklm01 using gmail.com>
> *Sent:* Wednesday, 5 July 2023 21:10
> *To:* Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> *Cc:* R Special Interest Group for Meta-Analysis <
> r-sig-meta-analysis using r-project.org>; Michael Dewey <lists using dewey.myzen.co.uk
> >
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>
> Thanks a lot Yefeng
> Regards,
> Gabriel
>
>
> On Wed, Jul 5, 2023 at 2:03 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> wrote:
>
> If you are doing an Egger's test using rma.mv, use SE of Zr estimates as
> the predictor and look at the slope's estimate and corresponding test.
> Also, add other important predictors that might cause variations.
>
> Best,
> Yefeng
> ------------------------------
> *From:* Gabriel Cotlier <gabiklm01 using gmail.com>
> *Sent:* Wednesday, 5 July 2023 20:35
> *To:* Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> *Cc:* R Special Interest Group for Meta-Analysis <
> r-sig-meta-analysis using r-project.org>; Michael Dewey <lists using dewey.myzen.co.uk
> >
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>
> Dear Michael and Yefeng ,
>
> Thank you very much for the interesting observations and orientation
> provided.
>
> - Regarding the strangeness of primary studies.
> Yes indeed, it is something I noticed before that some studies have a very
> low value or almost no correlation (0.001) while others have a very high
> value almost maximum possible (close to 1).
> All correlations included (n = 149) are the result of Fisher's z-to-r
> transformation, and original values of r also in some cases present such
> extreme values.
> The primary dataset of correlations were in some cases given in the
> screened studies, but for the vast majority of the correlations were
> calculated by myself as Pearsons's product-moment correlation employing the
> values reported by the different studies. These correlations (r) were later
> transformed to z by means of Fisher's *r-to-z* transform.
> Studies are very diverse, coming from different geographies, with varied
> types of treatments and applying different methods.
> I do not know the reason for such variability of the range of the
> correlations, but it would be interesting to have a test or quantitative
> way to give account of such variation in the range.
>
> - Regarding the the subjectivity of the interpretation of the funnel plot
> and that I cannot use the function regtest()since is I am using rma.mv()
> object, I also run numerical test for publication bias employing
> different predictors:
>
> 1. sampling variance
> 2. inverse of sampling variance
> 3. standard error
>
> However since for each of the cases/predictors used (see below) I got a
> full model result, I assume --may be wrongly--that the value I should take
> as the numerical estimation of the publication bised is the "intercept" of
> the model, is this correct?
>
> Is there a given range that might serve as a proxy indicator of potential publication
> bias?
>
> Code and model's results below.
>
> Thanks a lot.
> Kind regards,
> Gabriel
>
>
> ## NUMERICAL TEST FOR PUBLICATION BIAS
>
> ## extending Egger's test to more complex models.
> ## "regression test for funnel plot asymmetry".
>
> ## 1. using : the sampling variance
> PubB<-rma.mv(yi = yi,
> V = vi,
> mods = vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8908 0.9438 72 no Article
> # sigma^2.2 2.1970 1.4822 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24617.3110, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4381, p-val = 0.2304
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.5620 0.2584 2.1743 0.0297 0.0554 1.0685 *
> # mods -6.7997 5.6701 -1.1992 0.2304 -17.9130 4.3135
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
> ## 2. using : the inverse of the sampling variance
> PubB_1<-rma.mv(yi = yi,
> V = vi,
> mods = 1/vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_1
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.9176 0.9579 72 no Article
> # sigma^2.2 2.1797 1.4764 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 23656.2997, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4469, p-val = 0.2290
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.0957 0.2595 0.3689 0.7122 -0.4129 0.6044
> # mods 0.0040 0.0034 1.2029 0.2290 -0.0025 0.0106
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> ## 3. using : standard errors (square-root of the sampling variances)
>
> PubB_2<-rma.mv(yi = yi,
> V = vi,
> mods = sqrt(vi),
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_2
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8991 0.9482 72 no Article
> # sigma^2.2 2.1952 1.4816 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24489.6313, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.2528, p-val = 0.2630
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.7801 0.4374 1.7834 0.0745 -0.0772 1.6375 .
> # mods -2.6473 2.3652 -1.1193 0.2630 -7.2829 1.9883
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
>
> On Wed, Jul 5, 2023 at 12:45 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> wrote:
>
> Dear Gabriel
>
> Apart from Michael observation on data checking before analyses (which is
> always a good practice), I add one.
> The funnel plot is just a visual check of publication bias. So the
> observations based on the funnel plots are inevitably subjective - I mean
> you think that is "randomly scattering", while others might think not. In
> contrast, Egger's test is a more objective way to test the asymmetry of a
> funnel plot. Regarding how to do it, you can try to find them in the
> archives associated with this mailing list.
>
> Please be noted that whether none funnel plot and Egger's test can
> indicate publication bias directly. But it is common to assume the
> asymmetry of a funnel plot is caused by publication bias (or more
> precisely. small study effects), after accounting for heterogeneity.
>
> Best,
> Yefeng
> ------------------------------
> *From:* R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
> on behalf of Michael Dewey via R-sig-meta-analysis <
> r-sig-meta-analysis using r-project.org>
> *Sent:* Wednesday, 5 July 2023 19:31
> *To:* R Special Interest Group for Meta-Analysis <
> r-sig-meta-analysis using r-project.org>
> *Cc:* Michael Dewey <lists using dewey.myzen.co.uk>; Gabriel Cotlier <
> gabiklm01 using gmail.com>
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>
> Dear Gabriel
>
> My interpretation looking at your plots is that you have a very strange
> set of primary studies. If the x-axis is really the z transformation of
> r then some of the r are .999 and some 0.001 which seems worthy of
> investigation before looking further.
>
> Michael
>
> On 05/07/2023 09:14, Gabriel Cotlier via R-sig-meta-analysis wrote:
> > Hello all,
> >
> > I have produced a funnel plot on the basis of an rma.mv
> > <http://rma.mv>() objectapplied to all the data set together ( not
> > subsetting using moderators ) as follows:
> >
> > image.png
> >
> >
> > When looking at the figure I tried to think that maybe one of the
> > following two interpretations could be the correct one:
> >
> > a. There is a kind of random scattering of the effect sizes, therefore
> > no symmetry is found and thus publication bised is observed.
> > b. Given the randomness of the effect sizes distribution covering the
> > plot's space unevenly there is not a clear pattern that can indicate
> > publication bias is observed.
> >
> > Is any of this interpretation the correct one?
> >
> > Thanks a lot.
> > Kind regards,
> > Gabriel
> >
> >
> > #### CODE. ######
> > funnel_all <- rma.mv <http://rma.mv>(yi,
> > vi,
> > random = ~ 1 | Article / Sample_ID,
> > data=dat)
> > png(file = "funnel.png",
> > width = 250,
> > height = 200,
> > res = 600,
> > units = "mm")
> > # par(mfrow = c(2, 1))
> >
> > # full data
> > f1 <- funnel(funnel_all,
> > yaxis = "seinv",
> > level = c(90, 95, 99),
> > ylim = c(1, 20),
> > shade = c("white", "gray55", "gray75"),
> > refline = 0,
> > legend = TRUE)
> > mtext("A", side = 3, line = 0, adj = -0.13, cex = 2)
> >
> > dev.off()
> >
> > <
> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
> Virus-free.www.avg.com <
> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient
> >
> >
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
>
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list