[R-meta] Question on funnel plot interpretation
Michael Dewey
||@t@ @end|ng |rom dewey@myzen@co@uk
Wed Jul 5 14:09:37 CEST 2023
Dear Gabriel
Before you continue searching for small study effects I think a search
for possible moderators of the effects would be more worthwhile. If you
believe that the purpose of meta-analysis is to synthesise then the
spread of values you have calls that goal into question. If you believ
that the purpose of meta-analysis is to understand why effects differ
then searching for possible moderators must be more helpful than looking
for small study efects.
Michael
On 05/07/2023 12:31, Gabriel Cotlier wrote:
>
> Ok, so in my case would it be better to use for instance either inverse
> n or alternatively variance, is this correct?
>
> Regarding the high variability in the range of my Fisher's z transformed
> values you noticed before.
> This variability could be due to the fact that the values come
> from different geographies, use of varied treatments and
> different methods as well?
> Is there a quantitative measure or way that can help to give account of
> such variability or explain it?
>
> Thanks a lot again for your advice and guidance.
> Kind regards,
> Gabriel
>
> On Wed, Jul 5, 2023 at 2:20 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au
> <mailto:yefeng.yang1 using unsw.edu.au>> wrote:
>
> There is a misunderstanding. Fishers Zr's sampling variance (or
> error) is not correlated with Zr. Correlation coefficent r is
> correlated with is variance or error. So, if you transformed r to
> Zr, as you said in early email, variance should be a better measure
> of precision contrast to inverse n.
>
> Best,
> Yefeng
> ------------------------------------------------------------------------
> *From:* Gabriel Cotlier <gabiklm01 using gmail.com
> <mailto:gabiklm01 using gmail.com>>
> *Sent:* Wednesday, 5 July 2023 21:10
> *To:* Yefeng Yang <yefeng.yang1 using unsw.edu.au
> <mailto:yefeng.yang1 using unsw.edu.au>>
> *Cc:* R Special Interest Group for Meta-Analysis
> <r-sig-meta-analysis using r-project.org
> <mailto:r-sig-meta-analysis using r-project.org>>; Michael Dewey
> <lists using dewey.myzen.co.uk <mailto:lists using dewey.myzen.co.uk>>
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
> Thanks a lot Yefeng
> Regards,
> Gabriel
>
>
> On Wed, Jul 5, 2023 at 2:03 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au
> <mailto:yefeng.yang1 using unsw.edu.au>> wrote:
>
> If you are doing an Egger's test using rma.mv <http://rma.mv>,
> use SE of Zr estimates as the predictor and look at the slope's
> estimate and corresponding test. Also, add other important
> predictors that might cause variations.
>
> Best,
> Yefeng
> ------------------------------------------------------------------------
> *From:* Gabriel Cotlier <gabiklm01 using gmail.com
> <mailto:gabiklm01 using gmail.com>>
> *Sent:* Wednesday, 5 July 2023 20:35
> *To:* Yefeng Yang <yefeng.yang1 using unsw.edu.au
> <mailto:yefeng.yang1 using unsw.edu.au>>
> *Cc:* R Special Interest Group for Meta-Analysis
> <r-sig-meta-analysis using r-project.org
> <mailto:r-sig-meta-analysis using r-project.org>>; Michael Dewey
> <lists using dewey.myzen.co.uk <mailto:lists using dewey.myzen.co.uk>>
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
> Dear Michael and Yefeng ,
>
> Thank you very much for the interesting observations and
> orientation provided.
>
> - Regarding the strangeness of primary studies.
> Yes indeed, it is something I noticed before that some studies
> have a very low value or almost no correlation (0.001) while
> others have a very high value almost maximum possible (close to 1).
> All correlations included (n = 149) are the result of
> Fisher's z-to-r transformation, and original values of r also in
> some cases present such extreme values.
> The primary dataset of correlations were in some cases given in
> the screened studies, but for the vast majority of
> the correlations were calculated by myself as Pearsons's
> product-moment correlation employing the values reported by the
> different studies. These correlations (r) were later transformed
> to z by means of Fisher's /r-to-z/ transform.
> Studies are very diverse, coming from different geographies,
> with varied types of treatments and applying different methods.
> I do not know the reason for such variability of the range of
> the correlations, but it would be interesting to have a test or
> quantitative way to give account of such variation in the range.
>
> - Regarding the the subjectivity of the interpretation of the
> funnel plot and that I cannot use the function regtest()since is
> I am using rma.mv <http://rma.mv>() object, I also run numerical
> test for publication bias employing different predictors:
>
> 1. sampling variance
> 2. inverse of sampling variance
> 3. standard error
>
> However since for each of the cases/predictors used (see below)
> I got a full model result, I assume --may be wrongly--that the
> value I should take as the numerical estimation of the
> publication bised is the "intercept" of the model, is this correct?
> Is there a given range that might serve as a proxy indicator of
> potential publication bias?
>
> Code and model's results below.
>
> Thanks a lot.
> Kind regards,
> Gabriel
>
>
> ## NUMERICAL TEST FOR PUBLICATION BIAS
>
> ## extending Egger's test to more complex models.
> ## "regression test for funnel plot asymmetry".
>
> ## 1. using : the sampling variance
> PubB<-rma.mv <http://rma.mv>(yi = yi,
> V = vi,
> mods = vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8908 0.9438 72 no Article
> # sigma^2.2 2.1970 1.4822 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24617.3110, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4381, p-val = 0.2304
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb
> <http://ci.lb> ci.ub
> # intrcpt 0.5620 0.2584 2.1743 0.0297 0.0554 1.0685 *
> # mods -6.7997 5.6701 -1.1992 0.2304 -17.9130 4.3135
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
> ## 2. using : the inverse of the sampling variance
> PubB_1<-rma.mv <http://rma.mv>(yi = yi,
> V = vi,
> mods = 1/vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_1
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.9176 0.9579 72 no Article
> # sigma^2.2 2.1797 1.4764 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 23656.2997, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4469, p-val = 0.2290
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb <http://ci.lb>
> ci.ub
> # intrcpt 0.0957 0.2595 0.3689 0.7122 -0.4129 0.6044
> # mods 0.0040 0.0034 1.2029 0.2290 -0.0025 0.0106
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> ## 3. using : standard errors (square-root of the sampling
> variances)
>
> PubB_2<-rma.mv <http://rma.mv>(yi = yi,
> V = vi,
> mods = sqrt(vi),
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_2
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8991 0.9482 72 no Article
> # sigma^2.2 2.1952 1.4816 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24489.6313, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.2528, p-val = 0.2630
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb
> <http://ci.lb> ci.ub
> # intrcpt 0.7801 0.4374 1.7834 0.0745 -0.0772 1.6375 .
> # mods -2.6473 2.3652 -1.1193 0.2630 -7.2829 1.9883
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
> On Wed, Jul 5, 2023 at 12:45 PM Yefeng Yang
> <yefeng.yang1 using unsw.edu.au <mailto:yefeng.yang1 using unsw.edu.au>> wrote:
>
> Dear Gabriel
>
> Apart from Michael observation on data checking before
> analyses (which is always a good practice), I add one.
> The funnel plot is just a visual check of publication bias.
> So the observations based on the funnel plots are inevitably
> subjective - I mean you think that is "randomly scattering",
> while others might think not. In contrast, Egger's test is a
> more objective way to test the asymmetry of a funnel plot.
> Regarding how to do it, you can try to find them in the
> archives associated with this mailing list.
>
> Please be noted that whether none funnel plot and Egger's
> test can indicate publication bias directly. But it is
> common to assume the asymmetry of a funnel plot is caused by
> publication bias (or more precisely. small study effects),
> after accounting for heterogeneity.
>
> Best,
> Yefeng
> ------------------------------------------------------------------------
> *From:* R-sig-meta-analysis
> <r-sig-meta-analysis-bounces using r-project.org
> <mailto:r-sig-meta-analysis-bounces using r-project.org>> on
> behalf of Michael Dewey via R-sig-meta-analysis
> <r-sig-meta-analysis using r-project.org
> <mailto:r-sig-meta-analysis using r-project.org>>
> *Sent:* Wednesday, 5 July 2023 19:31
> *To:* R Special Interest Group for Meta-Analysis
> <r-sig-meta-analysis using r-project.org
> <mailto:r-sig-meta-analysis using r-project.org>>
> *Cc:* Michael Dewey <lists using dewey.myzen.co.uk
> <mailto:lists using dewey.myzen.co.uk>>; Gabriel Cotlier
> <gabiklm01 using gmail.com <mailto:gabiklm01 using gmail.com>>
> *Subject:* Re: [R-meta] Question on funnel plot interpretation
> Dear Gabriel
>
> My interpretation looking at your plots is that you have a
> very strange
> set of primary studies. If the x-axis is really the z
> transformation of
> r then some of the r are .999 and some 0.001 which seems
> worthy of
> investigation before looking further.
>
> Michael
>
> On 05/07/2023 09:14, Gabriel Cotlier via R-sig-meta-analysis
> wrote:
> > Hello all,
> >
> > I have produced a funnel plot on the basis of an rma.mv <http://rma.mv>
> > <http://rma.mv>() objectapplied to all the data set together
> ( not
> > subsetting using moderators ) as follows:
> >
> > image.png
> >
> >
> > When looking at the figure I tried to think that maybe one of the
> > following two interpretations could be the correct one:
> >
> > a. There is a kind of random scattering of the effect sizes, therefore
> > no symmetry is found and thus publication bised is observed.
> > b. Given the randomness of the effect sizes distribution covering the
> > plot's space unevenly there is not a clear pattern that can indicate
> > publication bias is observed.
> >
> > Is any of this interpretation the correct one?
> >
> > Thanks a lot.
> > Kind regards,
> > Gabriel
> >
> >
> > #### CODE. ######
> > funnel_all <- rma.mv <http://rma.mv> <http://rma.mv <http://rma.mv>>(yi,
> > vi,
> > random = ~ 1 | Article / Sample_ID,
> > data=dat)
> > png(file = "funnel.png",
> > width = 250,
> > height = 200,
> > res = 600,
> > units = "mm")
> > # par(mfrow = c(2, 1))
> >
> > # full data
> > f1 <- funnel(funnel_all,
> > yaxis = "seinv",
> > level = c(90, 95, 99),
> > ylim = c(1, 20),
> > shade = c("white", "gray55", "gray75"),
> > refline = 0,
> > legend = TRUE)
> > mtext("A", side = 3, line = 0, adj = -0.13, cex = 2)
> >
> > dev.off()
> >
> > <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>> Virus-free.www.avg.com <http://Virus-free.www.avg.com> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>>
> >
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> <mailto:R-sig-meta-analysis using r-project.org>
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> <https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis>
>
> --
> Michael
> http://www.dewey.myzen.co.uk/home.html
> <http://www.dewey.myzen.co.uk/home.html>
>
> _______________________________________________
> R-sig-meta-analysis mailing list @
> R-sig-meta-analysis using r-project.org
> <mailto:R-sig-meta-analysis using r-project.org>
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> <https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis>
>
--
Michael
http://www.dewey.myzen.co.uk/home.html
More information about the R-sig-meta-analysis
mailing list