[R-meta] Question on funnel plot interpretation
Gabriel Cotlier
g@b|k|m01 @end|ng |rom gm@||@com
Wed Jul 5 12:45:08 CEST 2023
Dear all,
Now that I see Wolfgang post at :
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2020-May/002086.html
since I am using correlations I should use sample size instead of sampling
variance. If I am not mistaken that would be something like in the code and
results below.
Regards,
Gabriel
## code and results
PubB_4<-rma.mv(yi = yi,
V = vi,
mods = ~ I(1/ni),
random = ~ 1 | Article / Sample_ID,
data = dat,
method = "REML")
PubB_4
Multivariate Meta-Analysis Model (k = 149; method: REML)
Variance Components:
estim sqrt nlvls fixed factor
sigma^2.1 0.8940 0.9455 72 no Article
sigma^2.2 2.1967 1.4821 149 no Article/Sample_ID
Test for Residual Heterogeneity:
QE(df = 147) = 24601.0926, p-val < .0001
Test of Moderators (coefficient 2):
QM(df = 1) = 1.3476, p-val = 0.2457
Model Results:
estimate se zval pval ci.lb ci.ub
intrcpt 0.5927 0.2840 2.0868 0.0369 0.0360 1.1494 *
I(1/ni) -9.0062 7.7582 -1.1609 0.2457 -24.2119 6.1995
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
On Wed, Jul 5, 2023 at 1:35 PM Gabriel Cotlier <gabiklm01 using gmail.com> wrote:
> Dear Michael and Yefeng ,
>
> Thank you very much for the interesting observations and orientation
> provided.
>
> - Regarding the strangeness of primary studies.
> Yes indeed, it is something I noticed before that some studies have a very
> low value or almost no correlation (0.001) while others have a very high
> value almost maximum possible (close to 1).
> All correlations included (n = 149) are the result of Fisher's z-to-r
> transformation, and original values of r also in some cases present such
> extreme values.
> The primary dataset of correlations were in some cases given in the
> screened studies, but for the vast majority of the correlations were
> calculated by myself as Pearsons's product-moment correlation employing the
> values reported by the different studies. These correlations (r) were later
> transformed to z by means of Fisher's *r-to-z* transform.
> Studies are very diverse, coming from different geographies, with varied
> types of treatments and applying different methods.
> I do not know the reason for such variability of the range of the
> correlations, but it would be interesting to have a test or quantitative
> way to give account of such variation in the range.
>
> - Regarding the the subjectivity of the interpretation of the funnel plot
> and that I cannot use the function regtest()since is I am using rma.mv()
> object, I also run numerical test for publication bias employing
> different predictors:
>
> 1. sampling variance
> 2. inverse of sampling variance
> 3. standard error
>
> However since for each of the cases/predictors used (see below) I got a
> full model result, I assume --may be wrongly--that the value I should take
> as the numerical estimation of the publication bised is the "intercept" of
> the model, is this correct?
>
> Is there a given range that might serve as a proxy indicator of potential publication
> bias?
>
> Code and model's results below.
>
> Thanks a lot.
> Kind regards,
> Gabriel
>
>
> ## NUMERICAL TEST FOR PUBLICATION BIAS
>
> ## extending Egger's test to more complex models.
> ## "regression test for funnel plot asymmetry".
>
> ## 1. using : the sampling variance
> PubB<-rma.mv(yi = yi,
> V = vi,
> mods = vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8908 0.9438 72 no Article
> # sigma^2.2 2.1970 1.4822 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24617.3110, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4381, p-val = 0.2304
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.5620 0.2584 2.1743 0.0297 0.0554 1.0685 *
> # mods -6.7997 5.6701 -1.1992 0.2304 -17.9130 4.3135
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
> ## 2. using : the inverse of the sampling variance
> PubB_1<-rma.mv(yi = yi,
> V = vi,
> mods = 1/vi,
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_1
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.9176 0.9579 72 no Article
> # sigma^2.2 2.1797 1.4764 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 23656.2997, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.4469, p-val = 0.2290
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.0957 0.2595 0.3689 0.7122 -0.4129 0.6044
> # mods 0.0040 0.0034 1.2029 0.2290 -0.0025 0.0106
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> ## 3. using : standard errors (square-root of the sampling variances)
>
> PubB_2<-rma.mv(yi = yi,
> V = vi,
> mods = sqrt(vi),
> random = ~ 1 | Article / Sample_ID,
> data = dat,
> method = "REML")
>
> PubB_2
> # Multivariate Meta-Analysis Model (k = 149; method: REML)
> #
> # Variance Components:
> #
> # estim sqrt nlvls fixed factor
> # sigma^2.1 0.8991 0.9482 72 no Article
> # sigma^2.2 2.1952 1.4816 149 no Article/Sample_ID
> #
> # Test for Residual Heterogeneity:
> # QE(df = 147) = 24489.6313, p-val < .0001
> #
> # Test of Moderators (coefficient 2):
> # QM(df = 1) = 1.2528, p-val = 0.2630
> #
> # Model Results:
> #
> # estimate se zval pval ci.lb ci.ub
> # intrcpt 0.7801 0.4374 1.7834 0.0745 -0.0772 1.6375 .
> # mods -2.6473 2.3652 -1.1193 0.2630 -7.2829 1.9883
> #
> # ---
> # Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
>
>
> On Wed, Jul 5, 2023 at 12:45 PM Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> wrote:
>
>> Dear Gabriel
>>
>> Apart from Michael observation on data checking before analyses (which is
>> always a good practice), I add one.
>> The funnel plot is just a visual check of publication bias. So the
>> observations based on the funnel plots are inevitably subjective - I mean
>> you think that is "randomly scattering", while others might think not. In
>> contrast, Egger's test is a more objective way to test the asymmetry of a
>> funnel plot. Regarding how to do it, you can try to find them in the
>> archives associated with this mailing list.
>>
>> Please be noted that whether none funnel plot and Egger's test can
>> indicate publication bias directly. But it is common to assume the
>> asymmetry of a funnel plot is caused by publication bias (or more
>> precisely. small study effects), after accounting for heterogeneity.
>>
>> Best,
>> Yefeng
>> ------------------------------
>> *From:* R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
>> on behalf of Michael Dewey via R-sig-meta-analysis <
>> r-sig-meta-analysis using r-project.org>
>> *Sent:* Wednesday, 5 July 2023 19:31
>> *To:* R Special Interest Group for Meta-Analysis <
>> r-sig-meta-analysis using r-project.org>
>> *Cc:* Michael Dewey <lists using dewey.myzen.co.uk>; Gabriel Cotlier <
>> gabiklm01 using gmail.com>
>> *Subject:* Re: [R-meta] Question on funnel plot interpretation
>>
>> Dear Gabriel
>>
>> My interpretation looking at your plots is that you have a very strange
>> set of primary studies. If the x-axis is really the z transformation of
>> r then some of the r are .999 and some 0.001 which seems worthy of
>> investigation before looking further.
>>
>> Michael
>>
>> On 05/07/2023 09:14, Gabriel Cotlier via R-sig-meta-analysis wrote:
>> > Hello all,
>> >
>> > I have produced a funnel plot on the basis of an rma.mv
>> > <http://rma.mv>() objectapplied to all the data set together ( not
>> > subsetting using moderators ) as follows:
>> >
>> > image.png
>> >
>> >
>> > When looking at the figure I tried to think that maybe one of the
>> > following two interpretations could be the correct one:
>> >
>> > a. There is a kind of random scattering of the effect sizes, therefore
>> > no symmetry is found and thus publication bised is observed.
>> > b. Given the randomness of the effect sizes distribution covering the
>> > plot's space unevenly there is not a clear pattern that can indicate
>> > publication bias is observed.
>> >
>> > Is any of this interpretation the correct one?
>> >
>> > Thanks a lot.
>> > Kind regards,
>> > Gabriel
>> >
>> >
>> > #### CODE. ######
>> > funnel_all <- rma.mv <http://rma.mv>(yi,
>> > vi,
>> > random = ~ 1 | Article / Sample_ID,
>> > data=dat)
>> > png(file = "funnel.png",
>> > width = 250,
>> > height = 200,
>> > res = 600,
>> > units = "mm")
>> > # par(mfrow = c(2, 1))
>> >
>> > # full data
>> > f1 <- funnel(funnel_all,
>> > yaxis = "seinv",
>> > level = c(90, 95, 99),
>> > ylim = c(1, 20),
>> > shade = c("white", "gray55", "gray75"),
>> > refline = 0,
>> > legend = TRUE)
>> > mtext("A", side = 3, line = 0, adj = -0.13, cex = 2)
>> >
>> > dev.off()
>> >
>> > <
>> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
>> Virus-free.www.avg.com <
>> http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient
>> >
>> >
>> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>> >
>> > _______________________________________________
>> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> > To manage your subscription to this mailing list, go to:
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>> --
>> Michael
>> http://www.dewey.myzen.co.uk/home.html
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> To manage your subscription to this mailing list, go to:
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list