[R-meta] Publication bias with multivariate meta analysis
Dr. Gerta Rücker
ruecker @end|ng |rom |mb|@un|-|re|burg@de
Tue Aug 31 13:07:02 CEST 2021
Dear Norman,
Please find my response inline below. (I skipped some lines of your e-mail).
Am 31.08.2021 um 00:07 schrieb Norman DAURELLE:
>
> Dear Gerta, dear list members
>
> thank you for your answer. I read the paper you indicated, and it was
> really interesting for me to learn about what is discussed in it.
>
> In a way it gives me more questions than answers : I used to think
> that heterogeneity was required for a meta-analysis. In my mind, if
> there was no heterogeneity, there was no point in even using the
> meta-analytic method (I thought no heterogeneity more or less meant
> all studies "reported the same result" / that they all "agreed").
This is interesting. Yes, heterogeneity has to be expected, see Julian
Higgins's paper https://academic.oup.com/ije/article/37/5/1158/871288 .
However, it never came to my mind that heterogeneity was *required*. Of
course, it is not: in cases where there is no heterogeneity at all
(probably true only in a minority of research questions), meta-analysis
makes much sense, as you can estimate the true effect with much more
precision as in each single study.
>
> [...]
>
> So : after reading the paper you suggested I read, is it possible that
> what I observed in the asymmetry of my funnel plot was actually mainly
> the heterogeneity of the data (say, differences in slopes due to
> temperature, or rainfall, or soil, etc... conditions in different
> studies carried out in different countries) ?
>
> I still believe that there might be a slight publication bias (because
> we expect yield to decrease with disease severity, and if the data
> does not show that we will tend to think that we did not observe "the
> effect of disease severity, all else being equal"), but I am not so
> sure anymore that publication bias could be the main reason why the
> funnel plot of this meta-analysis is asymmetrical.
>
> It seems that there may be more than one reason why it is, and some of
> these reasons I might (from what I understand now, but I might still
> be wrong) apparently never discover.
>
> However, I would definetly like to be able to offer possible reasons
> why it is asymmetrical, and what this means.
I just saw Michael's example what might have happened in a different
area, and in fact this seems to be a general pattern: Small studies tend
to show greater effects. As I am working in the medical field, I know
possible explanations from there (they cannot be well separated from
each other):
- Small studies often have been the first studies for the given research
question, conducted without funding, were less regularized, and had
particularly selected observations;
- Small studies consider more homogeneous study populations (in health
science therapy studies these are often subjects easier to treat).
This phenomenon has also be called the Proteus effect
https://journals.plos.org/plosclinicaltrials/article?id=10.1371/journal.pctr.0010036
.
As I don't know your field, it is difficult for me to guess what might
be behind the funnel plot asymmetry in your case.
Best,
Gerta
>
> Sorry for this long and not very to-the-point e-mail,
>
> I would very much be grateful for an answer.
>
> Best wishes,
> Norman
>
>
>
>
>
> ------------------------------------------------------------------------
> *De: *"Dr. Gerta Rücker" <ruecker using imbi.uni-freiburg.de>
> *À: *"Norman DAURELLE" <norman.daurelle using agroparistech.fr>, "Wolfgang
> Viechtbauer, SP" <wolfgang.viechtbauer using maastrichtuniversity.nl>
> *Cc: *"r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>,
> "Huang Wu" <huang.wu using wmich.edu>
> *Envoyé: *Lundi 30 Août 2021 11:40:23
> *Objet: *Re: [R-meta] Publication bias with multivariate meta analysis
>
> Dear Norman,
>
> If there is funnel plot asymmtery, there is always some relation between
> observed effects and their standard errors, the question is what causes
> this relationship. Possible causes are discussed in Sterne et al.
> (2011), see https://www.bmj.com/content/343/bmj.d4002
>
> Best wishes,
>
> Gerta
>
> Am 30.08.2021 um 10:22 schrieb Norman DAURELLE:
> > Dear list members, dear Huang and Wolfgang,
> >
> > thank you for explaining that there is no method for testing for
> publication bias, or more accurately, for explaining that a
> relationship between observed effects and their standard errors does
> not necessarily indicate publication bias (meaning that there are
> other reasons why one could encounter such a relationship).
> >
> > Outside of Huang's question : does funnel plot asymetry necessarily
> indicate a relationship between observed effects and their standard
> error ?
> >
> > I am going to have a deeper read at [
> https://www.metafor-project.org/ | https://www.metafor-project.org/ ]
> but I would be grateful for an answer.
> >
> > Best wishes,
> > Norman
> >
> >
> > De: "Wolfgang Viechtbauer, SP"
> <wolfgang.viechtbauer using maastrichtuniversity.nl>
> > À: "Huang Wu" <huang.wu using wmich.edu>, "r-sig-meta-analysis"
> <r-sig-meta-analysis using r-project.org>
> > Envoyé: Samedi 28 Août 2021 15:37:20
> > Objet: Re: [R-meta] Publication bias with multivariate meta analysis
> >
> > Dear Huang,
> >
> > Please find my comments below.
> >
> > Best,
> > Wolfgang
> >
> >> -----Original Message-----
> >> From: R-sig-meta-analysis
> [mailto:r-sig-meta-analysis-bounces using r-project.org] On
> >> Behalf Of Huang Wu
> >> Sent: Saturday, 28 August, 2021 3:19
> >> To: r-sig-meta-analysis using r-project.org
> >> Subject: [R-meta] Publication bias with multivariate meta analysis
> >>
> >> Hi all,
> >>
> >> I am conducting a multivariate meta-analysis using rma.mv. I want
> to test for
> >> publication bias.
> >> I noticed in a previous post, Dr. Pustejovsky provided the
> following code for
> >> Egger’s test.
> >>
> >> egger_multi <- rma.mv(yi = yi, V = sei^2, random = ~ 1 |
> studyID/effectID,
> >> mods = ~ sei, data = dat)
> >> coef_test(egger_multi, vcov = "CR2")
> >>
> >> Because I conducted a multivariate meta-analysis assuming rho =
> 0.8, I wonder for
> >> the Egger’s test, Do I need to let V equals to the imputed
> covariance matrix?
> >> Would anyone help me to see if my following code is correct? Thanks.
> >>
> >> V_listm <- impute_covariance_matrix(vi = meta$dv,
> >> cluster = meta$Study.ID,
> >> r = 0.8)
> >> egger_multi <- rma.mv(yi =Cohen.s.d, V = V_listm, random = ~ 1 |
> Study.ID/IID,
> >> mods = ~ sqrt(dv), data = meta)
> >> coef_test(egger_multi, vcov = "CR2")
> > If you used such an approximate V matrix for your analyses, then I
> would also use this in this model.
> >
> >> Also, I have tried V = V_listm and V = dv, but it gave me different
> results. When
> >> I use V = V_Vlistm, my results suggest the effect was no longer
> statistically
> >> significant but when I use V = dv, my result is still significant.
> >> Does that mean my results were sensitive to the value of rho? Thanks.
> > Yes, although it's not clear to me what exactly you mean by "the
> effect". The coefficient corresponding to 'sqrt(dv)'?
> >
> >> By the way, does anyone have any suggestions/codes for other
> methods of testing
> >> publication bias? Many thanks.
> > Just a pedantic note: There are no methods for testing for
> publication bias. One can for example test if there is a relationship
> between the observed effects and their standard errors (as done
> above), which could result from publication bias, but there could be
> other explanations for such a relationship besides publication bias.
> >
> > This aside, one can also examine if there is a relationship at the
> study level (not at the level of the individual estimates, as done
> above). A simple approach for this would be to aggregate the estimates
> to the study level, using the aggregate() function. In fact, at that
> point, you could apply all of the methods available in metafor or
> other packages related to the issue of publication bias (including
> things like trim-and-fill, selection models, and so on).
> >
> > Best,
> > Wolfgang
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
> --
>
> Dr. rer. nat. Gerta Rücker, Dipl.-Math.
>
> Institute of Medical Biometry and Statistics,
> Faculty of Medicine and Medical Center - University of Freiburg
>
> Zinkmattenstr. 6a, D-79108 Freiburg, Germany
>
> Mail: ruecker using imbi.uni-freiburg.de
> Homepage:
> https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker
--
Dr. rer. nat. Gerta Rücker, Dipl.-Math.
Institute of Medical Biometry and Statistics,
Faculty of Medicine and Medical Center - University of Freiburg
Zinkmattenstr. 6a, D-79108 Freiburg, Germany
Mail: ruecker using imbi.uni-freiburg.de
Homepage: https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list