[R-meta] R-sig-meta-analysis Digest, Vol 37, Issue 23
Tarun Khanna
kh@nn@ @end|ng |rom hert|e-@choo|@org
Fri Jun 12 16:15:03 CEST 2020
Thanks for the explnation Wolfgang. I also found it useful.
In your reply you mention that - "We can ignore those correlations and use the multilevel model as a working model that is an approximation to the model that also accounts for correlated sampling errors. After fitting the multilevel model with rma.mv(), one can then use cluster robust inference methods to 'fix things up'."
Do you have an example study that does this?
Best
Tarun
Tarun Khanna
PhD Researcher
Hertie School
Friedrichstraße 180
10117 Berlin ∙ Germany
khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-school.org/>
________________________________
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> on behalf of r-sig-meta-analysis-request using r-project.org <r-sig-meta-analysis-request using r-project.org>
Sent: 12 June 2020 01:09:31
To: r-sig-meta-analysis using r-project.org
Subject: R-sig-meta-analysis Digest, Vol 37, Issue 23
Send R-sig-meta-analysis mailing list submissions to
r-sig-meta-analysis using r-project.org
To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
or, via email, send a message with subject or body 'help' to
r-sig-meta-analysis-request using r-project.org
You can reach the person managing the list at
r-sig-meta-analysis-owner using r-project.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-meta-analysis digest..."
Today's Topics:
1. Re: weight in rmv metafor (Viechtbauer, Wolfgang (SP))
2. Re: weight in rmv metafor (Emerson Del Ponte)
3. Re: weight in rmv metafor (Norman DAURELLE)
----------------------------------------------------------------------
Message: 1
Date: Thu, 11 Jun 2020 13:33:02 +0000
From: "Viechtbauer, Wolfgang (SP)"
<wolfgang.viechtbauer using maastrichtuniversity.nl>
To: Norman DAURELLE <norman.daurelle using agroparistech.fr>
Cc: r-sig-meta-analysis <r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] weight in rmv metafor
Message-ID: <1b8c1463bcdf43baaa39788aa2a859c1 using UM-MAIL3214.unimaas.nl>
Content-Type: text/plain; charset="iso-8859-1"
Dear Norman,
To give a simple example: When (some of the) studies contribute multiple estimates, the dataset has a multilevel structure (with estimates nested within studies). A common way to deal with this is to fit a multilevel model with random effects for studies and estimates within studies. Like this:
http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
However, multiple estimates from the same study are actually often computed based on the same sample of subjects. In that case, the sampling errors are also correlated. The multilevel model does not capture this. For this, one would ideally want to fit a model that also allows for correlated sampling errors. Like this:
http://www.metafor-project.org/doku.php/analyses:berkey1998
However, computing the covariances between the sampling errors within a study is difficult and requires information that is often not available.
We can ignore those correlations and use the multilevel model as a working model that is an approximation to the model that also accounts for correlated sampling errors. After fitting the multilevel model with rma.mv(), one can then use cluster robust inference methods to 'fix things up'.
Quite a bit of this has been discussed at length in previous posts on this mailing list. You might want to search the archives for some of these posts.
Best,
Wolfgang
>-----Original Message-----
>From: Norman DAURELLE [mailto:norman.daurelle using agroparistech.fr]
>Sent: Thursday, 11 June, 2020 15:05
>To: Viechtbauer, Wolfgang (SP)
>Cc: r-sig-meta-analysis
>Subject: Re: [R-meta] weight in rmv metafor
>
>Thank you.
>I am not sure I understand exactly what you mean by " if the working model
>is only an approximation and doesn't cover all dependencies ".
>Could you please explain it ?
>
>For now I used the rma() function to synthesize the available literature
>existing on the blackleg - oil seed rape disease-yield relationship, using
>slopes as effect-sizes.
>the models that gave me the slopes I used in the meta-analysis are all Y = a
>+ bX, simple linear regressions with Y being the yield and X being the
>diseqse severity.
>So my slopes, b, are all negative, and I have obtained a "summary" effect
>size through the rma() function.
>
>But I indeed have two studies that for now contribute to most of the effect-
>sizes that are included in my meta-analysis.
>
>So why exactly is it necessary to use the rma.mv() function ?
>What exactly does the "multivariate" qualificative refer to ?
>
>Thank you,
>Norman.
>
>________________________________________
>De: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
>À: "Norman DAURELLE" <norman.daurelle using agroparistech.fr>, "r-sig-meta-
>analysis" <r-sig-meta-analysis using r-project.org>
>Envoyé: Jeudi 11 Juin 2020 22:34:55
>Objet: RE: [R-meta] weight in rmv metafor
>
>Dear Norman,
>
>If you only used rma(), then this is not correct. rma.mv() with an
>appropriately specified model (plus clubSandwich::coef_test() if the working
>model is only an approximation and doesn't cover all dependencies) would be
>more appropriate.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Norman DAURELLE [mailto:norman.daurelle using agroparistech.fr]
>>Sent: Thursday, 11 June, 2020 14:13
>>To: r-sig-meta-analysis
>>Cc: Viechtbauer, Wolfgang (SP)
>>Subject: Re: [R-meta] weight in rmv metafor
>>
>>Hi all,
>>
>>I read this discussion and one question came to my mind : I also had some
>>studies that contributed multiple effect sizes in the meta-analysis that I
>>recently ran thanks to Dr Viechtbauer's advice.
>>For now I only used the rma function, but should I have used rma.mv because
>>of these stuides that had multiple effect sizes ?
>>
>>Thank you !
>>
>>Norman
>>
>>________________________________________
>>De: "James Pustejovsky" <jepusto using gmail.com>
>>À: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
>>Cc: "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>, "Huang Wu"
>><huang.wu using wmich.edu>
>>Envoyé: Mercredi 10 Juin 2020 05:08:09
>>Objet: Re: [R-meta] weight in rmv metafor
>>
>>Hi Huang,
>>
>>I've written up some notes that add a bit of further intuition to the
>>discussion that Wolfgang provided. The main case that I focus on is a model
>>that is just a meta-analysis (i.e., no predictors) and that includes random
>>effects to capture both between-study and within-study heterogeneity. I
>>also say a little bit about meta-regression models with only study-level
>>predictors.
>>
>>https://www.jepusto.com/weighting-in-multivariate-meta-analysis/
>>
>>Best,
>>James
>>
>>On Sun, Jun 7, 2020 at 4:11 PM Viechtbauer, Wolfgang (SP) <
>>wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>>> Of course the weights "impact the estimated fixed effects". But whether
>>> studies with multiple effect sizes tend to receive more weight depends on
>>> various factors, including the variances of the random effects and the
>>> sampling error (co)variances.
>>>
>>> A more detailed discussion around the way weighting works in rma.mv
>>> models can be found here:
>>>
>>> http://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models
>>>
>>> Note that weights(res, type="rowsum") currently only works in the 'devel'
>>> version of metafor, so follow
>>> https://wviechtb.github.io/metafor/#installation if you want to reproduce
>>> this part as well.
>>>
>>> I hope this clarifies things.
>>>
>>> Best,
>>> Wolfgang
------------------------------
Message: 2
Date: Thu, 11 Jun 2020 10:43:05 -0300
From: Emerson Del Ponte <delponte using ufv.br>
To: Norman DAURELLE <norman.daurelle using agroparistech.fr>
Cc: Wolfgang Viechtbauer
<wolfgang.viechtbauer using maastrichtuniversity.nl>, r-sig-meta-analysis
<r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] weight in rmv metafor
Message-ID:
<CAFgcnpJEEXJdSw0ySSTz+WhLtm+D3RgWm9Y0e8RqFLj3UezjUw using mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Norman
You may want to check reproducible examples of my previous work on this
exact application context as a starting point.
https://emdelponte.github.io/paper-white-mold-meta-analysis/
https://emdelponte.github.io/paper-FHB-yield-loss/code_meta_analysis.html
Emerson
On Thu, 11 Jun 2020 at 10:06 Norman DAURELLE <
norman.daurelle using agroparistech.fr> wrote:
>
> Thank you.
> I am not sure I understand exactly what you mean by " i f the working
> model is only an approximation and doesn't cover all dependencies ".
> Could you please explain it ?
>
> For now I used the rma() function to synthesize the available literature
> existing on the blackleg - oil seed rape disease-yield relationship, using
> slopes as effect-sizes.
> the models that gave me the slopes I used in the meta-analysis are all Y =
> a + bX, simple linear regressions with Y being the yield and X being the
> diseqse severity.
> So my slopes, b, are all negative, and I have obtained a "summary" effect
> size through the rma() function.
>
> But I indeed have two studies that for now contribute to most of the
> effect-sizes that are included in my meta-analysis.
>
> So why exactly is it necessary to use the rma.mv() function ?
> What exactly does the "multivariate" qualificative refer to ?
>
> Thank you,
> Norman.
>
>
> De: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
> À: "Norman DAURELLE" <norman.daurelle using agroparistech.fr>,
> "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>
> Envoyé: Jeudi 11 Juin 2020 22:34:55
> Objet: RE: [R-meta] weight in rmv metafor
>
> Dear Norman,
>
> If you only used rma(), then this is not correct. rma.mv() with an
> appropriately specified model (plus clubSandwich::coef_test() if the
> working model is only an approximation and doesn't cover all dependencies)
> would be more appropriate.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Norman DAURELLE [mailto:norman.daurelle using agroparistech.fr]
> >Sent: Thursday, 11 June, 2020 14:13
> >To: r-sig-meta-analysis
> >Cc: Viechtbauer, Wolfgang (SP)
> >Subject: Re: [R-meta] weight in rmv metafor
> >
> >Hi all,
> >
> >I read this discussion and one question came to my mind : I also had some
> >studies that contributed multiple effect sizes in the meta-analysis that
> I
> >recently ran thanks to Dr Viechtbauer's advice.
> >For now I only used the rma function, but should I have used rma.mv
> because
> >of these stuides that had multiple effect sizes ?
> >
> >Thank you !
> >
> >Norman
> >
> >________________________________________
> >De: "James Pustejovsky" <jepusto using gmail.com>
> >À: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
> >Cc: "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>, "Huang
> Wu"
> ><huang.wu using wmich.edu>
> >Envoyé: Mercredi 10 Juin 2020 05:08:09
> >Objet: Re: [R-meta] weight in rmv metafor
> >
> >Hi Huang,
> >
> >I've written up some notes that add a bit of further intuition to the
> >discussion that Wolfgang provided. The main case that I focus on is a
> model
> >that is just a meta-analysis (i.e., no predictors) and that includes
> random
> >effects to capture both between-study and within-study heterogeneity. I
> >also say a little bit about meta-regression models with only study-level
> >predictors.
> >
> >https://www.jepusto.com/weighting-in-multivariate-meta-analysis/
> >
> >Best,
> >James
> >
> >On Sun, Jun 7, 2020 at 4:11 PM Viechtbauer, Wolfgang (SP) <
> >wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >
> >> Of course the weights "impact the estimated fixed effects". But whether
> >> studies with multiple effect sizes tend to receive more weight depends
> on
> >> various factors, including the variances of the random effects and the
> >> sampling error (co)variances.
> >>
> >> A more detailed discussion around the way weighting works in rma.mv
> >> models can be found here:
> >>
> >> http://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models
> >>
> >> Note that weights(res, type="rowsum") currently only works in the
> 'devel'
> >> version of metafor, so follow
> >> https://wviechtb.github.io/metafor/#installation if you want to
> reproduce
> >> this part as well.
> >>
> >> I hope this clarifies things.
> >>
> >> Best,
> >> Wolfgang
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
--
*Emerson M. Del Ponte*
Universidade Federal de Viçosa, Brazil
Chair of the Graduate Studies <http://www.dfp.ufv.br/graduate/> in Plant
Pathology
EIC for Tropical Plant Pathology <http://sbfitopatologia.org.br/tpp/>
Co-Founder of Open Plant Pathology <https://www.openplantpathology.org/>
My websites: Twitter <https://twitter.com/edelponte> | GitHub
<https://github.com/emdelponte> | Google Scholar
<https://scholar.google.com.br/citations?user=a1rPnI0AAAAJ> | ResearchGate
<https://www.researchgate.net/profile/Emerson_Del_Ponte>
Tel +55 31 36124830
[[alternative HTML version deleted]]
------------------------------
Message: 3
Date: Fri, 12 Jun 2020 01:08:31 +0200 (CEST)
From: Norman DAURELLE <norman.daurelle using agroparistech.fr>
To: Emerson Del Ponte <delponte using ufv.br>
Cc: Wolfgang Viechtbauer
<wolfgang.viechtbauer using maastrichtuniversity.nl>, r-sig-meta-analysis
<r-sig-meta-analysis using r-project.org>
Subject: Re: [R-meta] weight in rmv metafor
Message-ID:
<1316672195.29298546.1591916911202.JavaMail.zimbra using agroparistech.fr>
Content-Type: text/plain; charset="utf-8"
Dear all, Dr Viechtbauer, Dr Del Ponte,thank you for your answers ! I will look into what you advised me to read, and also go and read more of the archive.If I have more questions I will come back and ask them.Once again, thank you for developing the package metafor in R Dr Viechtbauer, and for creating this mailing list.Have a nice week-end,Norman
----- Mail d'origine -----
De: Emerson Del Ponte <delponte using ufv.br>
�: Norman DAURELLE <norman.daurelle using agroparistech.fr>
Cc: Wolfgang Viechtbauer <wolfgang.viechtbauer using maastrichtuniversity.nl>, r-sig-meta-analysis <r-sig-meta-analysis using r-project.org>
Envoy�: Thu, 11 Jun 2020 15:43:05 +0200 (CEST)
Objet: Re: [R-meta] weight in rmv metafor
Dear Norman
You may want to check reproducible examples of my previous work on this exact application context as a starting point.
https://emdelponte.github.io/paper-white-mold-meta-analysis/
https://emdelponte.github.io/paper-FHB-yield-loss/code_meta_analysis.html
Emerson
On Thu, 11 Jun 2020 at 10:06 Norman DAURELLE <norman.daurelle using agroparistech.fr> wrote:
Thank you.
I am not sure I understand exactly what you mean by " i f the working model is only an approximation and doesn't cover all dependencies ".
Could you please explain it ?
For now I used the rma() function to synthesize the available literature existing on the blackleg - oil seed rape disease-yield relationship, using slopes as effect-sizes.
the models that gave me the slopes I used in the meta-analysis are all Y = a + bX, simple linear regressions with Y being the yield and X being the diseqse severity.
So my slopes, b, are all negative, and I have obtained a "summary" effect size through the rma() function.
But I indeed have two studies that for now contribute to most of the effect-sizes that are included in my meta-analysis.
So why exactly is it necessary to use the rma.mv() function ?
What exactly does the "multivariate" qualificative refer to ?
Thank you,
Norman.
De: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
�: "Norman DAURELLE" <norman.daurelle using agroparistech.fr>, "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>
Envoy�: Jeudi 11 Juin 2020 22:34:55
Objet: RE: [R-meta] weight in rmv metafor
Dear Norman,
If you only used rma(), then this is not correct. rma.mv() with an appropriately specified model (plus clubSandwich::coef_test() if the working model is only an approximation and doesn't cover all dependencies) would be more appropriate.
Best,
Wolfgang
>-----Original Message-----
>From: Norman DAURELLE [mailto:norman.daurelle using agroparistech.fr]
>Sent: Thursday, 11 June, 2020 14:13
>To: r-sig-meta-analysis
>Cc: Viechtbauer, Wolfgang (SP)
>Subject: Re: [R-meta] weight in rmv metafor
>
>Hi all,
>
>I read this discussion and one question came to my mind : I also had some
>studies that contributed multiple effect sizes in the meta-analysis that I
>recently ran thanks to Dr Viechtbauer's advice.
>For now I only used the rma function, but should I have used rma.mv because
>of these stuides that had multiple effect sizes ?
>
>Thank you !
>
>Norman
>
>________________________________________
>De: "James Pustejovsky" <jepusto using gmail.com>
>�: "Wolfgang Viechtbauer" <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Cc: "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org>, "Huang Wu"
><huang.wu using wmich.edu>
>Envoy�: Mercredi 10 Juin 2020 05:08:09
>Objet: Re: [R-meta] weight in rmv metafor
>
>Hi Huang,
>
>I've written up some notes that add a bit of further intuition to the
>discussion that Wolfgang provided. The main case that I focus on is a model
>that is just a meta-analysis (i.e., no predictors) and that includes random
>effects to capture both between-study and within-study heterogeneity. I
>also say a little bit about meta-regression models with only study-level
>predictors.
>
>https://www.jepusto.com/weighting-in-multivariate-meta-analysis/
>
>Best,
>James
>
>On Sun, Jun 7, 2020 at 4:11 PM Viechtbauer, Wolfgang (SP) <
>wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
>> Of course the weights "impact the estimated fixed effects". But whether
>> studies with multiple effect sizes tend to receive more weight depends on
>> various factors, including the variances of the random effects and the
>> sampling error (co)variances.
>>
>> A more detailed discussion around the way weighting works in rma.mv
>> models can be found here:
>>
>> http://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models
>>
>> Note that weights(res, type="rowsum") currently only works in the 'devel'
>> version of metafor, so follow
>> https://wviechtb.github.io/metafor/#installation if you want to reproduce
>> this part as well.
>>
>> I hope this clarifies things.
>>
>> Best,
>> Wolfgang
[[alternative HTML version deleted]]
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
--
Emerson M. Del Ponte
Universidade Federal de Vi�osa, Brazil
Chair of the Graduate Studies in Plant PathologyEIC for Tropical Plant Pathology Co-Founder of Open Plant PathologyMy websites: Twitter | GitHub | Google Scholar | ResearchGateTel +55 31 36124830
[[alternative HTML version deleted]]
------------------------------
Subject: Digest Footer
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
------------------------------
End of R-sig-meta-analysis Digest, Vol 37, Issue 23
***************************************************
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis
mailing list