[R-meta] Nestedness in meta-analysis data using R (metafor or metaSEM packages)

[Reza Norouzian]

Thanks for your explanation. Very useful! I appreciate your time and clear
explanations.



On Tue, Nov 5, 2019 at 3:40 PM James Pustejovsky <jepusto using gmail.com> wrote:

> The adjustment is for the small number of clusters (in this instance,
> studies). Robust variance estimators are asymptotically consistent in the
> number of clusters, so they become as the number of clusters increases
> towards infinity. However, the basic RVE can be inaccurate and biased
> towards zero when the number of clusters is small. The small-sample
> adjustment corrects for this downward bias, yielding standard errors that
> are closer to unbiased, even when the number of clusters is small or
> moderate. In Tipton & Pustejovsky (2015), we recommended using the small
> sample adjustments by default because a) it can be hard to judge whether
> the number of clusters is "large enough" to omit them and b) it doesn't
> "hurt" to use them, even if they're not strictly necessary (the main
> downside is computational time, but that's not usually much of a
> constraint).
>
> James
>
> On Tue, Nov 5, 2019 at 2:02 PM Reza Norouzian <rnorouzian using gmail.com>
> wrote:
>
>> Thanks, James, very useful. If I may, the adjustment for small samples in
>> correlated models using `robumeta` package corrects for the small number of
>> effect sizes within each cluster, or the small number of clusters (i.e.,
>> studies) themselves, or both?
>>
>> Thanks, Reza
>>
>> On Mon, Nov 4, 2019 at 9:39 PM James Pustejovsky <jepusto using gmail.com>
>> wrote:
>>
>>> Reza,
>>>
>>> Here is a recent article provides a review of the main analytic
>>> approaches available for the data structure that you've described:
>>>
>>>    - Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J.,
>>>    Bunuan, R., & Van den Noortgate, W. (2017). Methods for dealing with
>>>    multiple outcomes in meta-analysis: a comparison between averaging effect
>>>    sizes, robust variance estimation and multilevel meta-analysis. *International
>>>    Journal of Social Research Methodology*, *20*(6), 559-572.
>>>
>>> The question has come up periodically on the mailing list too, so you
>>> might find past posts helpful. A few to get you started:
>>>
>>>    -
>>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-September/000197.html
>>>
>>>    -
>>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000094.html
>>>
>>>    -
>>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2019-March/001479.html
>>>
>>>    -
>>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-October/001153.html
>>>
>>>
>>> James
>>>
>>> On Sun, Nov 3, 2019 at 7:36 PM Reza Norouzian <rnorouzian using gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have 51 studies. These studies have produced 257 effect sizes
>>>> (denoted y).
>>>> In each study, each effect size is obtained some time (categorized as
>>>> 1, 2,
>>>> 3, 4) after a treatment. Also, in15 of the studies, some effect sizes
>>>> have
>>>> been reported on up to 4 correlated outcomes.
>>>>
>>>> What reasonable model using the library(metafor) [e.g., via rma.mv] or
>>>> library(metaSEM) [e.g., via meta3] can capture the structure of my data
>>>> described above?
>>>>
>>>> For concreteness, I'm providing demo data below.
>>>>
>>>> d <- read.csv("https://raw.githubusercontent.com/izeh/m/master/e.csv",
>>>> h = T) # DATA
>>>> # First 6 lines of DATA:
>>>>
>>>>           study.name            y        SD id outcome time 1
>>>> Al.Ahm_Al.Jar  0.533733731 0.4286817  1       1    1          2
>>>> Al.Ahm_Al.Jar  0.296116599 0.4132781  1       1    4          3
>>>> Al.Ahm_Al.Jar -0.155386371 0.4267881  1       1    1          4
>>>> Al.Ahm_Al.Jar  0.131015334 0.4384274  1       1    4          5
>>>>     Al_Ajmi  3.444226818 0.2182068  2       1    1          6
>>>>   Al_Ajmi  4.376457433 0.2090771  2       1    2
>>>>
>>>>         [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-meta-analysis mailing list
>>>> R-sig-meta-analysis using r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>>
>>>
>>
>> --
>> *Reza Norouzian*
>> Postdoctoral Research Associate | Lecturer
>> Second Language Acquisition & Research Methods, Ph.D.
>> College of Education & Human Development
>> Dep. of Teaching, Learning & Culture | Texas A&M University
>> College Station, TX 77843
>> Webpage: *https://directory.education.tamu.edu/view.epl?nid=rnorouzian
>> <https://directory.education.tamu.edu/view.epl?nid=rnorouzian>*
>> Email: rnorouzian using tamu.edu
>> Phone: (979)-422-7052
>> *Future L2 researchers will be challenged not only on the basis of their
>> substantive questions, but also on how they manage to answer those
>> questions in a methodical manner.*
>>
>

-- 
*Reza Norouzian*
Postdoctoral Research Associate | Lecturer
Second Language Acquisition & Research Methods, Ph.D.
College of Education & Human Development
Dep. of Teaching, Learning & Culture | Texas A&M University
College Station, TX 77843
Webpage: *https://directory.education.tamu.edu/view.epl?nid=rnorouzian
<https://directory.education.tamu.edu/view.epl?nid=rnorouzian>*
Email: rnorouzian using tamu.edu
Phone: (979)-422-7052
*Future L2 researchers will be challenged not only on the basis of their
substantive questions, but also on how they manage to answer those
questions in a methodical manner.*

	[[alternative HTML version deleted]]