[R-meta] Nestedness in meta-analysis data using R (metafor or metaSEM packages)

[James Pustejovsky]

The adjustment is for the small number of clusters (in this instance,
studies). Robust variance estimators are asymptotically consistent in the
number of clusters, so they become as the number of clusters increases
towards infinity. However, the basic RVE can be inaccurate and biased
towards zero when the number of clusters is small. The small-sample
adjustment corrects for this downward bias, yielding standard errors that
are closer to unbiased, even when the number of clusters is small or
moderate. In Tipton & Pustejovsky (2015), we recommended using the small
sample adjustments by default because a) it can be hard to judge whether
the number of clusters is "large enough" to omit them and b) it doesn't
"hurt" to use them, even if they're not strictly necessary (the main
downside is computational time, but that's not usually much of a
constraint).

James

On Tue, Nov 5, 2019 at 2:02 PM Reza Norouzian <rnorouzian using gmail.com> wrote:

> Thanks, James, very useful. If I may, the adjustment for small samples in
> correlated models using `robumeta` package corrects for the small number of
> effect sizes within each cluster, or the small number of clusters (i.e.,
> studies) themselves, or both?
>
> Thanks, Reza
>
> On Mon, Nov 4, 2019 at 9:39 PM James Pustejovsky <jepusto using gmail.com>
> wrote:
>
>> Reza,
>>
>> Here is a recent article provides a review of the main analytic
>> approaches available for the data structure that you've described:
>>
>>    - Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan,
>>    R., & Van den Noortgate, W. (2017). Methods for dealing with multiple
>>    outcomes in meta-analysis: a comparison between averaging effect sizes,
>>    robust variance estimation and multilevel meta-analysis. *International
>>    Journal of Social Research Methodology*, *20*(6), 559-572.
>>
>> The question has come up periodically on the mailing list too, so you
>> might find past posts helpful. A few to get you started:
>>
>>    -
>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-September/000197.html
>>
>>    -
>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000094.html
>>
>>    -
>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2019-March/001479.html
>>
>>    -
>>    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2018-October/001153.html
>>
>>
>> James
>>
>> On Sun, Nov 3, 2019 at 7:36 PM Reza Norouzian <rnorouzian using gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I have 51 studies. These studies have produced 257 effect sizes (denoted
>>> y).
>>> In each study, each effect size is obtained some time (categorized as 1,
>>> 2,
>>> 3, 4) after a treatment. Also, in15 of the studies, some effect sizes
>>> have
>>> been reported on up to 4 correlated outcomes.
>>>
>>> What reasonable model using the library(metafor) [e.g., via rma.mv] or
>>> library(metaSEM) [e.g., via meta3] can capture the structure of my data
>>> described above?
>>>
>>> For concreteness, I'm providing demo data below.
>>>
>>> d <- read.csv("https://raw.githubusercontent.com/izeh/m/master/e.csv",
>>> h = T) # DATA
>>> # First 6 lines of DATA:
>>>
>>>           study.name            y        SD id outcome time 1
>>> Al.Ahm_Al.Jar  0.533733731 0.4286817  1       1    1          2
>>> Al.Ahm_Al.Jar  0.296116599 0.4132781  1       1    4          3
>>> Al.Ahm_Al.Jar -0.155386371 0.4267881  1       1    1          4
>>> Al.Ahm_Al.Jar  0.131015334 0.4384274  1       1    4          5
>>>     Al_Ajmi  3.444226818 0.2182068  2       1    1          6
>>>   Al_Ajmi  4.376457433 0.2090771  2       1    2
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
>>
>
> --
> *Reza Norouzian*
> Postdoctoral Research Associate | Lecturer
> Second Language Acquisition & Research Methods, Ph.D.
> College of Education & Human Development
> Dep. of Teaching, Learning & Culture | Texas A&M University
> College Station, TX 77843
> Webpage: *https://directory.education.tamu.edu/view.epl?nid=rnorouzian
> <https://directory.education.tamu.edu/view.epl?nid=rnorouzian>*
> Email: rnorouzian using tamu.edu
> Phone: (979)-422-7052
> *Future L2 researchers will be challenged not only on the basis of their
> substantive questions, but also on how they manage to answer those
> questions in a methodical manner.*
>

	[[alternative HTML version deleted]]