[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11

[Viechtbauer, Wolfgang (SP)]

Dear Tarun,

Not sure where you are getting the impression from that method-of-moment estimators might be preferable for small sample sizes.

With respect to your theoretical question: It's not clear to me which specific results you are comparing. If you are comparing the fit of a standard RE/ME model from rma() with what you get from robust() or clubSandwich:: coef_test() where you specify some higher-order clustering variable, then an increase in the SEs is expected when there is indeed dependency within clusters. And that's as it should be. You are then in fact preventing too many Type I errors, so it's the other way around.

Also, in the future, please use a more meaningful subject than "[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11" (I won't change it now as this might break the threading in email clients).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On Behalf Of Tarun Khanna
>Sent: Tuesday, 21 April, 2020 16:44
>To: James Pustejovsky
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
>
>Thank you, James. I understand that the moments estimators might be more
>useful when working with small sample sizes. In my data set I have about 60
>studies and 110 effect sizes. So as such the dataset is not small. But I do
>want to estimate effect sizes for smaller sets of the data (there are
>multiple set of interventions which can be distinguished). In the smaller
>sets the number of effects decreases to as low as 15-30. In this context, I
>thought DL might be a better estimator. I will look into the robumeta
>package.
>
>I also have a theoretical question  around RVE. The estimates that I get for
>RVE have much higher standard errors compared to the DL/REML estimator. I
>understand that this is to be expected, RVE is also likely to result in
>higher Type I errors. Is there any way to control for that in the metafor
>package?
>
>Best
>Tarun
>
>Tarun Khanna
>PhD Researcher
>
>Hertie School
>
>Friedrichstraße 180
>10117 Berlin ∙ Germany
>khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-
>school.org/>
>
>________________________________
>From: James Pustejovsky <jepusto using gmail.com>
>Sent: 21 April 2020 16:22:38
>To: Tarun Khanna
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
>
>Hi Tarun,
>
>Good question! The Dersimonian-Laird variance estimator is in a general
>class of what are called moment estimators. In principle, it is possible to
>use moment estimation for models that are more complex than the basic meta-
>analysis/meta-regression model (as estimated with rma.uni, for instance). In
>fact, this is precisely what the robumeta package does. The correlated
>effects model and hierarchical effects model implemented in that package use
>moment estimators of the between-study variance component, and for the
>hierarchical model, also the within-study variance component. As far as I
>understand them, I think these estimators are not as precise as using
>ML/REML, and are mainly intended as way to get "quick-and-dirty" values for
>use in a working model (which need not be super accurate, if RVE is then
>used to get standard errors/confidence intervals).
>
>There has also been some statistical work on moment estimators for more
>complex multi-variate models:
>* Chen, H., Manning, A. K., & Dupuis, J. (2012). A method of moments
>estimator for random effect multivariate meta‐analysis. Biometrics, 68(4),
>1278-1284.
>I'm not sure if the methods described here are implemented in software
>though.
>
>Kind Regards,
>James