[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11

[Tarun Khanna]

Thank you for your reply Wolfgang.


1. I am refering to a book by Ringquist (2013) on "Meta analysis in Public Policy". In that book he mentions that "while REML estimate of tau2 is assymptotically unbiased and efficient, it does have three shortcomings. First, the beneefits of REML only occur is large samples. In smaller samples, other methods of estimating tau2 are preferred." He further discusses moments estimate as an alternate to REML.


2. With regards to clustering, I am using the rma.mv function with specification for inner (effect ID) and outer variable (study ID). In my data, there are several estimates from the same studies which are of course then correlated.  I understood that this would be equivalent to using what the book calles clustered robust variance estimation or VRE. In this context Ringquist mentions that "White's (1980) original covariance matrix is only consistent, not unbiased, which means that we can have confidence in parameter standard errors from this martix only in large samples."


He further states that "Under H0: bj = 0, the ratio bj/sbj i ~ N(0,1) and therefore Wald test for H0 will be a a Z score. While this is true asymptotically, with small numbes of clusters this assumption does not hold and hypotesis tests using Z scores will return inflated type I errors (Cameron, Gelbach, and Miller 2008)."


The book uses stata to implement these concepts. I am of course trying to find the equivalent in R so I am not 100 percent sure if these are equivalent. Please do tell me if you think otherwise. Can you also refer to another resource perhaps to understand the theory?


In your reply you mentioned " rma() with what you get from robust() or clubSandwich:: coef_test() ". I have not used this. How would this be different from rma.mv?


Best

Tarun





Tarun Khanna

PhD Researcher

Hertie School


Friedrichstraße 180

10117 Berlin ∙ Germany
khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-school.org/>

________________________________
From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: 23 April 2020 11:59:45
To: Tarun Khanna; r-sig-meta-analysis using r-project.org
Subject: RE: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11

Dear Tarun,

Not sure where you are getting the impression from that method-of-moment estimators might be preferable for small sample sizes.

With respect to your theoretical question: It's not clear to me which specific results you are comparing. If you are comparing the fit of a standard RE/ME model from rma() with what you get from robust() or clubSandwich:: coef_test() where you specify some higher-order clustering variable, then an increase in the SEs is expected when there is indeed dependency within clusters. And that's as it should be. You are then in fact preventing too many Type I errors, so it's the other way around.

Also, in the future, please use a more meaningful subject than "[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11" (I won't change it now as this might break the threading in email clients).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>On Behalf Of Tarun Khanna
>Sent: Tuesday, 21 April, 2020 16:44
>To: James Pustejovsky
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
>
>Thank you, James. I understand that the moments estimators might be more
>useful when working with small sample sizes. In my data set I have about 60
>studies and 110 effect sizes. So as such the dataset is not small. But I do
>want to estimate effect sizes for smaller sets of the data (there are
>multiple set of interventions which can be distinguished). In the smaller
>sets the number of effects decreases to as low as 15-30. In this context, I
>thought DL might be a better estimator. I will look into the robumeta
>package.
>
>I also have a theoretical question  around RVE. The estimates that I get for
>RVE have much higher standard errors compared to the DL/REML estimator. I
>understand that this is to be expected, RVE is also likely to result in
>higher Type I errors. Is there any way to control for that in the metafor
>package?
>
>Best
>Tarun
>
>Tarun Khanna
>PhD Researcher
>
>Hertie School
>
>Friedrichstraße 180
>10117 Berlin ∙ Germany
>khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-<http://www.hertie-school.org<http://www.hertie->
>school.org/>
>
>________________________________
>From: James Pustejovsky <jepusto using gmail.com>
>Sent: 21 April 2020 16:22:38
>To: Tarun Khanna
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11
>
>Hi Tarun,
>
>Good question! The Dersimonian-Laird variance estimator is in a general
>class of what are called moment estimators. In principle, it is possible to
>use moment estimation for models that are more complex than the basic meta-
>analysis/meta-regression model (as estimated with rma.uni, for instance). In
>fact, this is precisely what the robumeta package does. The correlated
>effects model and hierarchical effects model implemented in that package use
>moment estimators of the between-study variance component, and for the
>hierarchical model, also the within-study variance component. As far as I
>understand them, I think these estimators are not as precise as using
>ML/REML, and are mainly intended as way to get "quick-and-dirty" values for
>use in a working model (which need not be super accurate, if RVE is then
>used to get standard errors/confidence intervals).
>
>There has also been some statistical work on moment estimators for more
>complex multi-variate models:
>* Chen, H., Manning, A. K., & Dupuis, J. (2012). A method of moments
>estimator for random effect multivariate meta‐analysis. Biometrics, 68(4),
>1278-1284.
>I'm not sure if the methods described here are implemented in software
>though.
>
>Kind Regards,
>James

	[[alternative HTML version deleted]]