[R-sig-ME] Dependence of Effect Sizes in meta analysis metafor (some statistical theory questions included)

[Drwecki, Brian B]

Hello all,

I apologize for the long post, but I want to be thorough.

My Goal: To conduct the appropriate mixed-effects (random effects meta analysis model+ one fixed effects moderator with two categorical levels) meta analysis where 11 of 38 papers/studies present effects for both levels of my Fixed Effects moderator (i.e. these 11 studies provide 2 effect sizes each =  22 total estimates; each pair of effects is dependent and violates assumptions of independence).

Background Reading I've Done:  I read Modeling Dependent Effect Sizes With Three-Level Meta-Analyses:A Structural Equation Modeling Approach by Mike W.-L. Cheung and Three-level meta-analysis of dependent effect sizes
Wim Van den Noortgate & José Antonio López-López & Fulgencio Marín-Martínez & Julio Sánchez-Meca & http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011. Note: the last time I looked at meta analysis was 2006 and this was for a dumbed down version in a class, so a lot of the concepts are new to me, and I while I think I understand what's going on here, I'm sure there are areas that I am missing.

Explanation of my situation:  So, I have been gathering effects for a small meta analysis on the average correlation between self-esteem and academic achievement for African American students. I conceptualized self-esteem in two different ways (a priori): Global (I love myself) and Academic Specific (I'm good at school).  I have 39 different papers; 11 of these papers have both correlations (one for Global and one for Academic Specific; i.e. these effects are dependent not independent).  I want to run this analysis in one mixed effects model, where my moderator (TypeSE) is entered as a Fixed Effect, and where the error structure accounts for the dependence of the 11 studies (and 22 effects) that have measures at both levels of my moderator.

I spent a lot of time going through this example and adapting it to my situation http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011 only to read at the bottom of the article... "It is important to note that the models used above assume that the sampling errors of the effect size estimates are independent. This is typically an appropriate assumption as long as there is no overlap in the data/individuals used to compute the various estimates. However, when multiple estimates are obtained from the same group of individuals, then this assumption is most certainly violated."  But then I read Cheung's article that appears to suggest I could use the same type of 3-level meta analysis model to factor out dependence that occurs when multiple estimates are taken from the same study.  So, is the attached model doing what I think it is doing (creating a three level meta analysis model where the dependence between studies is being accounted for in the error structure)?  If not, then what is the appropriate model for my situation, in your opinion (using metafor)?

My METAFOR Model: resbasic6<-rma.mv(yi,vi,random=~factor(SampleID)|StudyID, mods =TypeSE, data=dat)
resbasic6 = nameholder
SampleID= unique number for all effects (so I have 58 effects total, and they are numbered 1-58; this was done in the original metaphor three level example referenced above)
StudyID = Identifier that denotes if two effects came from the same study
TypeSE = Fixed Effect Moderator (0 = Global; 1= Academic)

Additional Questions: Best textbook/paper/mook/webinar for understanding mixed effects meta analysis;  Same question, but mixed effects models in general (with a good focus on dealing with dependence in normal, not meta analysis, data sets).

Biggest Conceptual Difficulty I'm Having: I have difficulty translating the theoretical equations presented in the papers cited above into r code.

Thanks so much, and sorry for the long message!

Brian (ps my output is below)

Output if curious:

> resbasic6

Multivariate Meta-Analysis Model (k = 55; method: REML)

Variance Components:

outer factor: StudyID          (nlvls = 39)
inner factor: factor(SampleID) (nlvls = 55)

            estim    sqrt  fixed
tau^2      0.0085  0.0921     no
rho        0.3092             no

Test for Residual Heterogeneity:
QE(df = 53) = 162.1204, p-val < .0001

Test of Moderators (coefficient(s) 2):
QM(df = 1) = 37.1505, p-val < .0001

Model Results:

         estimate      se    zval    pval   ci.lb   ci.ub
intrcpt    0.1606  0.0234  6.8710  <.0001  0.1148  0.2064  ***
mods       0.1886  0.0309  6.0951  <.0001  0.1279  0.2492  ***

---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

>




	[[alternative HTML version deleted]]