[R-meta] Questions about fitting a multilevel meta-analysis model with effect sizes measured as different outcome

[Man Hey CHIU]

(It seems that I accidentally cancelled my last submission, so I resent this email. Sorry for the inconvenience caused.)

Dear Dr. Viechtbauer and All,

I am writing this letter to seek for your help on multilevel meta-analysis using the metafor package you have developed.

In our meta-analysis, we chose to conduct a mutilevel meta-analysis due to multiple effect sizes from a single study (i.e., dependence).

Thus, we are thinking about a 3-level random effect meta-analysis model with level 1 being sampling biases, level 2 being the variation of effect sizes measured as different outcomes within the same studies and level 3 being the variation between studies across the whole dataset.

During running the analyses, we have encountered a specific problem about the organization of the data file that is entered into the model. We have one column 'StudyName' representing each study, 'Outcome' representing each outcome within a study (p.s. different studies can have the same outcome) and 'EffectID' representing each unique effect size across all studies (p.s. if the whole dataset has 300 effect sizes, there would be 300 different EffectID.)

And I wonder whether the 'Outcome' (the first line of code below) or the 'EffectID' (the second line of code below) should be used to get our desired outcome:

(1) rma.mv(yi=Hedge.g, V = Variances, data=df, slab=StudyName, random = list(~1|Outcome, ~1|StudyName))

(2) rma.mv(yi=Hedge.g, V = Variances, data=df, slab=StudyName, random = list(~1|EffectID, ~1|StudyName))

I have read the demonstration of doing the multilevel meta-analysis on sample dataset dat.konstantopoulos2011, the 'school' column can have the same code which may be reused in different districts. Yet, actually, every school should be unique. In contrast, in my case, different studies can share the same outcomes. Since my case is different than the example that you showed, I am not so sure how should I fit my dataset to the model.

Another question would be whether the order of specific effect inside list() would matter. As the following two line of code would give the same result, how does the function know which level 2 and level 3 is? Since level 2 should be clustered in level 3, why don't we have to specify the level 2 and level 3?

The final question we have encountered would be about adding moderators to the above function. Since in multilevel-meta analysis, there might be covariates on the level 2 outcome and level 3 study respectively, e.g., similar outcome may be measured across different studies. Would the function automatically detect the covariate as a level 2 covariate when that value is different across the outcomes in a specific cluster (i.e. study) or do we have to specify the level of covariates?

Thank you in advance for your time and help, we really appreciate your input on these questions.

Regards,
Heidi Chiu

	[[alternative HTML version deleted]]