[R-meta] complex data structure for a meta-analysis

[Yogev Kivity]

Dear Wolfgang and all,
Thanks a lot for your detailed reply - this is very helpful! I have read the posts in the links and I feel like I have a better sense of the issues I am dealing with now. I do have several outstanding issues that I am hoping to get peoples’ opinion on.
1. Yes, my data are indeed multivariate, but I think it is also multilevel, because most studies had multiple groups (i.e. treatment arms) nested within the study. We are interested in examining the correlation between attachment style and treatment outcome within each group, and then as a second step to examine whether the effect size differs by treatment type (group level moderator). In addition, there is substantial variability across studies in the number of outcome measures (e.g., some report only one while others report 4-5) as well as the construct being measured (e.g., some measure depression and anxiety, while others measure functioning and quality of life). These issues raise several questions:
a. is there a way to account for the dependency between groups (in addition to simply including group as a level in the model)?
b. Say I choose to specify a ‘guestimated’ var-cov matrix to account for the dependencies among sampling variances, is it ok to have a different number of outcome measures, and different types of outcome measures in each study? The Berkey et al., 1998 study, for example, had the same number and type of outcome measures for all studies.
2. regarding I^2 – again, in my case (see above) I have both multilevel and multivariate data so I am not sure if the approaches mentioned in the link may be appropriate for my data. In general, I am interested in a general I^2 across outcome measures, and not necessarily by measure.

Thanks in advance,Yogev



--

Yogev Kivity, Ph.D.
Postdoctoral Fellow
Department of Psychology
The Pennsylvania State University
Bruce V. Moore Building
University Park, PA 16802
Office Phone: (814) 867-2330

 

    On Saturday, September 23, 2017, 5:51:56 PM EDT, Viechtbauer Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:  
 
 1) You do have what I would call 'multivariate' data -- that is, multiple outcomes for the same sample. Since you say that you do not have the information needed in order to compute the covariances between multiple outcomes for the same sample, a possible strategy to account for the dependency is the use cluster robust inferences. This has been discussed at quite some length in previous posts on this mailing list, so I would encourage you to look through the archives.

2) Similarly, this has come up before. There is no general concensus on how I^2 can be extended to more complex models. I have written down some ideas here:

http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate

I cannot tell you whether this is the best/right way.

3) A standard funnel plot isn't wrong, it just does not give any indication of what points are independent vs dependent. I am not aware of any suggestions on how a funnel plot could be drawn that does provide such an indication (maybe connecting dependent estimates with lines?).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Yogev Kivity
Sent: Wednesday, 20 September, 2017 13:42
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] complex data structure for a meta-analysis

Hi everyone,I am running a meta-analysis using 'metafor' and I came across several questions that I could not find answers
for in 'metafor's documentation.

In short, were are examining psychotherapy data, and how a specific measure collected at the beginning of treatment (attachment style in relationships) predicts outcome of therapy as measured at post-treatment. Both measures are usually dimensional, so we are using Pearson's r which we then convert to Fisher's z.

The design of the meta-analysis is multilevel and multivariate in that each study usually includes several different treatment groups with different patients, as well as several subscales of attachment (e.g., level of anxiety in attachment and level of avoidance in attachment) and several measures of outcome at post-treatment (e.g., anxiety, depression etc.). This is complicated by the fact that studies rarely use the same attachment and outcome measures, and for the most part, we do not have data on the covariance among these measures.

I am assuming that our design is most similar to Konstantopoulos (2011), but we have an additional level of effect sizes repeated within groups, so basically we have multiple effect size per treatment arm, nested within treatment arm, which in are turn nested within study. Would that be correct?

My main questions are:
1. what would be the best approach for modeling all of these levels of analyses, while taking into account the fact that the effect sizes within treatment arm are likely no independent. My understanding is that usually multivariate is interpreted to mean multiple outcome measures, but in our case we have multiple outcome as well as multiple predictors.

2. How should I squared be calculated for such models?

3. is there an extension of funnel plots to multi-level models that could reliably represent the data? I guess that using the standard funnel plot ignores the mutilevel structure of the data, is that correct?

Best,
Yogev  
	[[alternative HTML version deleted]]