[R-meta] complex data structure for a meta-analysis

[Viechtbauer Wolfgang (SP)]

1) You do have what I would call 'multivariate' data -- that is, multiple outcomes for the same sample. Since you say that you do not have the information needed in order to compute the covariances between multiple outcomes for the same sample, a possible strategy to account for the dependency is the use cluster robust inferences. This has been discussed at quite some length in previous posts on this mailing list, so I would encourage you to look through the archives.

2) Similarly, this has come up before. There is no general concensus on how I^2 can be extended to more complex models. I have written down some ideas here:

http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate

I cannot tell you whether this is the best/right way.

3) A standard funnel plot isn't wrong, it just does not give any indication of what points are independent vs dependent. I am not aware of any suggestions on how a funnel plot could be drawn that does provide such an indication (maybe connecting dependent estimates with lines?).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Yogev Kivity
Sent: Wednesday, 20 September, 2017 13:42
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] complex data structure for a meta-analysis

Hi everyone,I am running a meta-analysis using 'metafor' and I came across several questions that I could not find answers
for in 'metafor's documentation.

In short, were are examining psychotherapy data, and how a specific measure collected at the beginning of treatment (attachment style in relationships) predicts outcome of therapy as measured at post-treatment. Both measures are usually dimensional, so we are using Pearson's r which we then convert to Fisher's z.

The design of the meta-analysis is multilevel and multivariate in that each study usually includes several different treatment groups with different patients, as well as several subscales of attachment (e.g., level of anxiety in attachment and level of avoidance in attachment) and several measures of outcome at post-treatment (e.g., anxiety, depression etc.). This is complicated by the fact that studies rarely use the same attachment and outcome measures, and for the most part, we do not have data on the covariance among these measures.

I am assuming that our design is most similar to Konstantopoulos (2011), but we have an additional level of effect sizes repeated within groups, so basically we have multiple effect size per treatment arm, nested within treatment arm, which in are turn nested within study. Would that be correct?

My main questions are:
1. what would be the best approach for modeling all of these levels of analyses, while taking into account the fact that the effect sizes within treatment arm are likely no independent. My understanding is that usually multivariate is interpreted to mean multiple outcome measures, but in our case we have multiple outcome as well as multiple predictors.

2. How should I squared be calculated for such models?

3. is there an extension of funnel plots to multi-level models that could reliably represent the data? I guess that using the standard funnel plot ignores the mutilevel structure of the data, is that correct?

Best,
Yogev