[R-meta] Multivariate (multi-outcomes) meta-analysis

[James Pustejovsky]

HI Maciek,

It sounds like you have a data structure where estimating a full
multivariate model would be pretty challenging. With 21 distinct indices, a
fully multivariate model has 21 mean parameters (average effect size for
each index), 21 variances (one for each index), and 210 (= 21 choose 2)
correlation parameters. (If you used the reduced set of 11 indices, there
would still be 55 correlation parameters to estimate!) Even if you had data
for every index for each of your 110 studies, these model would be
difficult to estimate and I don't think I'd put much stock in the results.

I can think of three routes you might take here:

1) Respond to the reviewer that the multivariate model is a good idea in
principle, but is infeasible for the data you're working with (due to the
above).

2) Find a way to estimate a more constrained version of the multivariate
model. I'm not familiar with the mixmeta syntax, so I'll stick with
metafor. For instance, you could fit a model that has the same correlation
between the random effects for every pair of indices (struct = "HCS" in
metafor::rma.mv()). A step further would be to also constrain the variances
to be identical for every pair of indices (struct = "CS" in metafor::rma.mv()).
Or perhaps there is some intermediate structure that you could motivate
based on what you know about the different physiological indices.

3) Fit the univariate models for each index, but estimate them within one
larger "meta-model" so that you can make statistical comparisons between
the average effects for different indices. I describe this strategy in this
working paper, under the name of the "sub-group correlated effects" working
model: https://osf.io/preprints/metaarxiv/vyfcj/

In addition to these suggestions, you might find the following paper by
Richard Riley to be helpful:
https://doi.org/10.1111/j.1467-985X.2008.00593.x

Kind Regards,
James

On Wed, Feb 24, 2021 at 5:12 AM Maciej Behnke <macbehnke using gmail.com> wrote:

> Dear All,
>
> I would like to run a meta-analysis for the effects of emotion on
> physiological indexes reactivity (e.g., heart rate, blood pressure, skin
> temperature, etc.). My initial idea was to run series of univariate
> multilevel MA, for each pair of emotions and each physiological index (in
> sum, 21 physio indexes). However, in the rejection note from the major
> psychological journal,  the reviewers suggested that emotions are
> multivariate phenomena – when you experience strong emotions you display
> the physiological reactivity indexed by multiple (multivariate) indexes
> which are correlated. Thus, reviewers suggested using the multivariate MA
> approach to account for the relations between physiological indexes.
> Although I believe it is a great suggestion, it is hard to implement to my
> dataset. I found 3 main issues that raise my doubts about the superiority
> of the multivariate MA approach.
>
> 1)      To implement the multivariate MA approach, I would need to know the
> correlation matrix between the physio indexes. However, only 7 out of 110
> papers reported some correlations; thus, I would need to estimate the
> correlations based on data I collected in my lab during other experiments
> (a new limitation for the study).
>
> 2)      I have tried to implement a multivariate MA approach using mixmeta
> R package. Unfortunately, in my dataset, there are many situations in which
> two or more physio indexes were never observed jointly. Thus, I inputted
> missing values to base some estimations entirely on the indirect
> comparison. With this approach, I was able to build the multivariate model
> for 11 physio indexes.  However, adding more indexes caused that
> convergence not reached after maximum number of iterations or there are
> additional errors due to the missing values. Thus, I was not able to create
> the model for all physio indexes.
>
> 3)      There are only minimal differences between the conclusions from the
> univariate and multivariate models that I was able to pull from my dataset.
>
> In sum, do you know any approach that I could implement to my data? Or
> would you suggest addressing these issues as the limitations of the
> univariate multilevel MA approach that I originally implemented?
>
> Best,
> Maciek
>
> Maciej Behnke
> Manager of Psychophysiology and Health Lab
> Faculty of Psychology and Cognitive Science
> Adam Mickiewicz University
> 89 Szamarzewskiego
> <https://maps.google.com/?q=89+Szamarzewskiego&entry=gmail&source=g>
> Street
> PL-60-568, Poznan, Poland
> Tel. +48 725 859 990
> E-mail: macbeh using amu.edu.pl
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]