[R-meta] Dependent Measure Modelling Question

[James Pustejovsky]

Grace,

It sounds like the data that you're describing has two factors, emotion
type and task type, and that both are within-study factors (in other words,
a given study might report results for multiple emotion types and/or
multiple task types). Are the emotion types and task types also measured
within-participant, such that a given participant in a study gets assessed
with multiple task types, on multiple emotion types, or both? If so, then
one challenge in analyzing this data structure is that the effect size
estimates will be statistically dependent. There are several ways to handle
this (multivariate meta-analysis, robust variance estimation), which we've
discussed in many previous posts on the listserv.

Other than this issue, it sounds to me like it would be possible
to  analyze both factors---emotion type and task type---together in one big
model. The major advantage of doing so is that the joint model would let
you examine differences in emotion types *while controlling for task
types*, as well as examining differences in task types *while controlling
for emotion types*. Controlling for the other factor (and maybe other
covariates that are associated with effect size magnitude) should provide
clearer, more interpretable results for differences on a given factor.
There is also evidence that using a multivariate meta-analysis model can
potentially mitigate outcome reporting bias to some extent (see Kirkham,
Riley, & Williamson, 2012; Hwang & Desantis, 2018).

A further advantage of using one big model is that it would let you adjust
for other potential moderators that might have similar associations for
each emotion type and each task type. If you conduct separate analyses for
each emotion type (for example), you would have to analyze these moderators
separately, so you'd end up with 6 sets of moderator analyses instead of
just one.

The main challenge in the "one big meta-analysis model" approach is that it
requires careful checking of the model's assumptions. For example, you
would need to assess whether the between-study heterogeneity is similar
across the six emotion types and, if not, fit a model that allows for
different heterogeneity levels for each emotion.

James


Hwang, H., & DeSantis, S. M. (2018). Multivariate network meta‐analysis to
mitigate the effects of outcome reporting bias. *Statistics in medicine*.

Kirkham, J. J., Riley, R. D., & Williamson, P. R. (2012). A multivariate
meta‐analysis approach for reducing the impact of outcome reporting bias in
systematic reviews. *Statistics in medicine*, *31*(20), 2179-2195.

On Mon, Feb 11, 2019 at 3:16 AM Grace Hayes <grace.hayes3 using myacu.edu.au>
wrote:

> Hi all,
>
>
> I have a question regarding a meta-analysis of multiple dependent outcomes
> that I would like to conduct using metafor.
>
>
> For this meta-analysis of emotion recognition in ageing, I'm interested in
> age-effects (young adults vs. older adults) on four different emotion
> recognition tasks (Task A, Task B, Task C, Task D). Studies in this area
> typically compare older adults' performance to younger adults' performance
> on more than one of these emotion recognition task.
>
>
> For each task there are also multiple outcomes.  Each task produces an
> accuracy age-effect for each emotion type included (I.e., anger, sadness,
> fear). Up to 6 different emotions are included (Emotion 1, Emotion 2,
> Emotion 3, Emotion 4, Emotion 5, Emotion 6). I therefore have some studies
> with, for example, 6 different age-effects from 3 different emotions tasks;
> a total of 18 dependent outcomes.
>
>
> Ideally I would like to investigate age-effects for each of the six
> emotion types seperately (with Tasks A, B, C and D combined), and
> age-effects for each task type seperately (with Emotions 1-6 combined). I
> would then like to compare the effects for each emotion type (Emotions 1-6
> separately) produced by each task  (Measure A, B, C, D separately).
>
>
> My question is, can I have a model that analyses emotion type and task
> type all together? Is this possible and statistically appropriate? Will it
> tell me the age-effects produced for each emotion by each task, or will it
> only tell me if task type and emotion type are significant moderators?
>
>
> I am also interested to know if I can add additional moderators such as
> number of emotions included in the task and year of publication?
>
>
> One concern that has been brought to my attention is overfitting from too
> many factors. Another is that output would be difficult too interpret, and
> thus it has been recommended that I perhaps run separately analyses for
> each task.
>
>
> Any advice would be much appreciated.
>
>
> Sincerely,
>
> Grace Hayes
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]