[R-meta] rma.mv for studies reporting composite of and/or individual subscales

[Timothy MacKenzie]

Thank you so much Wolfgang!

I would tend to use (a) and (b) and for studies in group (c), I would
either use an effect size computed based on the composite or the
effect sizes computed based on the subscales (but not both). I would
also code a moderator that indicates whether an effect size comes from
a subscale or a composite measure.

>>>>You mean, for example, for this data, I should only 'include' the following rows?

study subscale  reporting  obs  include
1        A      subscale   1             yes
1        A      subscale   2             yes
1        B      subscale   3             yes
1        B      subscale   4             yes
2        A&C    composite  5         yes
3        G&H    composite  6        yes
4        Z      subscale   7             yes
4        T      subscale   8             yes
4        Z&T    composite  9         no

Then, will my model be a subgroup model like the following?

rma.mv(es ~ reporting:X1, random = list(~1 | study, ~ obs |
interaction(study, reporting) ), struct = "DIAG",  subset = include ==
"yes")

If the above model is correct, I would assume it's not meaningful to
compare the fixed or random estimates for subscales with those for
composites?

Also, I assume I shouldn't use 'subscale' in the random part because
the same subscales don't occur much across the studies, correct?

Thank you very much,
Tim M

On Wed, Nov 24, 2021 at 7:55 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
> Dear Tim,
>
> Please see below for my responses.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
> >Sent: Wednesday, 24 November, 2021 7:04
> >To: R meta
> >Cc: Viechtbauer, Wolfgang (SP)
> >Subject: rma.mv for studies reporting composite of and/or individual subscales
> >
> >Dear All,
> >
> >In my meta-analysis, I've faced two issues.
> >
> >First issue; each study can measure the same outcome using subscales
> >reported in the following ways:
> >
> >(a) Some studies report only separate subscales,
> >(b) Some studies report only composite of some subscales,
> >(c) Some studies report both composite of and separate subscales.
> >
> >Second issue; the same subscales don't quite occur across different
> >studies (indeed, the number of unique subscales is about the number of
> >studies).
> >
> >To tackle the first issue, can I include only studies that report
> >separate subscales from (a) and (c) studies?
>
> Sure you can. I don't think anybody here will come and stop you :)
>
> I would tend to use (a) and (b) and for studies in group (c), I would either use an effect size computed based on the composite or the effect sizes computed based on the subscales (but not both). For effect sizes computed based on separate subscales in the same sample, the dependency between the effect sizes needs to be take into consideration. I would also code a moderator that indicates whether an effect size comes from a subscale or a composite measure.
>
> >To tackle the second issue, can I only rely on the model below (data
> >structure is below)?
> >
> > rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = subscale  == "subscale")
>
> I think you meant:
>
> rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = reporting == "subscale")
>
> You could do that if you only want to include effect sizes computed based on subscales. That would throw out studies 2 and 3. Poor studies 2 and 3 :(
>
> >Thank you,
> >Tim M
> >
> >My data looks like this (please view this in a plain text editor):
> >
> >study subscale  reporting  obs
> >1        A      subscale   1
> >1        A      subscale   2
> >1        B      subscale   3
> >1        B      subscale   4
> >2        A&C    composite  5
> >3        G&H    composite  6
> >4        Z      subscale   7
> >4        T      subscale   8
> >4        Z&T    composite  9