[R-meta] rma.mv for studies reporting composite of and/or individual subscales

[Viechtbauer, Wolfgang (SP)]

Dear Tim,

Please see below for my responses.

Best,
Wolfgang

>-----Original Message-----
>From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>Sent: Wednesday, 24 November, 2021 7:04
>To: R meta
>Cc: Viechtbauer, Wolfgang (SP)
>Subject: rma.mv for studies reporting composite of and/or individual subscales
>
>Dear All,
>
>In my meta-analysis, I've faced two issues.
>
>First issue; each study can measure the same outcome using subscales
>reported in the following ways:
>
>(a) Some studies report only separate subscales,
>(b) Some studies report only composite of some subscales,
>(c) Some studies report both composite of and separate subscales.
>
>Second issue; the same subscales don't quite occur across different
>studies (indeed, the number of unique subscales is about the number of
>studies).
>
>To tackle the first issue, can I include only studies that report
>separate subscales from (a) and (c) studies?

Sure you can. I don't think anybody here will come and stop you :)

I would tend to use (a) and (b) and for studies in group (c), I would either use an effect size computed based on the composite or the effect sizes computed based on the subscales (but not both). For effect sizes computed based on separate subscales in the same sample, the dependency between the effect sizes needs to be take into consideration. I would also code a moderator that indicates whether an effect size comes from a subscale or a composite measure.

>To tackle the second issue, can I only rely on the model below (data
>structure is below)?
>
> rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = subscale  == "subscale")

I think you meant:

rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = reporting == "subscale")

You could do that if you only want to include effect sizes computed based on subscales. That would throw out studies 2 and 3. Poor studies 2 and 3 :(

>Thank you,
>Tim M
>
>My data looks like this (please view this in a plain text editor):
>
>study subscale  reporting  obs
>1        A      subscale   1
>1        A      subscale   2
>1        B      subscale   3
>1        B      subscale   4
>2        A&C    composite  5
>3        G&H    composite  6
>4        Z      subscale   7
>4        T      subscale   8
>4        Z&T    composite  9