[R-meta] rma.mv for studies reporting composite of and/or individual subscales

[Viechtbauer, Wolfgang (SP)]

>-----Original Message-----
>From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>Sent: Wednesday, 24 November, 2021 17:10
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: rma.mv for studies reporting composite of and/or individual
>subscales
>
>I may have misspecified your suggested subgroup-ish model in my
>previous email, I think the model could have been:
>
>rma.mv(es ~ reporting:X1, vi, random = list(~1| study, ~ reporting |
>obs), struct = "DIAG", subset = include == "yes")

Not sure what X1 is, but yes, this could be a plausible model, allowing for different within-study variances for 'subscale' versus 'composite' estimates.

>Regardless, one possible downside to the subgroup model in my data is
>that it becomes a bit subjective how to treat studies that both
>provide separate subscales and one composite subscales results. One
>can use only their subscales and exclude their composite part or vice
>versa. Thus, such subjectivity may have a bearing on the results of
>the model estimates for each subgroup depending on how one treats (c)
>studies referenced in my first email.

Sure, but excluding studies that only report a composite is also a subjective decision.

>Thanks,
>Tim M
>
>On Wed, Nov 24, 2021 at 9:11 AM Timothy MacKenzie <fswfswt using gmail.com> wrote:
>>
>> Thank you so much Wolfgang!
>>
>> I would tend to use (a) and (b) and for studies in group (c), I would
>> either use an effect size computed based on the composite or the
>> effect sizes computed based on the subscales (but not both). I would
>> also code a moderator that indicates whether an effect size comes from
>> a subscale or a composite measure.
>>
>> >>>>You mean, for example, for this data, I should only 'include' the following
>rows?
>>
>> study subscale  reporting  obs  include
>> 1        A      subscale   1             yes
>> 1        A      subscale   2             yes
>> 1        B      subscale   3             yes
>> 1        B      subscale   4             yes
>> 2        A&C    composite  5         yes
>> 3        G&H    composite  6        yes
>> 4        Z      subscale   7             yes
>> 4        T      subscale   8             yes
>> 4        Z&T    composite  9         no
>>
>> Then, will my model be a subgroup model like the following?
>>
>> rma.mv(es ~ reporting:X1, random = list(~1 | study, ~ obs |
>> interaction(study, reporting) ), struct = "DIAG",  subset = include ==
>> "yes")
>>
>> If the above model is correct, I would assume it's not meaningful to
>> compare the fixed or random estimates for subscales with those for
>> composites?
>>
>> Also, I assume I shouldn't use 'subscale' in the random part because
>> the same subscales don't occur much across the studies, correct?
>>
>> Thank you very much,
>> Tim M
>>
>> On Wed, Nov 24, 2021 at 7:55 AM Viechtbauer, Wolfgang (SP)
>> <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >
>> > Dear Tim,
>> >
>> > Please see below for my responses.
>> >
>> > Best,
>> > Wolfgang
>> >
>> > >-----Original Message-----
>> > >From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>> > >Sent: Wednesday, 24 November, 2021 7:04
>> > >To: R meta
>> > >Cc: Viechtbauer, Wolfgang (SP)
>> > >Subject: rma.mv for studies reporting composite of and/or individual
>subscales
>> > >
>> > >Dear All,
>> > >
>> > >In my meta-analysis, I've faced two issues.
>> > >
>> > >First issue; each study can measure the same outcome using subscales
>> > >reported in the following ways:
>> > >
>> > >(a) Some studies report only separate subscales,
>> > >(b) Some studies report only composite of some subscales,
>> > >(c) Some studies report both composite of and separate subscales.
>> > >
>> > >Second issue; the same subscales don't quite occur across different
>> > >studies (indeed, the number of unique subscales is about the number of
>> > >studies).
>> > >
>> > >To tackle the first issue, can I include only studies that report
>> > >separate subscales from (a) and (c) studies?
>> >
>> > Sure you can. I don't think anybody here will come and stop you :)
>> >
>> > I would tend to use (a) and (b) and for studies in group (c), I would either
>use an effect size computed based on the composite or the effect sizes computed
>based on the subscales (but not both). For effect sizes computed based on
>separate subscales in the same sample, the dependency between the effect sizes
>needs to be take into consideration. I would also code a moderator that indicates
>whether an effect size comes from a subscale or a composite measure.
>> >
>> > >To tackle the second issue, can I only rely on the model below (data
>> > >structure is below)?
>> > >
>> > > rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = subscale  ==
>"subscale")
>> >
>> > I think you meant:
>> >
>> > rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = reporting == "subscale")
>> >
>> > You could do that if you only want to include effect sizes computed based on
>subscales. That would throw out studies 2 and 3. Poor studies 2 and 3 :(
>> >
>> > >Thank you,
>> > >Tim M
>> > >
>> > >My data looks like this (please view this in a plain text editor):
>> > >
>> > >study subscale  reporting  obs
>> > >1        A      subscale   1
>> > >1        A      subscale   2
>> > >1        B      subscale   3
>> > >1        B      subscale   4
>> > >2        A&C    composite  5
>> > >3        G&H    composite  6
>> > >4        Z      subscale   7
>> > >4        T      subscale   8
>> > >4        Z&T    composite  9