[R-meta] rma.mv for studies reporting composite of and/or individual subscales

[Timothy MacKenzie]

I may have misspecified your suggested subgroup-ish model in my
previous email, I think the model could have been:

rma.mv(es ~ reporting:X1, vi, random = list(~1| study, ~ reporting |
obs), struct = "DIAG", subset = include == "yes")

Regardless, one possible downside to the subgroup model in my data is
that it becomes a bit subjective how to treat studies that both
provide separate subscales and one composite subscales results. One
can use only their subscales and exclude their composite part or vice
versa. Thus, such subjectivity may have a bearing on the results of
the model estimates for each subgroup depending on how one treats (c)
studies referenced in my first email.

Thanks,
Tim M





On Wed, Nov 24, 2021 at 9:11 AM Timothy MacKenzie <fswfswt using gmail.com> wrote:
>
> Thank you so much Wolfgang!
>
> I would tend to use (a) and (b) and for studies in group (c), I would
> either use an effect size computed based on the composite or the
> effect sizes computed based on the subscales (but not both). I would
> also code a moderator that indicates whether an effect size comes from
> a subscale or a composite measure.
>
> >>>>You mean, for example, for this data, I should only 'include' the following rows?
>
> study subscale  reporting  obs  include
> 1        A      subscale   1             yes
> 1        A      subscale   2             yes
> 1        B      subscale   3             yes
> 1        B      subscale   4             yes
> 2        A&C    composite  5         yes
> 3        G&H    composite  6        yes
> 4        Z      subscale   7             yes
> 4        T      subscale   8             yes
> 4        Z&T    composite  9         no
>
> Then, will my model be a subgroup model like the following?
>
> rma.mv(es ~ reporting:X1, random = list(~1 | study, ~ obs |
> interaction(study, reporting) ), struct = "DIAG",  subset = include ==
> "yes")
>
> If the above model is correct, I would assume it's not meaningful to
> compare the fixed or random estimates for subscales with those for
> composites?
>
> Also, I assume I shouldn't use 'subscale' in the random part because
> the same subscales don't occur much across the studies, correct?
>
> Thank you very much,
> Tim M
>
> On Wed, Nov 24, 2021 at 7:55 AM Viechtbauer, Wolfgang (SP)
> <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >
> > Dear Tim,
> >
> > Please see below for my responses.
> >
> > Best,
> > Wolfgang
> >
> > >-----Original Message-----
> > >From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
> > >Sent: Wednesday, 24 November, 2021 7:04
> > >To: R meta
> > >Cc: Viechtbauer, Wolfgang (SP)
> > >Subject: rma.mv for studies reporting composite of and/or individual subscales
> > >
> > >Dear All,
> > >
> > >In my meta-analysis, I've faced two issues.
> > >
> > >First issue; each study can measure the same outcome using subscales
> > >reported in the following ways:
> > >
> > >(a) Some studies report only separate subscales,
> > >(b) Some studies report only composite of some subscales,
> > >(c) Some studies report both composite of and separate subscales.
> > >
> > >Second issue; the same subscales don't quite occur across different
> > >studies (indeed, the number of unique subscales is about the number of
> > >studies).
> > >
> > >To tackle the first issue, can I include only studies that report
> > >separate subscales from (a) and (c) studies?
> >
> > Sure you can. I don't think anybody here will come and stop you :)
> >
> > I would tend to use (a) and (b) and for studies in group (c), I would either use an effect size computed based on the composite or the effect sizes computed based on the subscales (but not both). For effect sizes computed based on separate subscales in the same sample, the dependency between the effect sizes needs to be take into consideration. I would also code a moderator that indicates whether an effect size comes from a subscale or a composite measure.
> >
> > >To tackle the second issue, can I only rely on the model below (data
> > >structure is below)?
> > >
> > > rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = subscale  == "subscale")
> >
> > I think you meant:
> >
> > rma.mv(es ~ 1, random = ~ 1 | study / obs, subset = reporting == "subscale")
> >
> > You could do that if you only want to include effect sizes computed based on subscales. That would throw out studies 2 and 3. Poor studies 2 and 3 :(
> >
> > >Thank you,
> > >Tim M
> > >
> > >My data looks like this (please view this in a plain text editor):
> > >
> > >study subscale  reporting  obs
> > >1        A      subscale   1
> > >1        A      subscale   2
> > >1        B      subscale   3
> > >1        B      subscale   4
> > >2        A&C    composite  5
> > >3        G&H    composite  6
> > >4        Z      subscale   7
> > >4        T      subscale   8
> > >4        Z&T    composite  9