[R-meta] rma.mv for studies reporting composite of and/or individual subscales

[Viechtbauer, Wolfgang (SP)]

Sorry, I can't follow. What is 'standard view' and 'alternative view'? Those sound the same to me, except for different letters.

Best,
Wolfgang

>-----Original Message-----
>From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>Sent: Wednesday, 24 November, 2021 20:56
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: rma.mv for studies reporting composite of and/or individual
>subscales
>
>Appreciate it. Thank you very much. My response is below inline.
>
>Say there are also some studies that, for some reason, have broken
>down such a scale into a few subscales, say BDI1 and BDI2, and they do
>not report means and SDs for the overall BDI scale, only for these
>subscales.
>
>BDI is a mixture of BDI1 and BDI2 anyway, so if I only have BDI, then
>this is what the effect size reflects. If I include effect sizes based
>on BDI1 and BDI2 in the analysis, then the model essentially mixes
>them together.
>
>>>>>Sure, but, what if
>**one the one hand**: BDI *in standard view* is a mixture of A, B and
>C subscales and (1) some studies can mix and match them to create
>their unique composites (AB;  AC;  ABC), (2) some studies report some
>or all these subscales (A,B;  A,C;  A,B,C), and
>
>**on the other hand**: BDI *in alternative view* is a mixture of E, F
>and G subscales and (3) some studies can mix and match them to create
>their unique composites (EF;  EG;  EFG), and (4) some studies report
>some or all these subscales (E,F;  E,G;  E,F,G)?
>
>This is what is reflected in my data structure below (as mentioned
>earlier, the number of unique subscales is about the number of
>studies).
>
>Thanks, Tim M
>
>study subscale  reporting  obs  include
>1        A      subscale   1    yes
>1        A      subscale   2    yes
>1        B      subscale   3    yes
>1        B      subscale   4    yes
>2        A&C    composite  5    yes
>3        G&F    composite  6    yes
>4        E      subscale   7    yes
>4        F      subscale   8    yes
>4        E&F    composite  9    no
>
>On Wed, Nov 24, 2021 at 12:59 PM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>> Let me use a concrete example.
>>
>> Say I have studies assessing the effectiveness of a treatment on depression.
>Some studies report means and SDs of the treated and control groups for
>overall/composite scales such as the BDI, HAM-D, CES-D, and so on. For such a
>study, I would compute its effect size based on whatever scale it used.
>>
>> Studies may also have used multiple such scales. Then I would also compute
>multiple effect sizes, one per scale. Of course, I would then have to take the
>dependency of multiple effect sizes computed based on the same sample into
>consideration.
>>
>> Say there are also some studies that, for some reason, have broken down such a
>scale into a few subscales, say BDI1 and BDI2, and they do not report means and
>SDs for the overall BDI scale, only for these subscales.
>>
>> I would then compute effect sizes based on BDI1 and BDI2 and again, accounting
>for their dependency, include them in the same analysis as all of the above.
>>
>> I personally see no major issues with this. BDI is a mixture of BDI1 and BDI2
>anyway, so if I only have BDI, then this is what the effect size reflects. If I
>include effect sizes based on BDI1 and BDI2 in the analysis, then the model
>essentially mixes them together.
>>
>> Scales may also measure multiple inherently different types of outcomes, such
>as the HADS, which has subscales for anxiety and depression. Not sure if it
>common practice to ever report an overall mean for both of these outcome types
>together. If both outcome types are of interest (and not just depression), then I
>can again include both effect sizes (for depression and anxiety) in the same
>analysis (again, with their covariance, blah blah blah). Plus I'll need a
>moderator to distinguish the two outcome types. Not sure what I would do with a
>study that only reports an overall HADS score for the two groups (if this is ever
>done). I might still include this in the analysis and code the outcome type
>moderator with a third category for 'mixture'.
>>
>> If there are moderators that I want to examine, then I would be inclined to
>allow for separate relationships for different outcome types. I probably would
>not examine if the relationship differs for effect sizes that are based on
>subscales for the same outcome type versus effect sizes that are based on overall
>measures. Same goes with the random effects structure. But that would be my
>approach and one could of course separate things further.
>>
>> Best,
>> Wolfgang
>>
>> >-----Original Message-----
>> >From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>> >Sent: Wednesday, 24 November, 2021 19:36
>> >To: Viechtbauer, Wolfgang (SP)
>> >Cc: R meta
>> >Subject: Re: rma.mv for studies reporting composite of and/or individual
>> >subscales
>> >
>> >So, you think there is no need to keep everything (i.e., fixed and
>> >random) separate between studies that only contribute composite and
>> >studies that only contribute separate subscales?
>> >
>> >If there is no need, and both types of studies can be in one model,
>> >then methodologically, wouldn't it be mixing apples (different
>> >subscales) and oranges (different composites) in one model?
>> >
>> >Thanks,
>> >Tim M
>> >
>> >On Wed, Nov 24, 2021 at 12:22 PM Viechtbauer, Wolfgang (SP)
>> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >>
>> >> >-----Original Message-----
>> >> >From: Timothy MacKenzie [mailto:fswfswt using gmail.com]
>> >> >Sent: Wednesday, 24 November, 2021 19:18
>> >> >To: Viechtbauer, Wolfgang (SP)
>> >> >Cc: R meta
>> >> >Subject: Re: rma.mv for studies reporting composite of and/or individual
>> >> >subscales
>> >> >
>> >> >>rma.mv(es ~ reporting:X1, vi, random = list(~1| study, ~ reporting |
>> >> >>obs), struct = "DIAG", subset = include == "yes")
>> >> >
>> >> >Not sure what X1 is, but yes, this could be a plausible model,
>> >> >allowing for different within-study variances for 'subscale' versus
>> >> >'composite' estimates.
>> >> >
>> >> >>>>>X1 is a moderator but I think I should keep X1 separate between studies
>> >for
>> >> >which we have used their composite result and studies for which we have
>used
>> >> >their subscale results, no?
>> >>
>> >> That's up to you or one could empirically examine if the association between
>X1
>> >and es is different for the two types.
>> >>
>> >> Best,
>> >> Wolfgang