[R-meta] rma.mv: why some var components change but others don't across 2 models

[Viechtbauer, Wolfgang (SP)]

Yes, if the values of 'outcome' have inherent meaning, you can consider using it as a crossed random effect. That does not actually exclude the possibility of adding another random effect nested within studies, that is:

random = list(~ 1 | study / id, ~ 1 | outcome, ~ 1 | measure)

where 'id' is unique to every row in the dataset.

Best,
Wolfgang

>-----Original Message-----
>From: Stefanou Revesz [mailto:stefanourevesz using gmail.com]
>Sent: Tuesday, 02 November, 2021 13:58
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: rma.mv: why some var components change but others don't across 2
>models
>
>Thanks. In my case, each 'outcome' means the same thing across the studies. I
>take 'measure' as a crossed random effect, because I believe each 'measure' has
>its own inherent characteristics (its own questioning style, its own length etc)
>that affect effect sizes similarly in any study it has been used.
>
>Thus, by taking 'measure' as a crossed random-effect, I account for the
>dependence in effect sizes attributed to the use of a common 'measure' *anywhere*
>in the data.
>
>But I can say the same thing for 'outcome'. If each 'outcome' has an inherent
>nature (math vs. history), then one can make the same argument that applied to
>'measure', and use 'outcome' as a crossed random effect, no?
>
>(Or maybe, accounting for the within study heterogeneity due to the use of
>different outcomes should still be preferred.)
>
>On Tue, Nov 2, 2021, 1:41 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>Unless the values of 'outcome' are meaningful and not just (essentially
>arbitrary) values to distinguish different rows, using something like '~ 1 |
>outcome' makes no sense. For example, say the coding looks like this:
>
>study outcome yi vi
>1     1       .  .
>1     2       .  .
>2     1       .  .
>2     2       .  .
>2     3       .  .
>3     1       .  .
>...
>
>'~ 1 | study / outcome' makes sense to allow for between- and within-study
>heterogeneity. But unless a "1" for outcome in study 1 represents the same type
>of outcome as "1" is study 2 and 3, 'list(~ 1 | study, ~ 1 | outcome') makes no
>sense. If the numbers or values are only used to distinguish different outcomes
>within the same study but carry no inherent meaning beyond that, then one could
>just as well have coded the studies as:
>
>study outcome yi vi
>1     1       .  .
>1     2       .  .
>2     3       .  .
>2     4       .  .
>2     5       .  .
>3     6       .  .
>...
>
>and '~ 1 | study / outcome' would give identical results to the previous coding,
>but 'list(~ 1 | study, ~ 1 | outcome') would not. In fact, with the second
>coding, '~ 1 | study / outcome' and 'list(~ 1 | study, ~ 1 | outcome') are
>identical (because the second coding is implicitly creating the same nesting that
>'~ 1 | study / outcome' implies).
>
>Regardless of the coding, '~ 1 | study / outcome' and '~ outcome | study' with
>struct="CS" is identical (strictly speaking, the latter allows for a negative
>correlation and if so, then the equivalence breaks down, but let's not get into
>this). Structures like "HCS" and "UN" only make sense again when the values of
>'outcome' are inherently meaningful and not just arbitrary identifiers.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Stefanou Revesz [mailto:stefanourevesz using gmail.com]
>>Sent: Monday, 01 November, 2021 17:20
>>To: Viechtbauer, Wolfgang (SP)
>>Cc: R meta
>>Subject: Re: rma.mv: why some var components change but others don't across 2
>>models
>>
>>Thanks! Feel free to ignore this, but I don't think it has come up on
>>the mailing list before.
>>
>>If I use: list(~ 1 | study, ~1|outcome, ~ 1 | measure), then
>>everything else aside, it means I believe that there are inherent
>>differences in 'outcome' that would necessitate disentangling
>>'outcome' effects from those of study and measure (crossing outcome
>>with study and measure).
>>
>>On the other hand, I can use list(~ outcome | study, ~ 1 | measure),
>>struct="UN" which again adheres to the belief that there are inherent
>>differences in 'outcome' without necessitating disentangling 'outcome'
>>effects from those of study and measure (outcome nested in study).
>>
>>What's the difference between the two strategies above, and why I
>>never see: list(~ 1 | study, ~1|outcome) in the archives (all I see is
>>either '~1|study/outcome' or its multivariate reparametrization '~
>>outcome | study'?
>>
>>Stefanou