[R-meta] rma.mv: why some var components change but others don't across 2 models

[Stefanou Revesz]

Thanks. In my case, each 'outcome' means the same thing across the studies.
I take 'measure' as a crossed random effect, because I believe each
'measure' has its own inherent characteristics (its own questioning style,
its own length etc) that affect effect sizes similarly in any study it has
been used.

Thus, by taking 'measure' as a crossed random-effect, I account for the
dependence in effect sizes attributed to the use of a common 'measure'
*anywhere* in the data.

But I can say the same thing for 'outcome'. If each 'outcome' has an
inherent nature (math vs. history), then one can make the same argument
that applied to 'measure', and use 'outcome' as a crossed random effect,
no?

(Or maybe, accounting for the within study heterogeneity due to the use of
different outcomes should still be preferred.)



On Tue, Nov 2, 2021, 1:41 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Unless the values of 'outcome' are meaningful and not just (essentially
> arbitrary) values to distinguish different rows, using something like '~ 1
> | outcome' makes no sense. For example, say the coding looks like this:
>
> study outcome yi vi
> 1     1       .  .
> 1     2       .  .
> 2     1       .  .
> 2     2       .  .
> 2     3       .  .
> 3     1       .  .
> ...
>
> '~ 1 | study / outcome' makes sense to allow for between- and within-study
> heterogeneity. But unless a "1" for outcome in study 1 represents the same
> type of outcome as "1" is study 2 and 3, 'list(~ 1 | study, ~ 1 | outcome')
> makes no sense. If the numbers or values are only used to distinguish
> different outcomes within the same study but carry no inherent meaning
> beyond that, then one could just as well have coded the studies as:
>
> study outcome yi vi
> 1     1       .  .
> 1     2       .  .
> 2     3       .  .
> 2     4       .  .
> 2     5       .  .
> 3     6       .  .
> ...
>
> and '~ 1 | study / outcome' would give identical results to the previous
> coding, but 'list(~ 1 | study, ~ 1 | outcome') would not. In fact, with the
> second coding, '~ 1 | study / outcome' and 'list(~ 1 | study, ~ 1 |
> outcome') are identical (because the second coding is implicitly creating
> the same nesting that '~ 1 | study / outcome' implies).
>
> Regardless of the coding, '~ 1 | study / outcome' and '~ outcome | study'
> with struct="CS" is identical (strictly speaking, the latter allows for a
> negative correlation and if so, then the equivalence breaks down, but let's
> not get into this). Structures like "HCS" and "UN" only make sense again
> when the values of 'outcome' are inherently meaningful and not just
> arbitrary identifiers.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Stefanou Revesz [mailto:stefanourevesz using gmail.com]
> >Sent: Monday, 01 November, 2021 17:20
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: rma.mv: why some var components change but others don't
> across 2
> >models
> >
> >Thanks! Feel free to ignore this, but I don't think it has come up on
> >the mailing list before.
> >
> >If I use: list(~ 1 | study, ~1|outcome, ~ 1 | measure), then
> >everything else aside, it means I believe that there are inherent
> >differences in 'outcome' that would necessitate disentangling
> >'outcome' effects from those of study and measure (crossing outcome
> >with study and measure).
> >
> >On the other hand, I can use list(~ outcome | study, ~ 1 | measure),
> >struct="UN" which again adheres to the belief that there are inherent
> >differences in 'outcome' without necessitating disentangling 'outcome'
> >effects from those of study and measure (outcome nested in study).
> >
> >What's the difference between the two strategies above, and why I
> >never see: list(~ 1 | study, ~1|outcome) in the archives (all I see is
> >either '~1|study/outcome' or its multivariate reparametrization '~
> >outcome | study'?
> >
> >Stefanou
>

	[[alternative HTML version deleted]]