[R-meta] rma.mv: why some var components change but others don't across 2 models

[Stefanou Revesz]

Interesting, thanks! This possibility never came up on the list (at least
based on my thorough search).

This possibility make me wonder what methodological guidelines might be out
there regarding the pros and cons of using 'outcome' as a crossed random
effect vs. ~outcome | study, struct = 'UN' (or 'HCS').

Thank you for pointing me to the use of id,
Stefanou


On Tue, Nov 2, 2021, 8:09 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Yes, if the values of 'outcome' have inherent meaning, you can consider
> using it as a crossed random effect. That does not actually exclude the
> possibility of adding another random effect nested within studies, that is:
>
> random = list(~ 1 | study / id, ~ 1 | outcome, ~ 1 | measure)
>
> where 'id' is unique to every row in the dataset.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: Stefanou Revesz [mailto:stefanourevesz using gmail.com]
> >Sent: Tuesday, 02 November, 2021 13:58
> >To: Viechtbauer, Wolfgang (SP)
> >Cc: R meta
> >Subject: Re: rma.mv: why some var components change but others don't
> across 2
> >models
> >
> >Thanks. In my case, each 'outcome' means the same thing across the
> studies. I
> >take 'measure' as a crossed random effect, because I believe each
> 'measure' has
> >its own inherent characteristics (its own questioning style, its own
> length etc)
> >that affect effect sizes similarly in any study it has been used.
> >
> >Thus, by taking 'measure' as a crossed random-effect, I account for the
> >dependence in effect sizes attributed to the use of a common 'measure'
> *anywhere*
> >in the data.
> >
> >But I can say the same thing for 'outcome'. If each 'outcome' has an
> inherent
> >nature (math vs. history), then one can make the same argument that
> applied to
> >'measure', and use 'outcome' as a crossed random effect, no?
> >
> >(Or maybe, accounting for the within study heterogeneity due to the use of
> >different outcomes should still be preferred.)
> >
> >On Tue, Nov 2, 2021, 1:41 AM Viechtbauer, Wolfgang (SP)
> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >Unless the values of 'outcome' are meaningful and not just (essentially
> >arbitrary) values to distinguish different rows, using something like '~
> 1 |
> >outcome' makes no sense. For example, say the coding looks like this:
> >
> >study outcome yi vi
> >1     1       .  .
> >1     2       .  .
> >2     1       .  .
> >2     2       .  .
> >2     3       .  .
> >3     1       .  .
> >...
> >
> >'~ 1 | study / outcome' makes sense to allow for between- and within-study
> >heterogeneity. But unless a "1" for outcome in study 1 represents the
> same type
> >of outcome as "1" is study 2 and 3, 'list(~ 1 | study, ~ 1 | outcome')
> makes no
> >sense. If the numbers or values are only used to distinguish different
> outcomes
> >within the same study but carry no inherent meaning beyond that, then one
> could
> >just as well have coded the studies as:
> >
> >study outcome yi vi
> >1     1       .  .
> >1     2       .  .
> >2     3       .  .
> >2     4       .  .
> >2     5       .  .
> >3     6       .  .
> >...
> >
> >and '~ 1 | study / outcome' would give identical results to the previous
> coding,
> >but 'list(~ 1 | study, ~ 1 | outcome') would not. In fact, with the second
> >coding, '~ 1 | study / outcome' and 'list(~ 1 | study, ~ 1 | outcome') are
> >identical (because the second coding is implicitly creating the same
> nesting that
> >'~ 1 | study / outcome' implies).
> >
> >Regardless of the coding, '~ 1 | study / outcome' and '~ outcome | study'
> with
> >struct="CS" is identical (strictly speaking, the latter allows for a
> negative
> >correlation and if so, then the equivalence breaks down, but let's not
> get into
> >this). Structures like "HCS" and "UN" only make sense again when the
> values of
> >'outcome' are inherently meaningful and not just arbitrary identifiers.
> >
> >Best,
> >Wolfgang
> >
> >>-----Original Message-----
> >>From: Stefanou Revesz [mailto:stefanourevesz using gmail.com]
> >>Sent: Monday, 01 November, 2021 17:20
> >>To: Viechtbauer, Wolfgang (SP)
> >>Cc: R meta
> >>Subject: Re: rma.mv: why some var components change but others don't
> across 2
> >>models
> >>
> >>Thanks! Feel free to ignore this, but I don't think it has come up on
> >>the mailing list before.
> >>
> >>If I use: list(~ 1 | study, ~1|outcome, ~ 1 | measure), then
> >>everything else aside, it means I believe that there are inherent
> >>differences in 'outcome' that would necessitate disentangling
> >>'outcome' effects from those of study and measure (crossing outcome
> >>with study and measure).
> >>
> >>On the other hand, I can use list(~ outcome | study, ~ 1 | measure),
> >>struct="UN" which again adheres to the belief that there are inherent
> >>differences in 'outcome' without necessitating disentangling 'outcome'
> >>effects from those of study and measure (outcome nested in study).
> >>
> >>What's the difference between the two strategies above, and why I
> >>never see: list(~ 1 | study, ~1|outcome) in the archives (all I see is
> >>either '~1|study/outcome' or its multivariate reparametrization '~
> >>outcome | study'?
> >>
> >>Stefanou
>

	[[alternative HTML version deleted]]