[R-meta] rma.mv-When a higher level can't be modeled because of one row

[Viechtbauer, Wolfgang (SP)]

You can compute CIs with confint() and look at the profile likelihoods with profile().

Now I see that 'measure' has 10 different levels, not 2 (as I thought), so I am apparently not fully understanding the data structure.

Best,
Wolfgang

>-----Original Message-----
>From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
>Sent: Monday, 08 November, 2021 18:03
>To: Viechtbauer, Wolfgang (SP)
>Cc: R meta
>Subject: Re: rma.mv-When a higher level can't be modeled because of one row
>
>This is excellent advice, exactly what I was looking for. However, how
>can an rma.mv() user find out if any of her/his variance estimates is
>poor? Is there anything we can extract or compute from a fitted
>object?
>
>random = ~1 | measure/study/esID
>
>            estim    sqrt  nlvls  fixed              factor
>sigma^2.1  0.0502  0.2242     10     no             measure  <-- this
>must be very poor then
>sigma^2.2  0.3016  0.5491     51     no       measure/study
>sigma^2.3  0.1547  0.3934    405     no  measure/study/esID
>
>On Mon, Nov 8, 2021 at 10:36 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>>
>> On further though, I would then just ignore 'measure' altogether. Definitely do
>not use it at the top of a random effects hierarchy as that would induce possible
>dependence for all 499 rows falling under measure = 1 based on a very poor
>variance estimate.
>>
>> Best,
>> Wolfgang
>>
>> >-----Original Message-----
>> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
>> >Sent: Monday, 08 November, 2021 17:13
>> >To: Viechtbauer, Wolfgang (SP)
>> >Cc: R meta
>> >Subject: Re: rma.mv-When a higher level can't be modeled because of one row
>> >
>> >The actual data has 500 rows. That is just the structure of the data
>> >that I showed. That is why I want to make sure if it is reasonable to
>> >ignore 499 rows that agree with "measure/study" and just base my
>> >random effect specification on one row that suggests "study/measure".
>> >
>> >In other words, I want to make sure my random-effects seem a bit more
>> >in line with the generality of my data rather than an exception that
>> >has occurred in just one row.
>> >
>> >Is there a consequence if I switch from "measure/study" to
>> >"study/measure" given this situation?
>> >
>> >On Mon, Nov 8, 2021 at 10:00 AM Viechtbauer, Wolfgang (SP)
>> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >>
>> >> Like I said before, if this is really all the data, then I wouldn't do any
>of
>> >that, because this will be a way too complex model for so little data.
>> >>
>> >> >-----Original Message-----
>> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
>> >> >Sent: Monday, 08 November, 2021 16:51
>> >> >To: Viechtbauer, Wolfgang (SP)
>> >> >Cc: R meta
>> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of one row
>> >> >
>> >> >Sure, so, I shouldn't worry that all rows but one suggest
>> >> >"measure/study" and only because of that one exceptional row, do: "~ 1
>> >> >| study/measure/outcome" or "~ 1 | study/outcome/measure"?
>> >> >
>> >> >Fred
>> >> >
>> >> >On Mon, Nov 8, 2021 at 9:39 AM Viechtbauer, Wolfgang (SP)
>> >> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >> >>
>> >> >> No, I meant using "~ 1 | study/measure/outcome" or "~ 1 |
>> >> >study/outcome/measure".
>> >> >>
>> >> >> Best,
>> >> >> Wolfgang
>> >> >>
>> >> >> >-----Original Message-----
>> >> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
>> >> >> >Sent: Monday, 08 November, 2021 16:22
>> >> >> >To: Viechtbauer, Wolfgang (SP)
>> >> >> >Cc: R meta
>> >> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of one
>row
>> >> >> >
>> >> >> >Thanks Wolfgang.
>> >> >> >
>> >> >> >Yes, this is just the data structure. Focusing on the "making
>> >> >> >[measure] nested within study" part of your suggestion, you mean in
>> >> >> >row # 3, I recode the "measure" value of 1 to 2, or even delete row #
>> >> >> >3 altogether, or "~1 | measure/study/outcome" by default will take
>> >> >> >care of making "measure" nested in study?
>> >> >> >
>> >> >> >Thank you,
>> >> >> >Fred
>> >> >> >
>> >> >> >On Mon, Nov 8, 2021 at 4:13 AM Viechtbauer, Wolfgang (SP)
>> >> >> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> >> >> >>
>> >> >> >> Dear Fred,
>> >> >> >>
>> >> >> >> I would consider using measure as a fixed effect or making it nested
>> >within
>> >> >> >study (or within outcome). But none of this might really be appropriate
>for
>> >a
>> >> >> >dataset this small (but I assume this was just constructed for
>illustrating
>> >> >your
>> >> >> >question).
>> >> >> >>
>> >> >> >> Best,
>> >> >> >> Wolfgang
>> >> >> >>
>> >> >> >> >-----Original Message-----
>> >> >> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
>> >> >> >> >Sent: Friday, 05 November, 2021 2:59
>> >> >> >> >To: R meta
>> >> >> >> >Cc: Viechtbauer, Wolfgang (SP)
>> >> >> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of
>one
>> >row
>> >> >> >> >
>> >> >> >> >For clarity, by a solution, I mean how can I account for the
>> >> >> >> >heterogeneity in true effects attributable to "measure", while
>> >> >> >> >"measure" is neither a perfect candidate for being the nestor of
>> >> >> >> >study:
>> >> >> >> >
>> >> >> >> >random = ~1 | measure/study/outcome
>> >> >> >> >
>> >> >> >> >nor a perfect candidate for being crossed with study:
>> >> >> >> >
>> >> >> >> >random = list(~1 | study/outcome, ~1|measure)
>> >> >> >> >
>> >> >> >> >Thank you,
>> >> >> >> >Fred
>> >> >> >> >
>> >> >> >> >On Thu, Nov 4, 2021 at 2:26 PM Farzad Keyhan <f.keyhaniha using gmail.com>
>> >wrote:
>> >> >> >> >>
>> >> >> >> >> Dear Experts,
>> >> >> >> >>
>> >> >> >> >> In my toy data below, if in row # 3, "measure" was 2 (instead of
>1),
>> >> >> >> >> then, I could take "measure" as a level higher than study:
>> >> >> >> >>
>> >> >> >> >> random = ~1 | measure/study/outcome
>> >> >> >> >>
>> >> >> >> >> But right now, because in study 2 (rows # 3 and 4) "measure" can
>vary,
>> >> >> >> >> "measure" can't be considered a level higher than study.
>> >> >> >> >>
>> >> >> >> >> On the other hand, because "measure" varies only in one study, I
>can't
>> >> >> >> >> take "measure" as a crossed random-effect either.
>> >> >> >> >>
>> >> >> >> >> I was wondering what solutions the expert list members might have
>for
>> >> >> >> >> this situation?
>> >> >> >> >>
>> >> >> >> >> Thanks,
>> >> >> >> >> Fred
>> >> >> >> >>
>> >> >> >> >>    measure       row   study   outcome
>> >> >> >> >>    1             1     1       1
>> >> >> >> >>    1             2     1       2
>> >> >> >> >> #  1             3     2       1 <--- measure on this row
>> >> >> >> >>    2             4     2       1
>> >> >> >> >>    1             5     3       1
>> >> >> >> >>    1             6     3       2