# [R-meta] rma.mv-When a higher level can't be modeled because of one row

Farzad Keyhan |@keyh@n|h@ @end|ng |rom gm@||@com
Tue Nov 9 02:19:10 CET 2021

```For further clarity, each study uses only one level of the 10
available levels of "measure", except one study that uses two levels
of the 10 available levels of "measure".

The question is that if this one exceptional study was like all other
studies, then, we could use: ~ 1 | measure / study ...  .

But now because of this one exceptional study, we can't use: ~ 1 |
measure / study ... .

So, to use measure as a random factor what should we do? One
suggestion is to use: ~ 1 | study / measure ...  .

However, given that all studies, except one, agree with ~ 1 | measure
/ study..., is there a technical consequence to using ~ 1 | study /
measure?

Thank you for your help,
Fred

measure row   study   outcome
1             1          1       1
1             2          1       2

# 1              3          2       1 <--- measure on this row
2              4          2       1

1               5         3       1
1               6         3       2

2               7         4       1
9               8         5       2
9               9         6       4
...

On Mon, Nov 8, 2021 at 11:53 AM Farzad Keyhan <f.keyhaniha using gmail.com> wrote:
>
> Thanks, as I mentioned the following is just the structure. In the
> full dataset, "measure" has 10 unique levels. In 50 studies, only one
> "measure" is used for all rows of a study. But in one study two
> measures were used (this study has two rows, therefore, only one row
> of it has to change to allow one "measure" to take all its rows just
> like the other 50 studies).
>
> Does that make sense and does your advice still stand regarding either
> ignoring "measure" or using it as study/measure?
>
>    measure row   study   outcome
>    1             1          1       1
>    1             2          1       2
>
> # 1              3          2       1 <--- measure on this row
>    2              4          2       1
>
>    1               5         3       1
>    1               6         3       2
> ...
>
> On Mon, Nov 8, 2021 at 11:38 AM Viechtbauer, Wolfgang (SP)
> <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> >
> > You can compute CIs with confint() and look at the profile likelihoods with profile().
> >
> > Now I see that 'measure' has 10 different levels, not 2 (as I thought), so I am apparently not fully understanding the data structure.
> >
> > Best,
> > Wolfgang
> >
> > >-----Original Message-----
> > >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
> > >Sent: Monday, 08 November, 2021 18:03
> > >To: Viechtbauer, Wolfgang (SP)
> > >Cc: R meta
> > >Subject: Re: rma.mv-When a higher level can't be modeled because of one row
> > >
> > >This is excellent advice, exactly what I was looking for. However, how
> > >can an rma.mv() user find out if any of her/his variance estimates is
> > >poor? Is there anything we can extract or compute from a fitted
> > >object?
> > >
> > >random = ~1 | measure/study/esID
> > >
> > >            estim    sqrt  nlvls  fixed              factor
> > >sigma^2.1  0.0502  0.2242     10     no             measure  <-- this
> > >must be very poor then
> > >sigma^2.2  0.3016  0.5491     51     no       measure/study
> > >sigma^2.3  0.1547  0.3934    405     no  measure/study/esID
> > >
> > >On Mon, Nov 8, 2021 at 10:36 AM Viechtbauer, Wolfgang (SP)
> > ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > >>
> > >> On further though, I would then just ignore 'measure' altogether. Definitely do
> > >not use it at the top of a random effects hierarchy as that would induce possible
> > >dependence for all 499 rows falling under measure = 1 based on a very poor
> > >variance estimate.
> > >>
> > >> Best,
> > >> Wolfgang
> > >>
> > >> >-----Original Message-----
> > >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
> > >> >Sent: Monday, 08 November, 2021 17:13
> > >> >To: Viechtbauer, Wolfgang (SP)
> > >> >Cc: R meta
> > >> >Subject: Re: rma.mv-When a higher level can't be modeled because of one row
> > >> >
> > >> >The actual data has 500 rows. That is just the structure of the data
> > >> >that I showed. That is why I want to make sure if it is reasonable to
> > >> >ignore 499 rows that agree with "measure/study" and just base my
> > >> >random effect specification on one row that suggests "study/measure".
> > >> >
> > >> >In other words, I want to make sure my random-effects seem a bit more
> > >> >in line with the generality of my data rather than an exception that
> > >> >has occurred in just one row.
> > >> >
> > >> >Is there a consequence if I switch from "measure/study" to
> > >> >"study/measure" given this situation?
> > >> >
> > >> >On Mon, Nov 8, 2021 at 10:00 AM Viechtbauer, Wolfgang (SP)
> > >> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > >> >>
> > >> >> Like I said before, if this is really all the data, then I wouldn't do any
> > >of
> > >> >that, because this will be a way too complex model for so little data.
> > >> >>
> > >> >> >-----Original Message-----
> > >> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
> > >> >> >Sent: Monday, 08 November, 2021 16:51
> > >> >> >To: Viechtbauer, Wolfgang (SP)
> > >> >> >Cc: R meta
> > >> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of one row
> > >> >> >
> > >> >> >Sure, so, I shouldn't worry that all rows but one suggest
> > >> >> >"measure/study" and only because of that one exceptional row, do: "~ 1
> > >> >> >| study/measure/outcome" or "~ 1 | study/outcome/measure"?
> > >> >> >
> > >> >> >Fred
> > >> >> >
> > >> >> >On Mon, Nov 8, 2021 at 9:39 AM Viechtbauer, Wolfgang (SP)
> > >> >> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > >> >> >>
> > >> >> >> No, I meant using "~ 1 | study/measure/outcome" or "~ 1 |
> > >> >> >study/outcome/measure".
> > >> >> >>
> > >> >> >> Best,
> > >> >> >> Wolfgang
> > >> >> >>
> > >> >> >> >-----Original Message-----
> > >> >> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
> > >> >> >> >Sent: Monday, 08 November, 2021 16:22
> > >> >> >> >To: Viechtbauer, Wolfgang (SP)
> > >> >> >> >Cc: R meta
> > >> >> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of one
> > >row
> > >> >> >> >
> > >> >> >> >Thanks Wolfgang.
> > >> >> >> >
> > >> >> >> >Yes, this is just the data structure. Focusing on the "making
> > >> >> >> >[measure] nested within study" part of your suggestion, you mean in
> > >> >> >> >row # 3, I recode the "measure" value of 1 to 2, or even delete row #
> > >> >> >> >3 altogether, or "~1 | measure/study/outcome" by default will take
> > >> >> >> >care of making "measure" nested in study?
> > >> >> >> >
> > >> >> >> >Thank you,
> > >> >> >> >Fred
> > >> >> >> >
> > >> >> >> >On Mon, Nov 8, 2021 at 4:13 AM Viechtbauer, Wolfgang (SP)
> > >> >> >> ><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > >> >> >> >>
> > >> >> >> >> Dear Fred,
> > >> >> >> >>
> > >> >> >> >> I would consider using measure as a fixed effect or making it nested
> > >> >within
> > >> >> >> >study (or within outcome). But none of this might really be appropriate
> > >for
> > >> >a
> > >> >> >> >dataset this small (but I assume this was just constructed for
> > >illustrating
> > >> >> >your
> > >> >> >> >question).
> > >> >> >> >>
> > >> >> >> >> Best,
> > >> >> >> >> Wolfgang
> > >> >> >> >>
> > >> >> >> >> >-----Original Message-----
> > >> >> >> >> >From: Farzad Keyhan [mailto:f.keyhaniha using gmail.com]
> > >> >> >> >> >Sent: Friday, 05 November, 2021 2:59
> > >> >> >> >> >To: R meta
> > >> >> >> >> >Cc: Viechtbauer, Wolfgang (SP)
> > >> >> >> >> >Subject: Re: rma.mv-When a higher level can't be modeled because of
> > >one
> > >> >row
> > >> >> >> >> >
> > >> >> >> >> >For clarity, by a solution, I mean how can I account for the
> > >> >> >> >> >heterogeneity in true effects attributable to "measure", while
> > >> >> >> >> >"measure" is neither a perfect candidate for being the nestor of
> > >> >> >> >> >study:
> > >> >> >> >> >
> > >> >> >> >> >random = ~1 | measure/study/outcome
> > >> >> >> >> >
> > >> >> >> >> >nor a perfect candidate for being crossed with study:
> > >> >> >> >> >
> > >> >> >> >> >random = list(~1 | study/outcome, ~1|measure)
> > >> >> >> >> >
> > >> >> >> >> >Thank you,
> > >> >> >> >> >Fred
> > >> >> >> >> >
> > >> >> >> >> >On Thu, Nov 4, 2021 at 2:26 PM Farzad Keyhan <f.keyhaniha using gmail.com>
> > >> >wrote:
> > >> >> >> >> >>
> > >> >> >> >> >> Dear Experts,
> > >> >> >> >> >>
> > >> >> >> >> >> In my toy data below, if in row # 3, "measure" was 2 (instead of
> > >1),
> > >> >> >> >> >> then, I could take "measure" as a level higher than study:
> > >> >> >> >> >>
> > >> >> >> >> >> random = ~1 | measure/study/outcome
> > >> >> >> >> >>
> > >> >> >> >> >> But right now, because in study 2 (rows # 3 and 4) "measure" can
> > >vary,
> > >> >> >> >> >> "measure" can't be considered a level higher than study.
> > >> >> >> >> >>
> > >> >> >> >> >> On the other hand, because "measure" varies only in one study, I
> > >can't
> > >> >> >> >> >> take "measure" as a crossed random-effect either.
> > >> >> >> >> >>
> > >> >> >> >> >> I was wondering what solutions the expert list members might have
> > >for
> > >> >> >> >> >> this situation?
> > >> >> >> >> >>
> > >> >> >> >> >> Thanks,
> > >> >> >> >> >> Fred
> > >> >> >> >> >>
> > >> >> >> >> >>    measure       row   study   outcome
> > >> >> >> >> >>    1             1     1       1
> > >> >> >> >> >>    1             2     1       2
> > >> >> >> >> >> #  1             3     2       1 <--- measure on this row
> > >> >> >> >> >>    2             4     2       1
> > >> >> >> >> >>    1             5     3       1
> > >> >> >> >> >>    1             6     3       2

```