[R-meta] Multivariate meta regression and predict for robust estimates

[Reza Norouzian]

I guess I responded too quickly (1:30 am answer effect:). CORRECTION:

First, if your data is just like clubSandwich::SATcoaching, then yes
your current model works, as no repetition of the same levels of
outcome occurs.

Second, in my own second model, you can account for repetition of the
same levels of outcome by adding random row effects:

rma.mv(yi, V, random = list(~ outcome | study, ~ outcome |
interaction(study, group), ~1|row_id), struct = c("UN","UN"))

Now, this model will recognize the repetition of the same levels of outcome.

Sorry for the confusion,
Reza


On Thu, Oct 21, 2021 at 12:15 AM Reza Norouzian <rnorouzian using gmail.com> wrote:
>
> Dear Ivan,
>
> I leave question (B) to James or Wolfgang (or other list members).
> Regarding question (A), I discuss three situations.
>
> First, you current model assumes that in each study, the same levels
> of outcome don't repeat, something along the lines of:
>
> study  outcome
> 1      A
> 1      B
> 2      A
> 2      B
> 3      B
> 4      A
>
> If your data has the above structure, then your current model seems
> reasonable. It assumes that levels of outcome are correlated with one
> another in each study across all studies.
>
> Since you have assumed a UN structure and a V matrix, your more
> frequently occurring levels of outcome lend support to less frequently
> occurring levels of outcome thereby improving the fixed coefficients
> (in terms of bias) and the standard errors (in terms of magnitude) of
> the less frequently occurring levels of outcome.
>
> Second, if your data structure is more along the lines of:
>
> study group outcome
>     1     1       A
>     1     1       B
>     1     2       A
>     1     2       B
>     2     1       A
>     2     1       B
>     2     2       A
>     2     2       B
>     3     1       B
>     4     1       A
>
> That is, only due to a particular "coded for" design-related feature
> (e.g., some studies having more than one treatment group), you can
> have the same levels of outcome (e.g., A) repeated in some studies,
> then, you can try:
>
> rma.mv(yi, V, random = list(~ outcome | study, ~ outcome |
> interaction(study, group) struct = c("UN","UN"))
>
> Or simplify the `struct =` (perhaps to "HCS" in case of overparameterization).
>
> This second model assumes that in addition to the study-level
> correlations between the levels of outcome, we can have separate
> group-level correlations between the levels of outcome. This will then
> recognize the repetition of the same levels of outcome due to the
> existence of multi-group studies.
>
> A third situation might be that your data structure is exactly like
> above (i.e., the same levels of outcome repeat in some studies) but
> that you "haven't coded for" the design-related feature that has
> caused that repetition, that is:
>
> study outcome  row_id
>     1       A  1
>     1       B  2
>     1       A  3
>     1       B  4
>     2       A  5
>     2       B  6
>     2       A  7
>     2       B  8
>     3       B  9
>     4       A  10
>
> Then, you can try:
>
> rma.mv(yi, V, random = list(~ outcome | study, ~ 1| row_id, struct = "UN"))
>
> This last model shares the same assumption at the study-level with the
> previous models, but then it simply allows each level of outcome to be
> heterogeneous (have variation in it) accounting for the repetitions of
> the same level of outcome.
>
> Kind regards,
> Reza
>
>
>
> On Wed, Oct 20, 2021 at 10:46 PM Ivan Jukic <ivan.jukic using aut.ac.nz> wrote:
> >
> > Dear all,
> >
> > Let's say that one wants to perform a multivariate random-effects meta regression where the data structure can be described as follows: 1) There are 2 outcomes; 2) there is a continious moderator of interest; 3) all studies reported on both outcomes; and 4) most of the studies reported multiple effect sizes for at least one of the outcomes. This means that some participants, from certain groups and for a given outcome, provided data multiple times.
> >
> > Following the examples below (where 1 is extremely relevant)
> >
> > 1. https://www.jepusto.com/imputing-covariance-matrices-for-multi-variate-meta-analysis/
> > 2. http://www.metafor-project.org/doku.php/analyses:berkey1998
> > 3. https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000097.html
> >
> > I would specify the model as follows:
> >
> > res <- rma.mv(yi = yi,
> >                   V = V,
> >                   data = dat,
> >                   random = ~ outcome | study,
> >                   method = "REML",
> >                   test = "t",
> >                   slab = study,
> >                   struct = "UN",
> >                   mods = ~ mod1*outcome)
> >
> > A) I'm wondering if this would account for the fact that there are multiple effect sizes coming from the same study for a given outcome? In a "regular" multilevel model, I would typically have study/es_id.
> >
> > B) In addition, is anyone aware of the predict function that could be used with robust estimates (e.g., after using coef_test from clubSandwich package)? Predict.rma.mv works wonderfuly in combination with robust from metafor, but I would like to take the advantage of clubSandwich's "CR2" that should in principle lead to more accurate results in small samples.
> >
> > There is something similar that apparently works with robu package.
> > https://rdrr.io/github/zackfisher/robumeta/src/R/predict.robu.R
> >
> > Thank you for your time,
> > Ivan
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis