[R-meta] 3 candidate random structures

[Timothy MacKenzie]

Dear James and Jack,

Thank you both for your help and insights.

Many thanks,
Tim

On Mon, Aug 16, 2021 at 12:08 PM Jack Solomon <kj.jsolomon using gmail.com> wrote:

> Dear Tim,
>
> To your second question (using struct="GEN"), I believe at least `random =
> ~ treat_length + outcome | study, struct = "GEN"` (using `treat_length` as
> a control variable) is supported by rma.mv.
>
> You can also then turn your correlation estimates (note
> that struct="GEN" considers a semi-definite positive structure for the
> var-covar matrix of random slopes and intercepts, kind of "UN"ish) into
> regression coefficients in which case you essentially can perform a latent
> regression (i.e., regression on random effects using any one of them as DV
> and one or more of them as IVs).
>
> I'm a bit new to rma.mv and I'm still exploring it myself, but given that
> the behavior of random effects under struct="GEN" is apparently similar to
> the conventional multilevel models, such a regression can be performed
> perhaps using something along the lines of:
> https://github.com/rnorouzian/e/blob/master/e.r#L264 .
>
> Then, you can probably also perform statistical significance on the
> resultant regression coefficients using some form of bootstrapping like:
> https://github.com/rnorouzian/e/blob/master/e.r#L303 .
>
> Generally, such structures help bringing continuous variables more into
> the discussion of multilevel meta-regression models which IMHO offers more
> opportunities to ask and answer novel questions in meta-analysis. The
> trouble, though, as always, is the size of the data needed to support such
> models.
>
> Definitely, Wolfgang has more to say about this, as struct="GEN" is as of
> yet not documented.
>
> Just my two cents,
> Jack
>
> On Mon, Aug 16, 2021 at 9:44 AM James Pustejovsky <jepusto using gmail.com>
> wrote:
>
>> Responses below.
>>
>> James
>>
>> On Sat, Aug 14, 2021 at 10:28 PM Timothy MacKenzie <fswfswt using gmail.com>
>> wrote:
>>
>> > Dear James,
>> >
>> > Thank you, this is very helpful to know. Is there currently a way to
>> > statistically test individual correlations between outcome levels
>> (assuming
>> > a "UN" structure, say in outcome | study, where the outcome has 3
>> levels)?
>> >
>>
>> Yes. This can be tested using a likelihood ratio test comparing the full
>> model to a model with one (or more) of the correlations fixed to specified
>> values. You can fix these values using the rho argument in rma.mv. See
>> the
>> section "Fixing Variance Components and/or Correlations" of ?rma.mv.
>>
>>
>> >
>> > Also, do you think that struct="GEN" might also allow such correlations
>> to
>> > be investigated with more thoroughly? For example, if I specify my
>> random
>> > part as `random = ~ treat_length * outcome | study, struct = "GEN"`
>> would
>> > that allow understanding for how the correlations among outcome levels
>> > change for various `treat_length`?
>> >
>> >
>> Does that syntax even work? I'm not sure how you would interpret it.
>>
>>
>> > Also, in your hypothetical D specification, you suggested `~ outcome |
>> > interaction(study,gr,time)` as one of the terms, that had me wondering
>> why
>> > you took `outcome` to be nested in `time` (I always thought the other
>> way
>> > around i.e., time being nested in outcome).
>> >
>> >
>> Either way (outcomes nested in timepoints or timepoints nested in
>> outcomes)
>> entails some simplifying assumptions. It seems more plausible that there
>> would be some structured correlation between all of the effect sizes
>> within
>> a given study (including correlation between effects from different
>> outcomes at different time points) but I don't think it's possible to fit
>> something like that with rma.mv().
>>
>>
>> > Kind regards,
>> > Tim
>> >
>> > On Sat, Aug 14, 2021 at 9:37 PM James Pustejovsky <jepusto using gmail.com>
>> > wrote:
>> >
>> >> See responses below.
>> >> Cheers,
>> >> James
>> >>
>> >> On Fri, Aug 6, 2021 at 12:27 PM Timothy MacKenzie <fswfswt using gmail.com>
>> >> wrote:
>> >>
>> >>> Dear James,
>> >>>
>> >>> I seem to have forgotten to answer your question at the start of your
>> >>> answer. Yes, my outcomes are comparable across my studies. However, I
>> have
>> >>> no intention of generalizing beyond my outcome levels. This is
>> because the
>> >>> levels of my outcome correspond to a specific theory in my area of
>> research
>> >>> and can't be beyond what the theory describes.
>> >>>
>> >>
>> >> This makes sense. In multivariate models like these, generalization is
>> to
>> >> a (hypothetical) population of the units on the right-hand side of the
>> | in
>> >> the random formula---that is, the units corresponding to the IDs---not
>> to
>> >> the levels of the outcomes. For instance, say that the outcome
>> variable has
>> >> levels A, B, C. If you use random = ~ outcome | study, then the model
>> is
>> >> describing the multivariate distribution of the outcomes (the joint
>> >> distribution of A, B, C) in a population of studies, of which you have
>> a
>> >> sample. Some studies in the sample might report a only subset of
>> outcomes
>> >> (only A, or only A and B), but we could imagine that all of the
>> outcomes
>> >> *could* have been measured in every study.
>> >>
>> >>
>> >>>
>> >>> However, I want to take a somewhat multivariate approach and let my
>> >>> outcome levels correlate with one another across my studies because I
>> >>> actually want to investigate the interrelationships among the existing
>> >>> levels of my outcome themselves.
>> >>>
>> >>> That's a good reason to use a multivariate model.
>> >>
>> >>
>> >>> Given this context, can I ignore the generalizability aspect of the
>> >>> multilevel/multivariate approach, and instead take this approach
>> because it
>> >>> allows for the correlation among the existing outcome levels to be
>> >>> investigated?
>> >>>
>> >>>>
>> >>>>>>
>> >> Yes, I think so.
>> >>
>> >
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

	[[alternative HTML version deleted]]