[R-meta] 3 candidate random structures

[Timothy MacKenzie]

Dear James,

I seem to have forgotten to answer your question at the start of your
answer. Yes, my outcomes are comparable across my studies. However, I have
no intention of generalizing beyond my outcome levels. This is because the
levels of my outcome correspond to a specific theory in my area of research
and can't be beyond what the theory describes.

However, I want to take a somewhat multivariate approach and let my outcome
levels correlate with one another across my studies because I actually want
to investigate the interrelationships among the existing levels of my
outcome themselves.

Given this context, can I ignore the generalizability aspect of the
multilevel/multivariate approach, and instead take this approach because it
allows for the correlation among the existing outcome levels to be
investigated?

Much appreciated,
Tim

On Thu, Aug 5, 2021 at 10:40 AM Timothy MacKenzie <fswfswt using gmail.com> wrote:

> Dear James,
>
> This is great, thank you very much!
>
> Tim
>
> On Wed, Aug 4, 2021 at 2:47 PM James Pustejovsky <jepusto using gmail.com>
> wrote:
>
>> Hi Tim,
>>
>> In your data, do the outcome numbers correspond to categories that are
>> comparable across studies? In other words, does outcome = 1 in study 1 mean
>> the same thing as outcome = 1 in study 2? Or, alternately, are they just ID
>> variables, with arbitrary labels that could be re-ordered within each study
>> without losing any information?
>>
>> The reason I ask is that your first set of random effects (~ outcome |
>> study) would only make sense if the outcome variable corresponds to
>> categories that are comparable across studies.
>>
>> To question about which specification is most appropriate for the data
>> structure, I'll comment below.
>>
>> Specification A: list(~ outcome | study, ~ time |
>> interaction(study,outcome)), struct=c("HCS","HAR")
>>
>> This specification includes correlated random effects for each
>> study-outcome, which have a common influence across groups and time-points.
>> It also includes auto-regressive random effects across time-points for each
>> study-outcome combination. Since gr does not appear in the random effects
>> ID terms, the assumption is that the auto-regressive random effects have a
>> common influence across groups within each study. That seems a bit
>> restrictive.
>>
>> Specification B:   list(~ outcome | study, ~ time |
>> interaction(study,gr,outcome)), struct=c("HCS","HAR")
>>
>> This specification includes correlated random effects for each
>> study-outcome, which have a common influence across groups and time-points.
>> It includes auto-regressive random effects across time-points for each
>> study-outcome -group combination. The structure of the auto-regressive
>> random effects is assumed to be common across study-outcome-group
>> combinations. That might be reasonable, but given that you're allowing the
>> study-level random effects to have a unique variance for each outcome, it
>> does lead one to wonder why the time-point random effects are treated as
>> homogeneous across outcomes.
>>
>> Specification C:   list(~ outcome | interaction(study,gr), ~ time |
>> interaction(study,gr,outcome)), struct =c("HCS","HAR")
>>
>> This specification includes correlated random effects for each
>> study-outcome-group, which have a common influence across time-points. It
>> includes auto-regressive random effects across time-points for each
>> study-outcome -group combination. The structure of the auto-regressive
>> random effects is assumed to be common across study-outcome-group
>> combinations. Relative to B, specification C seems less reasonable as a
>> starting point because it does not allow for study-level variation that is
>> common across groups.
>>
>> Specification D:   list(~ outcome | study, ~ outcome |
>> interaction(study,gr), ~ outcome | interaction(study,gr,time)), struct
>> =c("HCS","HCS","HCS")
>>
>> I've added this specification as a further hypothetical
>> alternative---hypothetical because it's not actually possible to fit it
>> with rma.mv(), due to the constraint that you can only have two sets of
>> random effects with a term on the left hand side of the bar. But
>> hypothetically, this might be another parameterization to consider. It
>> includes correlated random effects for each study-outcome, each
>> study-group-outcome, and each study-group-outcome-timepoint, capturing the
>> idea of a (multivariate) outcome, nested within time-points, nested within
>> groups, nested within studies. It might be simplified in various ways, such
>> as by using ~ 1 | instead of ~ outcome | at one or more levels.
>>
>> In any case, as you noted in your query, ultimately these different
>> specifications should be investigated empirically.
>>
>> Cheers,
>> James
>>
>> On Wed, Jul 28, 2021 at 11:12 AM Timothy MacKenzie <fswfswt using gmail.com>
>> wrote:
>>
>>> Dear Colleagues,
>>>
>>> I just wanted to clarify that I fully understand that the 3 models that I
>>> suggested need to be empirically tested with my data. However, my
>>> question
>>> is that:
>>>
>>> Given that I'm new to the `rma.mv()` function in metafor, I was
>>> wondering
>>> if the candidate syntax that I have used (below) matches the general
>>> definition of my data structure (i.e., studies>groups>outcomes>time)?
>>>
>>> Thank you for your assistance, Tim
>>>
>>> list(~ outcome | study, ~ time | interaction(study,outcome)), struct
>>> =c("HCS","HAR")
>>>
>>> list(~ outcome | study, ~ time | interaction(study,gr,outcome)), struct
>>> =c("HCS","HAR")
>>>
>>> list(~ outcome | interaction(study,gr), ~ time |
>>> interaction(study,gr,outcome)), struct =c("HCS","HAR")
>>>
>>> # Data structure:
>>> study gr time outcome
>>> 1       1   0     1
>>> 1       1   0     2
>>> 1       1   1     1
>>> 1       1   1     2
>>> 1       1   2     1
>>> 1       1   2     2
>>> 1       2   0     1
>>> 1       2   0     2
>>> 1       2   1     1
>>> 1       2   1     2
>>> 1       2   2     1
>>> 1       2   2     2
>>> 2       1   0     1
>>> 2       1   0     2
>>> 2       2   0     1
>>> 2       2   0     2
>>>
>>> On Mon, Jul 26, 2021 at 1:06 PM Timothy MacKenzie <fswfswt using gmail.com>
>>> wrote:
>>>
>>> > Dear Meta-Analysis Experts,
>>> >
>>> > Several of my studies (15 out of 50) have a data structure similar to
>>> what
>>> > I'm showing below. That is, each such study has multiple experimental
>>> > groups ("gr"), tested several times, on 2 or more outcomes.
>>> >
>>> > I'm considering 3 different random structures (below). But I was
>>> wondering
>>> > which one better aligns to such studies' data structure?
>>> >
>>> > Thank you for your expertise,
>>> > Tim
>>> >
>>> > list(~ outcome | study, ~ time | interaction(study,outcome)), struct
>>> > =c("HCS","HAR")
>>> >
>>> > list(~ outcome | study, ~ time | interaction(study,gr,outcome)), struct
>>> > =c("HCS","HAR")
>>> >
>>> > list(~ outcome | interaction(study,gr), ~ time |
>>> > interaction(study,gr,outcome)), struct =c("HCS","HAR")
>>> >
>>> > # Data structure:
>>> > study gr time outcome
>>> > 1       1   0     1
>>> > 1       1   0     2
>>> > 1       1   1     1
>>> > 1       1   1     2
>>> > 1       1   2     1
>>> > 1       1   2     2
>>> > 1       2   0     1
>>> > 1       2   0     2
>>> > 1       2   1     1
>>> > 1       2   1     2
>>> > 1       2   2     1
>>> > 1       2   2     2
>>> > 2       1   0     1
>>> > 2       1   0     2
>>> > 2       2   0     1
>>> > 2       2   0     2
>>> >
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
>>

	[[alternative HTML version deleted]]