[R-meta] Adding random effect to SCE models

Jack Solomon kj@j@o|omon @end|ng |rom gm@||@com
Thu Aug 5 01:50:50 CEST 2021 

Dear Fred,

Just to clarify my answer in the post you linked, I misspoke when I
mentioned in brackets (what was a passing thought) the variance component
for "trt_grp" is non-interpretable (realized I had to edit that part out 1
minute later). You will not get "any" variance component for "trt_grp" at
all to worry about its interpretation.  You simply are using "trt_grp" (or
"grp" in your case) to make your grouping variable (i.e., what appears
to the right of `|`) more specific (e.g., combination of study-groups,
etc.) for the levels of the variable appearing to the left of `|` (often
taken to be the target of generalization).

Thus, you can't generalize beyond the levels of "trt_grp" or the like
unless you place "trt_grp" on the left of the `|`. To digress a bit, in the
general multilevel modeling literature, it is debated what really is
generalizable (the grouping variable, the varying intercepts/slopes, or the
predictions/estimates of random effects).

At any rate, we simply use an indexing variable (or what James referred to
as an ID variable) as a grouping variable in whatever combinations to make
our assumptions regarding true effects more specific. Remember that this
also means that we are adding more parameters to the model.

Thus, we may sometimes create but end up not using such ID variables if the
data structure or size (or both) don't support encoding such specific
assumptions about the random effects.

Just my two cents, James will have more to say, for sure,
Jack

ps. I'm developing an R function to make it visually possible to see
different study structures as well as their counts in
meta-analytic datasets and hope that such an approach would make the
modeling choices a tiny bit more clear (will share that in near future).

On Wed, Aug 4, 2021 at 12:42 PM Farzad Keyhan <f.keyhaniha using gmail.com> wrote:

> For sure, much appreciated. I'll follow your comments in that thread, then.
> Fred
>
> On Wed, Aug 4, 2021 at 11:47 AM James Pustejovsky <jepusto using gmail.com>
> wrote:
>
> > I see. To keep things organized, I'll offer some comments in the thread
> > that you linked above.
> >
> > On Wed, Aug 4, 2021 at 11:34 AM Farzad Keyhan <f.keyhaniha using gmail.com>
> > wrote:
> >
> >> Hi James,
> >>
> >> Thank you for your informative response. I also appreciate Jack
> Solomon's
> >> helpful off-list comments (they may be helpful to other list members as
> >> well).
> >>
> >> To pick up on your last point which was that leaving the SCE's original
> >> random specification as is would imply that "true effect sizes at
> different
> >> time points are *fully* correlated if they come from the same study and
> the
> >> same category", one concern I have, as noted in my original email, is
> that
> >> most of my studies have multiple treatment groups in them.
> >>
> >> I realized that this issue has recently come up on the list (
> >>
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-July/003019.html).
> >> My collaborators have only "for indexing purposes" distinguished between
> >> treatment groups in each study (variable "grp").
> >>
> >> The post linked in the previous paragraph says: "it may be more
> >> "realistic" to assume that true effect sizes at different time points
> are
> >> correlated IF they come from the same treatment group in each study;
> rather
> >> than simply each study as a whole disregarding the groups."
> >>
> >> I agree (and perhaps you do too?) with that statement. In the case of my
> >> SCE model, my "time" random effect, then, has to change from: "~ time |
> >> interaction(study, mod_cat)" to now: "~ time | interaction(study, grp,
> >> mod_cat)". But as the follow-up answer (
> >>
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-July/003020.html)
> >> also explains, this strategy will produce a perhaps non-interpretable
> >> variance component for "grp".
> >>
> >> So, I was wondering what your thoughts are on using this strategy (i.e.,
> >> adding an indexing variable like "grp" in the random specification of
> >> "time")?
> >>
> >> All the best,
> >> Fred
> >>
> >> On Tue, Aug 3, 2021 at 10:24 PM James Pustejovsky <jepusto using gmail.com>
> >> wrote:
> >>
> >>> Hi Farzad,
> >>>
> >>> It's definitely possible to add further levels of random effects within
> >>> each subgroup. (We didn't give examples of this in the paper due to
> space
> >>> constraints.)
> >>>
> >>> I think the specification that you wrote in your original email could
> >>> work with variety of different structures for the second set of random
> >>> effects. For instance, struct = "CS" would imply a simple random
> effect for
> >>> each study-category combination, struct = "HCS" would allow differing
> >>> variance by time-point, or struct = "AR" would allow for an
> auto-regressive
> >>> structure per time-point. The one drawback of these specifications is
> that
> >>> the variance of the random effects is assumed to be constant across
> >>> moderator categories. That might or might not be reasonable--it's a
> >>> question of empirical fit.
> >>>
> >>> Alternately, you could try something along the lines of
> >>> random = list(~ mod_cat | study, ~ mod_cat | interaction(study,time)),
> >>> struct = c("DIAG","DIAG")
> >>> which will give you a time-point specific random effect with a unique
> >>> variance component for every level of mod_cat.
> >>>
> >>> Alternately, you could just leave it as the SCE model. You wrote: "we
> >>> want to encode the assumption that the true effect sizes at different
> time
> >>> points are correlated if they come from the study and the same
> category."
> >>> The regular SCE model (using random = ~ mod_cat | study, struct =
> "DIAG")
> >>> already encodes that assumption. It implies that true effect sizes at
> >>> different time points are *fully* correlated if they come from the same
> >>> study and the same category, due to there being just one random effect
> per
> >>> study-category.
> >>>
> >>> Kind Regards,
> >>> James
> >>>
> >>> On Wed, Jul 21, 2021 at 3:42 PM Farzad Keyhan <f.keyhaniha using gmail.com>
> >>> wrote:
> >>>
> >>>> Dear James (and the List Member),
> >>>>
> >>>> First of all, what a useful paper (congrats!). My colleagues and I
> want
> >>>> to
> >>>> fit separate SCE models to a number of categorical moderators in our
> >>>> longitudinal studies (data structure is below, note that multiple rows
> >>>> with
> >>>> the same time indicator are due to multiple treatment groups in each
> >>>> study,
> >>>> not any additional outcome measures).
> >>>>
> >>>> In your paper, although you talk about adding random effects per data
> >>>> particulars, you seem to mainly mean that for your CHE model.
> >>>>
> >>>> In our SCE models, we want to encode the assumption that the true
> effect
> >>>> sizes at different time points are correlated if they come from the
> >>>> study
> >>>> and the same category (*a bit unsure whether being from the same
> >>>> category
> >>>> is quite needed*).
> >>>>
> >>>> (1) Does this assumption make sense given the purpose of the SCE model
> >>>> that you had in mind?
> >>>> (2) If yes, what choices of "struct=" would make sense (to be
> >>>> empirically
> >>>> confirmed)?
> >>>>
> >>>> ** mods = ~ mod_cat*time, random = list(~ mod_cat | study, ~ time |
> >>>> interaction(study,mod_cat)), struct = c("DIAG","???") **
> >>>>
> >>>> study mod_cat time
> >>>> 1       1             0
> >>>> 1       2             0
> >>>> 1       1             1
> >>>> 1       2             1
> >>>> 2       1             0
> >>>> 2       2             1
> >>>> 3       1             0
> >>>>
> >>>> Thanks,
> >>>> Fred
> >>>>
> >>>>         [[alternative HTML version deleted]]
> >>>>
> >>>> _______________________________________________
> >>>> R-sig-meta-analysis mailing list
> >>>> R-sig-meta-analysis using r-project.org
> >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >>>>
> >>>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]

More information about the R-sig-meta-analysis mailing list