[R-meta] nesting an inner | outer formula

Ross Neville ro@@@nev|||e @end|ng |rom ucd@|e
Tue Feb 18 13:40:51 CET 2025 

Thanks Wolfgang
I will take some time to digest the contents. I appreciate the detail and
guidance.
Quickly, the suggested model provides a sigma^2.2 variance of 0, as shown
below.
This makes me feel like the random effects random = ~ 1 | StudyID /
TreatmentGroup / Informant are too complex for the available data.
Thought on that?
[image: Screenshot 2025-02-18 at 12.38.17.png]

On Tue, 18 Feb 2025 at 12:34, Viechtbauer, Wolfgang (NP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Ross,
>
> Thanks for the clarification. Based on this, I would consider the
> following structure:
>
> random = ~ 1 | StudyID / TreatmentGroup / Informant
>
> This captures overall differences in the outcomes (across the experimental
> and control groups and across all informants) between studies. It also
> allows for heterogeneity in how much experimental and control groups differ
> from each other across studies (I assume you will add something like mods =
> ~ TreatmentGroup to the model, since presumably you are interested in the
> size of the (average) difference between the two groups). And it allows for
> heterogeneity in outcomes that arises due to some studies using multiple
> informants. By nesting Informant within TreatmentGroup, this model
> automatically implies a certain degree of correlation in the true outcomes
> for different informants within experimental/control groups. This last
> random effect is also the 'outcome level' random effect, since based on
> your description, every combination of StudyID, TreatmentGroup, and
> Informant should yield a unique value for each row.
>
> Strictly speaking, this structure does not capture the correlation in the
> sampling errors that arises because multiple informants are reporting on
> the *same children*. If reports, say from parents and teachers, are
> correlated, then this implies that the sampling errors of the means are
> also correlated. Such a correlation (or more precisely, the covariance)
> should go into the V matrix. However, to compute this covariance, you would
> need to know what the correlation (r) between the parent and teacher
> reports. This is probably not reported, but maybe can be guestimated from
> other or your own studies. Say the data for the first two studies looks
> like this:
>
> Study  Group  Informant   Outcome
> 1      Exp    Child       .
> 1      Ctrl   Child       .
> 2      Exp    Parent      .
> 2      Exp    Teacher     .
> 2      Ctrl   Parent      .
> 2      Ctrl   Teacher     .
>
> Then the corresponding V matrix would be (use a fixed width font to view
> this so that things are lined up properly):
>
> [s_1E^2/n_1E
>               ]
> [            s_1C^2/n_1C
>               ]
> [                        s_2EP^2/n_2E r*s_2EP*s_2ET/n_2E
>               ]
> [                                     s_2ET^2/n_2E
>               ]
> [                                                        s_2CP^2/n_2C
> r*s_2CP*s_2CT/n_2C]
> [
>  s_2CT^2/n_2C      ]
>
> where s stands for standard deviation, n for sample size, and the
> subscripts are for study, group, and informant in that order (single letter
> abbreviations). Elements left blank are equal to 0 (zero covariance). I did
> not put a subscript on r since this is probably a single guestimated value
> across all studies (and informant pairs). Such a V matrix can be easily
> generated using the vcalc() function.
>
> Since V is probably just going to be an approximation (especially if you
> decide not to bother creating the V matrix, which in essence implies
> assuming r=0), I would then consider using cluster-robust inference methods
> (robust(model, cluster=StudyID, clubSandwich=TRUE)) at least as a
> sensitivity check.
>
> I think the above is a good and sensible starting point. A more complex
> structure might be:
>
> dat$StudyID.TreatmentGroup <- paste0(dat$StudyID, ".", dat$TreatmentGroup)
> random = list(~ TreatmentGroup | StudyID, ~ Informant |
> StudyID.TreatmentGroup), struct="UN")
>
> This would allow for different variances for experimental and control
> groups and it would allow for different variances for the different types
> of informants and allow the correlation in the random effects to differ
> depending on the type of informant pair (child-parent, child-teacher,
> parent-teacher, etc.). But I would only attempt to fit this model if there
> is plenty of data. One could use a LRT to compare the two model structures.
> There is also a structure with intermediate complexity by using
> struct=c("UN","HCS"), where we assume different variances for the different
> informants, but a single correlation irrespective of the pair.
>
> One might also consider TreatmentGroup and Informant to be crossed random
> effects (within studies), but I think this is overcomplicating things.
>
> Best,
> Wolfgang
>
> > -----Original Message-----
> > From: Ross Neville <ross.neville using ucd.ie>
> > Sent: Friday, February 14, 2025 14:32
> > To: Viechtbauer, Wolfgang (NP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl>
> > Cc: r-sig-meta-analysis using r-project.org
> > Subject: Re: nesting an inner | outer formula
> >
> > Dear Wolfgang
> >
> > Thanks for the speedy response, and for seeking clarification and
> correcting my
> > error.
> >
> > Rather than try to correct my interpretation of the SAS code, perhaps it
> would
> > make more sense to tell you what structure I really want.
> >
> > The data structure is such that I have studies (StudyID) reporting post-
> > intervention means for children in an experimental and control group
> > (TreatmentGroup). The variable Informant tells us who is reporting on
> behalf of
> > the child. Some studies have child report only (so two rows for such a
> StudyID
> > corresponding to the post-intervention means for the experimental and
> control
> > group). Studies with parent report or teacher report only are the same
> (two
> > rows). There are also studies where there is data from child and parent,
> child
> > and teacher, teacher and parent, or even child parent and teacher. So,
> > essentially, a StudyID could have two rows, four rows, or six rows,
> depending on
> > how many Informants there are. Children are in the experimental or
> control group
> > only, so one would expect Informants to be nested and correlated within
> > TreatmentGroup within studies.
> >
> > Because of the data structure (multiple rows of sample means rather than
> fewer
> > rows of pairwise comparisions) the degree to which control and
> intervention
> > group means are more or less similar in a given StudyID is capture in
> part
> > (maybe even large part) by ~ 1 | StudyID.
> >
> > What is missing from this random = list (~ 1 | StudyID, ~ Informant |
> > TreatmentGroup) is the fact that the inner | outer is saying, for
> example, that
> > parents in the experimental or control group across studies share
> correlated
> > random effects. When, in fact, one would expect parents, children, and
> teachers
> > in the experimental or control group to share correlated random effects
> within a
> > given study.
> >
> > Perhaps given the data structure, you would advise something else. Or
> perhaps I
> > am still being unclear in my description and understanding.
> >
> > Regards
> > Ross
> >
> > On Fri, 14 Feb 2025 at 13:00, Viechtbauer, Wolfgang (NP)
> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> > Dear Ross,
> >
> > my proc mixed knowledge is a bit rusty, but unless I am confused, your
> proc
> > mixed statement specifies a random intercept for StudyID and an UN
> structure for
> > Informant within StudyID allowing for different variances/covariances
> for the
> > different levels of TreatmentGroup.
> >
> > > random StudyID;
> > > random Informant / subject=StudyID group=TreatmentGroup type=un;
> > > parms  1  1 1 1  1 1 1  1 1 1  1 1 1  1 / hold=14;
> >
> > I don't think this quite matches up with your description:
> >
> > > I would like for the different levels of Informant to be correlated
> within
> > > TreatmentGroup within StudyID, and I would like the different levels of
> > > TreatmentGroup to be correlated within StudyID too.
> >
> > For example, there is nothing in your proc mixed statement that allows
> for
> > "TreatmentGroup to be correlated within StudyID". Also, the second random
> > statement allows for Informant to be correlated within StudyID, but *not*
> > "within TreatmentGroup within StudyID".
> >
> > So before I attempt to recreate the same structure, it would need to be
> clear
> > exactly what kind of structure you really want.
> >
> > Best,
> > Wolfgang
> >
> > > -----Original Message-----
> > > From: Ross Neville <mailto:ross.neville using ucd.ie>
> > > Sent: Thursday, February 13, 2025 18:28
> > > To: Viechtbauer, Wolfgang (NP)
> > <mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>;
> > > mailto:r-sig-meta-analysis using r-project.org
> > > Subject: nesting an inner | outer formula
> > >
> > > Hi Wolfgang
> > >
> > > I hope this email finds you well
> > >
> > > I was wondering if you could tell me whether the following list of
> random
> > > effects can be updated to make the the inner | outer formula
> conditional.
> > >
> > > random = list (~ 1 | StudyID, ~ Informant | TreatmentGroup)
> > >
> > > I would like for the different levels of Informant to be correlated
> within
> > > TreatmentGroup within StudyID, and I would like the different levels of
> > > TreatmentGroup to be correlated within StudyID too.
> > >
> > > In SAS Proc Mixed, I've managed to run this model and I want to
> replicate it
> > in
> > > metafor http://rma.mv.
> > >
> > > random StudyID;
> > > random Informant/subject=StudyID, group=TreatmentGroup type=un;
> > > parms  1  1 1 1  1 1 1  1 1 1  1 1 1  1/hold=14;
> > >
> > > Any help you can provide to let me know if this is possible in
> http://rma.mv
> > > would be much appreciated.
> > >
> > > Regards
> > > Ross
> > >
> > > --
> > > Dr Ross D. Neville, PhD, ProfCert University Teaching and Learning
> > > Head of Subject - Sport Management
> > > School of Public Health, Physiotherapy and Sport Science
> > > University College Dublin (UCD)
> > > Room G6 - Woodview House
> > > Belfield, Dublin 4
> > > mailto:mailto:ross.neville using ucd.ie
> > > +353 (0) 1 716 3419
>


-- 
Dr Ross D. Neville, PhD, ProfCert University Teaching and Learning
Head of Subject - Sport Management
School of Public Health, Physiotherapy and Sport Science
University College Dublin (UCD)
Room G6 - Woodview House
Belfield, Dublin 4
ross.neville using ucd.ie
+353 (0) 1 716 3419

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20250218/ce2a8204/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2025-02-18 at 12.38.17.png
Type: image/png
Size: 47847 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20250218/ce2a8204/attachment-0001.png>

More information about the R-sig-meta-analysis mailing list