<div dir="ltr">Thanks Wolfgang<div>I will take some time to digest the contents. I appreciate the detail and guidance.</div><div>Quickly, the suggested model provides a sigma^2.2 variance of 0, as shown below. </div><div>This makes me feel like the random effects random = ~ 1 | StudyID / TreatmentGroup / Informant are too complex for the available data.</div><div>Thought on that?</div><div><img src="cid:ii_m7ah1yvz0" alt="Screenshot 2025-02-18 at 12.38.17.png" width="562" height="75"><br></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Tue, 18 Feb 2025 at 12:34, Viechtbauer, Wolfgang (NP) <<a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl">wolfgang.viechtbauer@maastrichtuniversity.nl</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear Ross,<br>
<br>
Thanks for the clarification. Based on this, I would consider the following structure:<br>
<br>
random = ~ 1 | StudyID / TreatmentGroup / Informant<br>
<br>
This captures overall differences in the outcomes (across the experimental and control groups and across all informants) between studies. It also allows for heterogeneity in how much experimental and control groups differ from each other across studies (I assume you will add something like mods = ~ TreatmentGroup to the model, since presumably you are interested in the size of the (average) difference between the two groups). And it allows for heterogeneity in outcomes that arises due to some studies using multiple informants. By nesting Informant within TreatmentGroup, this model automatically implies a certain degree of correlation in the true outcomes for different informants within experimental/control groups. This last random effect is also the 'outcome level' random effect, since based on your description, every combination of StudyID, TreatmentGroup, and Informant should yield a unique value for each row.<br>
<br>
Strictly speaking, this structure does not capture the correlation in the sampling errors that arises because multiple informants are reporting on the *same children*. If reports, say from parents and teachers, are correlated, then this implies that the sampling errors of the means are also correlated. Such a correlation (or more precisely, the covariance) should go into the V matrix. However, to compute this covariance, you would need to know what the correlation (r) between the parent and teacher reports. This is probably not reported, but maybe can be guestimated from other or your own studies. Say the data for the first two studies looks like this:<br>
<br>
Study Group Informant Outcome<br>
1 Exp Child .<br>
1 Ctrl Child .<br>
2 Exp Parent .<br>
2 Exp Teacher .<br>
2 Ctrl Parent .<br>
2 Ctrl Teacher .<br>
<br>
Then the corresponding V matrix would be (use a fixed width font to view this so that things are lined up properly):<br>
<br>
[s_1E^2/n_1E ]<br>
[ s_1C^2/n_1C ]<br>
[ s_2EP^2/n_2E r*s_2EP*s_2ET/n_2E ]<br>
[ s_2ET^2/n_2E ]<br>
[ s_2CP^2/n_2C r*s_2CP*s_2CT/n_2C]<br>
[ s_2CT^2/n_2C ]<br>
<br>
where s stands for standard deviation, n for sample size, and the subscripts are for study, group, and informant in that order (single letter abbreviations). Elements left blank are equal to 0 (zero covariance). I did not put a subscript on r since this is probably a single guestimated value across all studies (and informant pairs). Such a V matrix can be easily generated using the vcalc() function.<br>
<br>
Since V is probably just going to be an approximation (especially if you decide not to bother creating the V matrix, which in essence implies assuming r=0), I would then consider using cluster-robust inference methods (robust(model, cluster=StudyID, clubSandwich=TRUE)) at least as a sensitivity check.<br>
<br>
I think the above is a good and sensible starting point. A more complex structure might be:<br>
<br>
dat$StudyID.TreatmentGroup <- paste0(dat$StudyID, ".", dat$TreatmentGroup)<br>
random = list(~ TreatmentGroup | StudyID, ~ Informant | StudyID.TreatmentGroup), struct="UN")<br>
<br>
This would allow for different variances for experimental and control groups and it would allow for different variances for the different types of informants and allow the correlation in the random effects to differ depending on the type of informant pair (child-parent, child-teacher, parent-teacher, etc.). But I would only attempt to fit this model if there is plenty of data. One could use a LRT to compare the two model structures. There is also a structure with intermediate complexity by using struct=c("UN","HCS"), where we assume different variances for the different informants, but a single correlation irrespective of the pair.<br>
<br>
One might also consider TreatmentGroup and Informant to be crossed random effects (within studies), but I think this is overcomplicating things.<br>
<br>
Best,<br>
Wolfgang<br>
<br>
> -----Original Message-----<br>
> From: Ross Neville <<a href="mailto:ross.neville@ucd.ie" target="_blank">ross.neville@ucd.ie</a>><br>
> Sent: Friday, February 14, 2025 14:32<br>
> To: Viechtbauer, Wolfgang (NP) <<a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl" target="_blank">wolfgang.viechtbauer@maastrichtuniversity.nl</a>><br>
> Cc: <a href="mailto:r-sig-meta-analysis@r-project.org" target="_blank">r-sig-meta-analysis@r-project.org</a><br>
> Subject: Re: nesting an inner | outer formula<br>
><br>
> Dear Wolfgang<br>
><br>
> Thanks for the speedy response, and for seeking clarification and correcting my<br>
> error.<br>
><br>
> Rather than try to correct my interpretation of the SAS code, perhaps it would<br>
> make more sense to tell you what structure I really want.<br>
><br>
> The data structure is such that I have studies (StudyID) reporting post-<br>
> intervention means for children in an experimental and control group<br>
> (TreatmentGroup). The variable Informant tells us who is reporting on behalf of<br>
> the child. Some studies have child report only (so two rows for such a StudyID<br>
> corresponding to the post-intervention means for the experimental and control<br>
> group). Studies with parent report or teacher report only are the same (two<br>
> rows). There are also studies where there is data from child and parent, child<br>
> and teacher, teacher and parent, or even child parent and teacher. So,<br>
> essentially, a StudyID could have two rows, four rows, or six rows, depending on<br>
> how many Informants there are. Children are in the experimental or control group<br>
> only, so one would expect Informants to be nested and correlated within<br>
> TreatmentGroup within studies.<br>
><br>
> Because of the data structure (multiple rows of sample means rather than fewer<br>
> rows of pairwise comparisions) the degree to which control and intervention<br>
> group means are more or less similar in a given StudyID is capture in part<br>
> (maybe even large part) by ~ 1 | StudyID.<br>
><br>
> What is missing from this random = list (~ 1 | StudyID, ~ Informant |<br>
> TreatmentGroup) is the fact that the inner | outer is saying, for example, that<br>
> parents in the experimental or control group across studies share correlated<br>
> random effects. When, in fact, one would expect parents, children, and teachers<br>
> in the experimental or control group to share correlated random effects within a<br>
> given study.<br>
><br>
> Perhaps given the data structure, you would advise something else. Or perhaps I<br>
> am still being unclear in my description and understanding.<br>
><br>
> Regards<br>
> Ross<br>
><br>
> On Fri, 14 Feb 2025 at 13:00, Viechtbauer, Wolfgang (NP)<br>
> <mailto:<a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl" target="_blank">wolfgang.viechtbauer@maastrichtuniversity.nl</a>> wrote:<br>
> Dear Ross,<br>
><br>
> my proc mixed knowledge is a bit rusty, but unless I am confused, your proc<br>
> mixed statement specifies a random intercept for StudyID and an UN structure for<br>
> Informant within StudyID allowing for different variances/covariances for the<br>
> different levels of TreatmentGroup.<br>
><br>
> > random StudyID;<br>
> > random Informant / subject=StudyID group=TreatmentGroup type=un;<br>
> > parms 1 1 1 1 1 1 1 1 1 1 1 1 1 1 / hold=14;<br>
><br>
> I don't think this quite matches up with your description:<br>
><br>
> > I would like for the different levels of Informant to be correlated within<br>
> > TreatmentGroup within StudyID, and I would like the different levels of<br>
> > TreatmentGroup to be correlated within StudyID too.<br>
><br>
> For example, there is nothing in your proc mixed statement that allows for<br>
> "TreatmentGroup to be correlated within StudyID". Also, the second random<br>
> statement allows for Informant to be correlated within StudyID, but *not*<br>
> "within TreatmentGroup within StudyID".<br>
><br>
> So before I attempt to recreate the same structure, it would need to be clear<br>
> exactly what kind of structure you really want.<br>
><br>
> Best,<br>
> Wolfgang<br>
><br>
> > -----Original Message-----<br>
> > From: Ross Neville <mailto:<a href="mailto:ross.neville@ucd.ie" target="_blank">ross.neville@ucd.ie</a>><br>
> > Sent: Thursday, February 13, 2025 18:28<br>
> > To: Viechtbauer, Wolfgang (NP)<br>
> <mailto:<a href="mailto:wolfgang.viechtbauer@maastrichtuniversity.nl" target="_blank">wolfgang.viechtbauer@maastrichtuniversity.nl</a>>;<br>
> > mailto:<a href="mailto:r-sig-meta-analysis@r-project.org" target="_blank">r-sig-meta-analysis@r-project.org</a><br>
> > Subject: nesting an inner | outer formula<br>
> ><br>
> > Hi Wolfgang<br>
> ><br>
> > I hope this email finds you well<br>
> ><br>
> > I was wondering if you could tell me whether the following list of random<br>
> > effects can be updated to make the the inner | outer formula conditional.<br>
> ><br>
> > random = list (~ 1 | StudyID, ~ Informant | TreatmentGroup)<br>
> ><br>
> > I would like for the different levels of Informant to be correlated within<br>
> > TreatmentGroup within StudyID, and I would like the different levels of<br>
> > TreatmentGroup to be correlated within StudyID too.<br>
> ><br>
> > In SAS Proc Mixed, I've managed to run this model and I want to replicate it<br>
> in<br>
> > metafor <a href="http://rma.mv" rel="noreferrer" target="_blank">http://rma.mv</a>.<br>
> ><br>
> > random StudyID;<br>
> > random Informant/subject=StudyID, group=TreatmentGroup type=un;<br>
> > parms 1 1 1 1 1 1 1 1 1 1 1 1 1 1/hold=14;<br>
> ><br>
> > Any help you can provide to let me know if this is possible in <a href="http://rma.mv" rel="noreferrer" target="_blank">http://rma.mv</a><br>
> > would be much appreciated.<br>
> ><br>
> > Regards<br>
> > Ross<br>
> ><br>
> > --<br>
> > Dr Ross D. Neville, PhD, ProfCert University Teaching and Learning<br>
> > Head of Subject - Sport Management<br>
> > School of Public Health, Physiotherapy and Sport Science<br>
> > University College Dublin (UCD)<br>
> > Room G6 - Woodview House<br>
> > Belfield, Dublin 4<br>
> > mailto:<a href="mailto:mailto" target="_blank">mailto</a>:<a href="mailto:ross.neville@ucd.ie" target="_blank">ross.neville@ucd.ie</a><br>
> > +353 (0) 1 716 3419<br>
</blockquote></div><div><br clear="all"></div><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><font face="arial, sans-serif">Dr Ross D. Neville, PhD, ProfCert University Teaching and Learning</font></div><div dir="ltr"><font face="arial, sans-serif">Head of Subject - Sport Management</font></div><div dir="ltr"><font face="arial, sans-serif">School of Public Health, Physiotherapy and Sport Science</font><div><font face="arial, sans-serif">University College Dublin (UCD)<br></font></div><div><font face="arial, sans-serif">Room G6 - Woodview House</font></div><div><font face="arial, sans-serif">Belfield, Dublin 4</font></div><div><a href="mailto:ross.neville@ucd.ie" target="_blank"><font face="arial, sans-serif">ross.neville@ucd.ie</font></a></div><div><font face="arial, sans-serif">+353 (0) 1 716 3419<br></font></div><div><font face="arial, sans-serif"><img width="200" height="141" src="https://ci3.googleusercontent.com/mail-sig/AIorK4zgpRRECYGkD4xGeWEz5P3oap2yQyz0ERCN16nRb_tRqAXyQNkeXpOIvrwialTOF-rPUtn_RJHbUBxz"><br></font></div></div></div><div dir="ltr"></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>