[R-meta] Multi-informant multi-level meta-analysis

[Viechtbauer, Wolfgang (NP)]

Dear Ross,

Please see my responses below.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Ross Neville via R-sig-meta-analysis
> Sent: Wednesday, November 27, 2024 07:24
> To: r-sig-meta-analysis using r-project.org
> Cc: Ross Neville <ross.neville using ucd.ie>
> Subject: [R-meta] Multi-informant multi-level meta-analysis
>
> Dear Professor Viechtbauer
>
> I'm conducting a meta-analysis where I have a nested data structure.
>
> Each StudyID has an EstimateID for the control (EstimateID=1) and
> intervention (EstimateID=2) arms.
>
> Each arm also has multiple informants. The variable InformantID identifies
> whether the EstimateID reflects data reported by teachers or parents.
>
> Would the following random effect accurately capture that nested structure:
>
> random = list(~ 1 | StudyID/EstimateID/InformantID)

Yes, this does look reasonable based on your description.

You can also run robust(<model>, clubSandwich=TRUE) to see if the the model-based SEs are similar to those from robust variance estimation. This can serve as a *rough* check for model misspecification.

> The resulting random effects seem to align with what one would want to know:
>
>   estim sqrt nlvls fixed factor
> sigma^2.1 8.6938 2.9485 40 no StudyID
> sigma^2.2 7.1745 2.6785 78 no StudyID/EstimateID
> sigma^2.3 2.8651 1.6927 92 no StudyID/EstimateID/InformantID
>
> My interpretation of the above table is that:
>
>    - sigma^2.1 represents the between study variables
>    - sigma^2.2 represents the between estimate (i.e., between condition)
>    variability within studies
>    - sigma^2.3 represents the between informant (i.e., teacher vs parent)
>    variability within each condition within studies.
>
> Does this interpretation sound correct to you?

Yes, that is correct.

> Appreciate any feedback or guidance you can provide.
>
> Regards
> Ross
>
> --
> Dr Ross D. Neville, PhD, ProfCert University Teaching and Learning
> Head of Subject - Sport Management
> School of Public Health, Physiotherapy and Sport Science
> University College Dublin (UCD)
> Room G6 - Woodview House
> Belfield, Dublin 4
> ross.neville using ucd.ie
> +353 (0) 1 716 3419