[R-meta] Meta-analysis of intra class correlation coefficients

[Viechtbauer, Wolfgang (NP)]

Dear Beate,

I assume you are talking about ICC(1) values. We did a meta-analysis of ICC(1) values here:

Nicolaï, S. P. A., Viechtbauer, W., Kruidenier, L. M., Candel, M. J. J. M., Prins, M. H., & Teijink, J. A. W. (2009). Reliability of treadmill testing in peripheral arterial disease: A meta-regression analysis. Journal of Vascular Surgery, 50(2), 322-329. https://doi.org/10.1016/j.jvs.2009.01.042

For ICC(1) values, one can apply a variance-stabilizing transformation with:

y = 1/2 * log((1 + (m-1)*icc) / (1 - icc))

where 'm' is the number of measurement occasions and 'n' is the number of participants. The large-sample variance is then:

Var[y] = m / (2*(m-1)*(n-2)).

This goes back to Fisher (1925; Statistical methods for research workers).

In your application (where you dealing with pairs), n is the number of pairs and m is 2. In that case, you can treat ICC(1) values like regular correlations. However, if you do apply the r-to-z transformation, then Fisher suggests to use 1/(n-3/2) as the variance (instead of 1/(n-3) as we typically use for r-to-z transformed Pearson product-moment correlation coefficients) and simulation studies I have done confirm this.

Best,
Wolfgang

--
Wolfgang Viechtbauer, PhD, Statistician | Department of Psychiatry and
Neuropsychology | Maastricht University | PO Box 616 (VIJV1) | 6200 MD
Maastricht, The Netherlands | +31(43)3884170 | https://www.wvbauer.com

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of St Pourcain, Beate via R-sig-meta-analysis
> Sent: Saturday, October 12, 2024 10:11
> To: r-sig-meta-analysis using r-project.org
> Cc: St Pourcain, Beate <Beate.StPourcain using mpi.nl>
> Subject: [R-meta] Meta-analysis of intra class correlation coefficients
>
> Dear R-sig group,
>
> We would like to carry out a meta-analysis of intra class correlation (ICC)
> coefficients (i.e. the correlation between pairs of individuals, for the same
> trait). For this reason, we aim to derive the variance of ICCs from published
> studies, without access to individual data, thus, preventing bootstrapping.
> Could anyone advise us what the options are to derive the variance for ICCs
> using existing R packages?
>
> We were looking into the escalc function from the metafor package, but could not
> find options for ICCs, but options for Pearson product-moment correlations
> ("COR" option).
>
> We have seen research proposing to apply the escalc "COR" option also to derive
> the variance for ICCs. However, we are unsure about this, as we may get
> undesirable properties for the derived variance, given that the definition of
> the correlations is different.
>
> Any advice is welcome!
>
> Best wishes,
> Beate
>
> Beate St Pourcain, PhD
> Senior Investigator & Group Leader
> Room A207
> Max Planck Institute for Psycholinguistics | Wundtlaan 1 | 6525 XD Nijmegen |
> The Netherlands
>
> @bstpourcain
> Tel: +31 24 3521964
> Fax: +31 24 3521213
> ORCID: https://orcid.org/0000-0002-4680-3517
> Web: https://www.mpi.nl/departments/language-and-genetics/projects/population-
> variation-and-human-communication/
> Further affiliations with:
> MRC Integrative Epidemiology Unit | University of Bristol | UK
> Donders Institute for Brain, Cognition and Behaviour | Radboud University | The
> Netherlands