[R-meta] SMD from three-level nested design (raw data available)

[Viechtbauer, Wolfgang (SP)]

Catching up on some older posts.

1) The transformations of a log odds ratio to a standardized mean difference (measures "OR2DN" and "OR2DL" in escalc()) are quite simple. For OR2DN, it is essentially:

yi <- lnori / 1.65
vi <- selnori^2 / 1.65^2

where lnori is the log odds ratio and selnori the corresponding standard error. So, you basically can just take the coefficient from the generalized mixed-effects model (which *is* a log odds ratio, assuming you fitted a logistic mixed-effects model) and its SE and plug those into these equations. The SE of the coefficient already incorporates the additional source of variability due to the nested structure of the data (assuming you include appropriate variance components in the mixed-effects model to reflect the nesting). One could debate whether the logic underlying this transformation is still applicable when the data have a nested structure, but I think this approach is defensible.

2) If I understand you correctly, you are considering the following case: A dichotomous variables (which we consider a dichotomized version of a latent continuous variable) and a continuous variable are measured in a group of subjects and there is again nesting. The question is how to estimate the correlation between the latent continuous variable and the observed continuous variable based on such multilevel data. So, the data structure would be:

group  subject  outcome  y
1      1        dich     1
1      1        cont     5
1      2        dich     0
1      2        cont     2
1      3        dich     1
1      3        cont     4
...

so two rows of data per subject. First of all, modeling such data jointly is not straightforward, because you need two different types of models for the two outcomes (e.g., logistic or probit for the dichotomous variable and normal for the continuous variable). You would need to look into what are sometimes called 'joint models', such as:

https://cran.r-project.org/package=merlin

Then one needs to allow for correlation between the two outcomes. Finally, one would really need to allow for correlation between the latent continuous variable and the observed continuous variable, so this might require specifying a model with latent variables (so, a joint model with latent variables). Implementing something like this would be a rather interesting exercise, but might actually be breaking some novel ground. So, I can only say that conceptually this should be possible, but I do not know of any solutions 'out of the box'. Maybe merlin can actually do this, but I haven't looked into this.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Fabian Schellhaas
Sent: Thursday, 29 November, 2018 22:42
To: James Pustejovsky
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] SMD from three-level nested design (raw data available)

Dear all,

We had a couple of related questions, so I will add to this thread. If
preferred, I'd be happy to start a new thread instead.

1. Another study, for which we obtained the raw data from the authors, has
a nested data structure with two levels – individual participants nested in
clusters. What complicates matters here is that the construct of interest,
which is continuous, was operationalized as a dichotomous measure. In the
non-nested case, we would just compute the transformed log-odds ratio
(e.g., escalc(measure = "OR2DN") in metafor). However, since the data are
nested, we can fit a generalized linear mixed model (GLMM) to predict this
dichotomous outcome. How would we then extract an effect size analogous to
the transformed log-odds ratio from this model? The Hedges chapter and
papers only describe the case of a continuous outcome variable, so we're
not sure about the correct approach.

2. We also wondered, more generally, how effect-size calculation from
nested data with a dichotomous outcome would be handled in a meta-analysis
of correlations. In the non-nested case, we could compute the biserial
correlation when the predictor is continuous and the outcome is
dichotomized (e.g., Jacobs & Viechtbauer, 2017). However, how would we
extract an effect size analogous to the biserial correlation from a GLMM,
which could then be combined with correlations from single-level data?

Many thanks for any pointers!
Fabian

---
Fabian M. H. Schellhaas | Ph.D. Candidate | Department of Psychology | Yale
University