[R-meta] Correcting Hedges' g vs. Log response ratio in nested studies

[James Pustejovsky]

One other thought on this question, for the extra-nerdy.

The formulas for the Hedges' g SMD estimator involve what statisticians
would call "second-order" bias corrections, meaning corrections arising
from having a limited sample size. In contrast, the usual estimator of the
LRR is just a "plug-in" estimator that works for large sample sizes but can
have small biases with limited sample sizes. Lajeunesse (2015;
https://doi.org/10.1890/14-2402.1) provides formulas for the second-order
bias correction of the LRR estimator with independent samples. These bias
correction formulas actually *would* need to be different if you
have clustered observations. So, the two effect size metrics are maybe more
similar than it initially seemed:
- Both metrics have plug-in estimators that are not really affected by the
dependence structure of the sample, but whose variance estimators do need
to take into account the dependence structure
- Both metrics have second-order corrected estimators, the exact form for
which does need to take into account the dependence structure.

James

On Thu, Nov 2, 2023 at 8:14 AM James Pustejovsky <jepusto using gmail.com> wrote:

>
> Wolfgang is correct. The WWC correction factor arises because the sample
> variance is not quite unbiased as an estimator for the total population
> variance in a design with clusters of dependent observations, which leads
> to a small bias in the SMD.
>
> The thing is, though, this correction factor is usually negligible. Say
> you’ve got a clustered design with n = 21 kids per cluster and 20 clusters,
> and an ICC of 0.2. Then the correction factor is going to be about 0.99 and
> so will make very little difference for the effect size estimate. It only
> starts to matter if you’re looking at studies with very few clusters and
> non-trivial ICCs.
>
> James
>
> > On Nov 2, 2023, at 3:04 AM, Viechtbauer, Wolfgang (NP) via
> R-sig-meta-analysis <r-sig-meta-analysis using r-project.org> wrote:
> > Dear Yuhang,
> >
> > I haven't looked deeply into this, but an immediate thought I have is
> that for SMDs, you divide by some measure of variability within the groups.
> If that measure of variability is affected by your study design, then this
> will also affect the SMD value. On the other hand, this doesn't have any
> impact on LRRs since they are only the (log-transformed) ratio of the means.
> >
> > Best,
> > Wolfgang
> >
> >> -----Original Message-----
> >> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org>
> On Behalf
> >> Of Yuhang Hu via R-sig-meta-analysis
> >> Sent: Thursday, November 2, 2023 05:42
> >> To: R meta <r-sig-meta-analysis using r-project.org>
> >> Cc: Yuhang Hu <yh342 using nau.edu>
> >> Subject: [R-meta] Correcting Hedges' g vs. Log response ratio in nested
> studies
> >>
> >> Hello All,
> >>
> >> I know that when correcting Hedges' g (i.e., bias-corrected SMD, aka
> "g")
> >> in nested studies, we have to **BOTH** adjust our initial "g" and its
> >> sampling variance "vi_g"
> >> (https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-41-Supplement-
> >> 508_09212020.pdf).
> >>
> >> But when correcting Log Response Ratios (LRR) in nested studies, we
> have to
> >> **ONLY** adjust its initial sampling variance "vi_LRR" but not "LRR"
> itself
> >> (
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-October/003486.html
> ).
> >>
> >> I wonder why the two methods of correction differ for Hedge's g and LRR?
> >>
> >> Thanks,
> >> Yuhang
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]