[R-meta] effect size calculation with a null standard deviation

[James Pustejovsky]

Hi Marianne,

I agree with Wolfgang that this is an unusual case and that figuring out
how to handle it requires some contextual judgements, which folks on the
mailing list aren't really in a position to advise on. That said, here are
two avenues that you might like to investigate further:

With only 3 observations in a given group, using a sample standard
deviation to calculate a standardized mean difference for the group will
give an EXTREMELY noisy estimate. Pooling the pre-test variance across
groups (as in the Morris "dppc2" estimator) might mitigate the problem a
bit--especially if only one of the groups is small and the other is larger.

More broadly, the fact that you're encountering these sorts of situations
makes me wonder whether it would be better to move to a different effect
size metric, such as the response ratio (Hedges, L. V., Gurevitch, J., &
Curtis, P. S. (1999). The meta‐analysis of response ratios in experimental
ecology. *Ecology*, *80*(4), 1150-1156.). This might be a good way to go if
all of your outcomes are ratio scale measurements. My understanding is that
this is pretty common in ecology.

James


On Thu, Feb 18, 2021 at 2:49 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Dear Marianne,
>
> I suspect that you did not receive any responses, since what you are
> describing are really unusual cases which have not been discussed in the
> literature (as far as I know).
>
> I think you will just have to make a decision yourself how to handle these
> cases and be transparent about how you handled them.
>
> Best,
> Wolfgang
>
> >-----Original Message-----
> >From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On
> >Behalf Of Marianne DEBUE
> >Sent: Thursday, 21 January, 2021 15:57
> >To: r-sig-meta-analysis using r-project.org
> >Subject: [R-meta] effect size calculation with a null standard deviation
> >
> >Hi everyone,
> >
> >I'm conducting a meta-analysis in ecology. I'm using Morris "dpcc1"
> formulas ( [
> >https://journals.sagepub.com/doi/10.1177/1094428106291059 |
> >https://journals.sagepub.com/doi/10.1177/1094428106291059 ] ) to
> calculate the
> >effect size and its variance.
> >
> >The effect size calculation implies a difference between post- and
> pre-Mean which
> >is then divided by the pre-Standard deviation.
> >I was wondering how to deal with studies which have pre-Standard
> deviation = 0
> >(leading to a division by 0) ?
> >How to deal with studies which have pre-Mean = post-Mean and a
> pre-Standard
> >deviation = 0 (leading to 0 divided by 0) ? If pre-Mean = post-Mean , can
> we
> >consider that the effect size is null, whatever the pre-Standard
> deviation ?
> >
> >The variance calculation implies a division by (N - 3) (N : sample size).
> >How to deal with studies which have N = 3 ( leading to a division by 0) ?
> >
> >Thank you for your help,
> >
> >Marianne
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]