[R-meta] effect size calculation with a null standard deviation

[Viechtbauer, Wolfgang (SP)]

Conceivable in the sense of "one can think it up"? Sure, you just did :)

Conceivable in the sense of "credible"? I don't think anybody knows.

It sounds to me though as if you are dealing with count data. Maybe measures such as standardized mean differences and response ratios (which are meant to be used for continuous variables) are not the most suitable choice in the first place. For counts, one could also consider using incidence rates (and differences/ratios thereof).

Best,
Wolfgang

>-----Original Message-----
>From: Marianne DEBUE [mailto:marianne.debue using mnhn.fr]
>Sent: Friday, 19 February, 2021 9:25
>To: James Pustejovsky; Viechtbauer, Wolfgang (SP)
>Cc: r-sig-meta-analysis
>Subject: Re: [R-meta] effect size calculation with a null standard deviation
>
>Dear Wolfgang and James,
>
>Thank you for your answers.
>
>I also considered using the response ratio but I am still confronted to a problem
>of division by 0. There are indeed some species which are completely absent either
>before or after the intervention due to changes in ecological conditions so pre-
>Mean or post-Mean can also be equal to 0.
>
>I was wondering if I could slightly change the results of the studies where
>SDpre=0 by adding one individual in the pre and post-data, that is to say, instead
>of having N samples with 0 individual of a given species in the pre-data,
>considering N-1 samples with 0 individual and 1 sample with 1 individual, and
>similarly adding virtually one individual in one sample in the post-data. In this
>particular case, if I am not mistaken, the difference between means is unchanged
>and the new SDpre is equal to 1/N.
>Is it a conceivable solution?
>
>Regards,
>Marianne
>
>________________________________________
>De: "James Pustejovsky" <jepusto using gmail.com>
>À: "Wolfgang Viechtbauer, SP" <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Cc: "Marianne DEBUE" <marianne.debue using mnhn.fr>, "r-sig-meta-analysis" <r-sig-meta-
>analysis using r-project.org>
>Envoyé: Jeudi 18 Février 2021 20:15:40
>Objet: Re: [R-meta] effect size calculation with a null standard deviation
>
>Hi Marianne,
>I agree with Wolfgang that this is an unusual case and that figuring out how to
>handle it requires some contextual judgements, which folks on the mailing list
>aren't really in a position to advise on. That said, here are two avenues that you
>might like to investigate further:
>
>With only 3 observations in a given group, using a sample standard deviation to
>calculate a standardized mean difference for the group will give an EXTREMELY
>noisy estimate. Pooling the pre-test variance across groups (as in the Morris
>"dppc2" estimator) might mitigate the problem a bit--especially if only one of the
>groups is small and the other is larger.
>
>More broadly, the fact that you're encountering these sorts of situations makes me
>wonder whether it would be better to move to a different effect size metric, such
>as the response ratio (Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The
>meta‐analysis of response ratios in experimental ecology. Ecology, 80(4), 1150-
>1156.). This might be a good way to go if all of your outcomes are ratio scale
>measurements. My understanding is that this is pretty common in ecology.
>
>James
>
>
>On Thu, Feb 18, 2021 at 2:49 AM Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>Dear Marianne,
>
>I suspect that you did not receive any responses, since what you are describing
>are really unusual cases which have not been discussed in the literature (as far
>as I know).
>
>I think you will just have to make a decision yourself how to handle these cases
>and be transparent about how you handled them.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>Behalf Of Marianne DEBUE
>>Sent: Thursday, 21 January, 2021 15:57
>>To: r-sig-meta-analysis using r-project.org
>>Subject: [R-meta] effect size calculation with a null standard deviation
>>
>>Hi everyone,
>>
>>I'm conducting a meta-analysis in ecology. I'm using Morris "dpcc1" formulas ( [
>>https://journals.sagepub.com/doi/10.1177/1094428106291059 |
>>https://journals.sagepub.com/doi/10.1177/1094428106291059 ] ) to calculate the
>>effect size and its variance.
>>
>>The effect size calculation implies a difference between post- and pre-Mean which
>>is then divided by the pre-Standard deviation.
>>I was wondering how to deal with studies which have pre-Standard deviation = 0
>>(leading to a division by 0) ?
>>How to deal with studies which have pre-Mean = post-Mean and a pre-Standard
>>deviation = 0 (leading to 0 divided by 0) ? If pre-Mean = post-Mean , can we
>>consider that the effect size is null, whatever the pre-Standard deviation ?
>>
>>The variance calculation implies a division by (N - 3) (N : sample size).
>>How to deal with studies which have N = 3 ( leading to a division by 0) ?
>>
>>Thank you for your help,
>>
>>Marianne