[R-meta] effect size calculation with a null standard deviation

[Marianne DEBUE]

Dear Wolfgang and James, 

Thank you for your answers. 

I also considered using the response ratio but I am still confronted to a problem of division by 0. There are indeed some species which are completely absent either before or after the intervention due to changes in ecological conditions so pre-Mean or post-Mean can also be equal to 0. 

I was wondering if I could slightly change the results of the studies where SDpre=0 by adding one individual in the pre and post-data, that is to say, instead of having N samples with 0 individual of a given species in the pre-data, considering N-1 samples with 0 individual and 1 sample with 1 individual, and similarly adding virtually one individual in one sample in the post-data. In this particular case, if I am not mistaken, the difference between means is unchanged and the new SDpre is equal to 1/N. 
Is it a conceivable solution? 

Regards, 
Marianne 



De: "James Pustejovsky" <jepusto using gmail.com> 
À: "Wolfgang Viechtbauer, SP" <wolfgang.viechtbauer using maastrichtuniversity.nl> 
Cc: "Marianne DEBUE" <marianne.debue using mnhn.fr>, "r-sig-meta-analysis" <r-sig-meta-analysis using r-project.org> 
Envoyé: Jeudi 18 Février 2021 20:15:40 
Objet: Re: [R-meta] effect size calculation with a null standard deviation 

Hi Marianne, 
I agree with Wolfgang that this is an unusual case and that figuring out how to handle it requires some contextual judgements, which folks on the mailing list aren't really in a position to advise on. That said, here are two avenues that you might like to investigate further: 

With only 3 observations in a given group, using a sample standard deviation to calculate a standardized mean difference for the group will give an EXTREMELY noisy estimate. Pooling the pre-test variance across groups (as in the Morris "dppc2" estimator) might mitigate the problem a bit--especially if only one of the groups is small and the other is larger. 

More broadly, the fact that you're encountering these sorts of situations makes me wonder whether it would be better to move to a different effect size metric, such as the response ratio ( Hedges, L. V., Gurevitch, J., & Curtis, P. S. (1999). The meta‐analysis of response ratios in experimental ecology. Ecology , 80 (4), 1150-1156.). This might be a good way to go if all of your outcomes are ratio scale measurements. My understanding is that this is pretty common in ecology. 

James 


On Thu, Feb 18, 2021 at 2:49 AM Viechtbauer, Wolfgang (SP) < [ mailto:wolfgang.viechtbauer using maastrichtuniversity.nl | wolfgang.viechtbauer using maastrichtuniversity.nl ] > wrote: 


Dear Marianne, 

I suspect that you did not receive any responses, since what you are describing are really unusual cases which have not been discussed in the literature (as far as I know). 

I think you will just have to make a decision yourself how to handle these cases and be transparent about how you handled them. 

Best, 
Wolfgang 

>-----Original Message----- 
>From: R-sig-meta-analysis [mailto: [ mailto:r-sig-meta-analysis-bounces using r-project.org | r-sig-meta-analysis-bounces using r-project.org ] ] On 
>Behalf Of Marianne DEBUE 
>Sent: Thursday, 21 January, 2021 15:57 
>To: [ mailto:r-sig-meta-analysis using r-project.org | r-sig-meta-analysis using r-project.org ] 
>Subject: [R-meta] effect size calculation with a null standard deviation 
> 
>Hi everyone, 
> 
>I'm conducting a meta-analysis in ecology. I'm using Morris "dpcc1" formulas ( [ 
> [ https://journals.sagepub.com/doi/10.1177/1094428106291059 | https://journals.sagepub.com/doi/10.1177/1094428106291059 ] | 
> [ https://journals.sagepub.com/doi/10.1177/1094428106291059 | https://journals.sagepub.com/doi/10.1177/1094428106291059 ] ] ) to calculate the 
>effect size and its variance. 
> 
>The effect size calculation implies a difference between post- and pre-Mean which 
>is then divided by the pre-Standard deviation. 
>I was wondering how to deal with studies which have pre-Standard deviation = 0 
>(leading to a division by 0) ? 
>How to deal with studies which have pre-Mean = post-Mean and a pre-Standard 
>deviation = 0 (leading to 0 divided by 0) ? If pre-Mean = post-Mean , can we 
>consider that the effect size is null, whatever the pre-Standard deviation ? 
> 
>The variance calculation implies a division by (N - 3) (N : sample size). 
>How to deal with studies which have N = 3 ( leading to a division by 0) ? 
> 
>Thank you for your help, 
> 
>Marianne 

_______________________________________________ 
R-sig-meta-analysis mailing list 
[ mailto:R-sig-meta-analysis using r-project.org | R-sig-meta-analysis using r-project.org ] 
[ https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis | https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis ] 





	[[alternative HTML version deleted]]