[R-meta] Threshold values for Cook's Distances and DFBETAS

[Viechtbauer, Wolfgang (NP)]

And to follow-up on this:

There isn't any formula anyway. It's a bit like asking if there a formula that we can use to determine when a person is 'unusually tall' based on having measured people's height. It's all relative. We can just compare the height of the tallest person with that of the rest. If he/she sticks out (in the literal sense), then we might say that this person is quite tall compared to the rest.

Therefore, even the rules mentioned here are essentially arbitrary:

https://wviechtb.github.io/metafor/reference/influence.rma.uni.html

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Michael Dewey
>Sent: Friday, 30 September, 2022 11:08
>To: Thölking, Theresa; r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] Threshold values for Cook's Distances and DFBETAS
>
>Dear Theresa
>
>My feeling is that these diagnostics are better seen as identifying
>values which may benefit from further investigation. If I was concerned
>that they were unduly influential in the model I might try fitting
>leave-one-out models but since those models are essentially data-driven
>they are also only useful in an exploratory sense.
>
>So, even if there is a formula I would be sceptical about its value in
>answering the scientific question.
>
>Michael
>
>On 29/09/2022 21:35, Thölking, Theresa wrote:
>> Hello everybody,
>>
>> I am trying to apply the outlier and influence diagnostics as described by
>Viechtbauer and Cheung (2010) on a meta-analysis I conducted, using a four-level
>random-effects model with 528 included effect sizes. I have calculated the Cook's
>Distances and DFBETAS in R, using the functions for model diagnostics for rma.mv
>objects as described here:
>https://wviechtb.github.io/metafor/reference/influence.rma.mv.html
>>
>> However, I don't know how to decide about which studies are influential based
>on these values. I read the paper by Viechtbauer and Cheung, but I don't quite
>understand it. Is there some kind of formula to determine the threshold values of
>CD and DFBETAS for a meta-analytic model with k outcomes? Thanks in advance for
>your help.
>>
>> Best,
>>
>> Theresa