[R-meta] effect size estimates regardless of direction

[Dave Daversa]

Thanks to you both for the helpful information, and sorry for the delay in
responding. To remind everyone, I wrote a couple weeks ago seeking advice
on how to estimate the mean magnitude of effect sizes (i.e. the absolute
value of effects without considering direction), rather than estimating
true means as most models are intended for.

Daniel suggested a bayesian approach that The analyze-then-transform method
proposed by Morrisey et al 2016.  This approach does seem to be just what I
need for estimating mean effect magnitudes without generating upward
biases.

A few follow up questions:
1. I am using multi-level mixed models to estimate mean effect sizes (using
rma.mv in metafor).  Any reason why the function for the mean of a folded
distribution (the mu.fnorm function in the Rejoinder paper) could not be
applied to these more complex models?

2. I am also testing the influence of various moderators on effect sizes
using likelihood ratio tests (seeing whether dropping certain factors
reduces goodness of fit). I can not think of how the analyze-then-transform
method could be applicable here.  Have you ever done these types of
analyses with magnitudes?

3. do you have recommendations for estimating confidence intervals about
the mean magnitudes?



On Mon, May 21, 2018 at 10:39 PM, Daniel Noble <daniel.wa.noble using gmail.com>
wrote:

> Hi Dave and Wolfgang,
>
> If you don't mind going Bayesian, you can try the "analyse and transform"
> option. This is done by estimating the overall mean estimate and applying
> that to the folded normal. Check out Mike's two papers.
>
> Morrisey,M.B.(2016). Meta-analysis of magnitudes, differences and
> variation in evolutionary parameters. Journal of Evolutionary Biology 29,
> 1882–1904.
>
> Morrisey,M.B.(2016). Rejoinder: further considerations for meta-analysis
> of transformed quantities such as absolute values. Journal of Evolutionary
> Biology 29, 1922–1931.
>
> The second one has some R code that can help.
>
> Cheers,
> Dan
>
>
> –––
> Dr. Daniel Noble | ARC DECRA Fellow
> Level 5 West, Biological Sciences Building (E26)
> Ecology & Evolution Research Centre (E&ERC)
> School of Biological, Earth and Environmental Sciences (BEES)
> *The University of New South Wales*
> Sydney, NSW 2052
> AUSTRALIA
>
> T : +61 430 290 053
> E : daniel.noble using unsw.edu.au <daniel.noble using mq.edu.au>
> W: www.nobledan.com
> Github: https://github.com/daniel1noble
>
> On Tue, May 22, 2018 at 7:03 AM, Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>
>> Hi Dave,
>>
>> You cannot just take absolute values and proceed with standard methods.
>> As you noted, by taking absolute values, you end up with folded normal
>> distributions. My approach would be to use ML estimation where the absolute
>> values have folded normal distributions and then compute a profile
>> likelihood confidence interval for the mean parameter, since I suspect a
>> Wald-type CI would perform poorly.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bo
>> unces using r-project.org] On Behalf Of Dave Daversa
>> Sent: Monday, 21 May, 2018 13:47
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] effect size estimates regardless of direction
>>
>> ATTACHMENT(S) REMOVED: forest.plot.example.pdf | dummy.forest.plot.code.R
>>
>> Hi all,
>>
>> My question regards how to estimate overall magnitudes of effect sizes
>> from compiled studies regardless of the direction.  I have attached a
>> figure to illustrate, which I developed using made-up data and the attached
>> code.
>>
>> In the figure five studies have significantly positive effect sizes,
>> while 5 have significantly negative effect sizes.  Each have equal
>> variances.  So, the overall estimated mean effect size from a random
>> effects model is 0.   However, what if we simply want to estimate the mean
>> effect size regardless of direction (i.e. the average magnitude of
>> effects)?  In this example, that value would be 9.58 (CI: 6.48, 12.67),
>> correct?
>>
>> I have heard that taking absolute values of effect sizes generates an
>> upward bias in estimates of the standardized mean difference.  Also, this
>> would create a folded normal distribution, which would violate assumptions
>> of the model and would require an alternative method of estimating
>> confidence intervals.  What would be your approach to setting up a model
>> for answering the question of how much the overall magnitude of responses
>> is?
>>
>> I suspect this question has come up in this email group in the past.  If
>> so, my apologies for the redundancy, and please send me any reference that
>> may be helpful.
>>
>> Dave Daversa
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>
>


-- 
****************************************************************
*David Daversa, PhD*

*Postdoctoral Researcher*


*Institute for Integrative Biology, University of
Liverpoolddaversa using gmail.com <ddaversa using gmail.com>D.Daversa using liv.ac.uk
<ddaversa using wustl.edu>*
https://www.liverpool.ac.uk/integrative-biology/staff/david-daversa/
<http://www.zoo.cam.ac.uk/zoostaff/manica/drdaversa.htm>

	[[alternative HTML version deleted]]