[R-meta] effect size estimates regardless of direction

[Daniel Noble]

Hi Dave,

 nice paper in Biological reviews, Daniel.  The types of analyses I am
> doing are very similar, so that will be a good reference.
>
> In general, I am still confused about how to obtain information other than
> effect magnitudes from the analyze-then-transform method, because the
> models used for the approach use the raw means as the response.  So, other
> information,  such as variance estimation and the importance of various
> moderators in explaining variance, can not be obtained from the model
> outputs.
>

Yep, there are still some limitations to what can be done with this
approach at the moment. I am a little confused with this point though as
the approach is not a "modelling" approach *per se*. It simply transforms
means that have been estimated from a model that assumes a normal
probability distribution to what they would be if they were to come from a
folded normal distribution. Arguably, if you hypothsized certain
categorical moderators are important in explaining heterogenity then you
would want to estimate the overall mean estimate for each level regardless
of how much variance they explain. But, I do see your point – one maybe
interested in understanding variance explained assuming a folded normal.
I'm not aware of an easy solution on how to deal with this presently.

>
> Wolfgang, with your suggestion to apply a folded normal distribution to
> the means and obtaining profile likelihood CI's, I presume you were
> envisioning the absolute effect sizes being used as the response (yi) in
> the models?    That would be more representative of Morrisey's
> "transform-then-analyse" method, then?  Can a folded normal distribution be
> specified in rma.mv models?
>
> And to follow up again on previous points:
>
> Regarding question 1:  To deal with estimation of variance for moderator
> levels, I am not sure how to explicitly model the variance.  Doing a subset
> analysis sounds straightforward enough, but I am keen to explore the other
> option as well.   Any tips, Daniel?
>

I'm not sure how to in metafor, off hand, but with MCMCglmm it's fairly
easy by modifying the "rcov" argument. If you look at the code for the
paper I sent, it should show how to do this there (see links below).

>
> Regarding question 2:  I am only dealing with categorical moderators.  I
> was looking at the importance of various moderators in explaining effect
> sizes, using anova type analyses described here
> <http://www.metafor-project.org/doku.php/analyses:berkey1995>.  The
> absolute effects among different moderator levels is what I am truly
> interested in, so perhaps this analysis isn't necessary.
>

To me, it isn't. But I suppose it depends on your specific question. I
would just estimate the effect sizes and the credible intervals.

>
> 3.  I noticed that you used  the MCMCglmm package to run your bayesian
> models.  I have not used that package but am happy to have a look.
> I tried to find the code for Noble et al. 2018, to no avail.  If it is
> open access, let me know where I can find it. That would serve as a nice
> guide to navigate that package.
>

Very sorry. I should have given the DOI and link in last email. It should
be in the paper itself, but you can access the code here: DOI 10.17605/
OSF.IO/ZBGSS <http://osf.io/ZBGSS>or the link https://osf.io/zbgss/

Hope this helps,

Cheers,
Dan

	[[alternative HTML version deleted]]