<div dir="ltr">Vojtech,<div><br></div><div>I do not know enough about the performance of the adjustment to be able to unequivocally recommend it or not. All the same, I will offer a couple of observations in case they are useful to you:</div><div><br></div><div>1. The adjustments described by Doncaster & Spake are very similar to methods proposed by Hunter & Schmidt in their book, Methods of Meta-Analysis. So they are not entirely unknown. </div><div>2. This adjustment should only matter much if you are dealing with exceedingly small sample sizes, which as Doncaster & Spake demonstrate are not uncommon in ecology. If your sample sizes are much larger (say, smallest total sample sizes are in the 20's, not the single digits), then perhaps it is less of a concern. </div><div>3. The range of effect size estimates is also a consideration. In psychology and education, I don't usually think about standardized mean differences bigger than 1 or 1.5. For SMDs larger than 3, I often start to wonder whether a different effect size metric might be more appropriate. </div><div>4. An ideal way to address your question about whether to use the adjustment method would be to run some simulations that emulate the conditions (sample sizes, ranges of effects, number of studies) you observed in your meta-analysis. The authors provide R code for their simulations, which could be modified to resemble the conditions in your meta. But of course nobody has unlimited time and resources so this might not be feasible.</div><div>5. I think it would useful to also report standard errors/confidence intervals based on other techniques, such as the Knapp-Hartung adjustment or Sidik & Jonkman's robust standard errors. Reporting results based on these other techniques would, I think, help to build the reader's confidence that your ultimate findings are credible rather than being contingent on use of an uncommon set of methods. The Knapp-Hartung adjustment is available in metafor using test = "knha". Robust standard errors can be calculated using robust() in metafor or coef_test() in the clubSandwich package. In either case, you would specify a unique id variable for the cluster = argument.</div><div><br></div><div>James </div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 27, 2018 at 6:56 AM, Vojtěch Brlík <span dir="ltr"><<a href="mailto:vojtech.brlik@gmail.com" target="_blank">vojtech.brlik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><font face="verdana, sans-serif">Dear all,<br><br>I have conducted a meta-analysis for my bachelor thesis (that means I am highly inexperienced) using the unbiased standardized mean difference (Hedges‘ g) as a measure of the effect size. I have noticed recently published study (<a href="https://doi.org/10.1111/2041-210X.12927" target="_blank">https://doi.org/10.1111/2041-<wbr>210X.12927</a>) suggesting the adjustment in the standard error calculation as the weights of the effect sizes are not corresponding to their sample sizes symmetrically. This inequality causes the biased estimates of pooled effect size variance.<br><br>I decided to use this adjustment but it does not cause the same adjustment in all same-sized studies as the differences between the adjusted and <div class="gmail_default" style="font-family:verdana,sans-serif;display:inline">non</div><div class="gmail_default" style="font-family:verdana,sans-serif;display:inline">-</div>adjusted errors are not symmetric (see below the plots in four categories of effect I want to recalculate<div class="gmail_default" style="font-family:verdana,sans-serif;display:inline">, also attached below</div>).<br><br><img src="cid:ii_jf9lp5l22_162674c970ef7a75" width="358" height="299" style="margin-right:0px"><br><br><br>Please, write me in case you cannot see the figures.<br><br>However, the effect size<div class="gmail_default" style="font-family:verdana,sans-serif;display:inline">s</div> remain unchanged and the variance is wider as Doncaster & Spake 2018 suggested.<br><br>What is you opinion about this study, do you recommend the use the adjustment for the standard error calculation or not?<br><br>Thank you for your advises and comments.<br><br>With kind regards,<br><br>Vojtech<div class="gmail_default" style="font-family:verdana,sans-serif;display:inline"> Brlik</div></font></div>
<br>______________________________<wbr>_________________<br>
R-sig-meta-analysis mailing list<br>
<a href="mailto:R-sig-meta-analysis@r-project.org">R-sig-meta-analysis@r-project.<wbr>org</a><br>
<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis" rel="noreferrer" target="_blank">https://stat.ethz.ch/mailman/<wbr>listinfo/r-sig-meta-analysis</a><br>
<br></blockquote></div><br></div>