[R-meta] What to do if the residuals are not normally distributed?

[Viechtbauer Wolfgang (SP)]

The standardized residuals from the robust model are not useful for diagnostic purposes. That is actually one of the disadvantages when using (cluster) robust inference methods. Some methods we might want to apply to the adjusted model object just don't make much sense anymore.

Given that the results from robust() are very similar to what you obtained from rma.mv() directly, the non-normality might not be a major issue here. But I want to emphasize that I have not examined myself to what extent cluster-robust methods are really an adequate approach for diagnosing model misspecification and/or handling non-normality in the first place -- I am basing this just on a very cursory reading of Maas and Hox (2004) (who examined the issue in the context of a primary data analysis using multilevel models, not in a meta-analytic context).

>Ps. How can I best acknowledge your contribution to my paper, Wolfgang?

Send me some beer.

Just kidding. Don't worry about it. If you insist, an acknowledgement in the paper would be fine.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>project.org] On Behalf Of Jonas Duus Stevens Lekfeldt
>Sent: Friday, 10 November, 2017 12:35
>To: r-sig-meta-analysis at r-project.org
>Subject: Re: [R-meta] What to do if the residuals are not normally
>distributed?
>
>ATTACHMENT(S) REMOVED: QQnorm_robust.pdf
>
>Thanks again!
>
>I tried both robust(model, cluster=dat$ExpName) and robust(model,
>cluster=dat$ControlName) and they both essentially gave the same result
>(see attached PDF).
>
>This plot was drawn using:
>
>>meta_resid <- rstandard.rma.mv(robust_model)
>>z_val <- meta_resid$z
>>qqnorm(z_val)
>>qqline(z_val)
>
>I am unsure how to interpret this result. Now the "se" of the estimates
>are closely following the residuals obtained which results in what looks
>like a bi-modal distribution of the z-values (around -1 and +1, or am I
>mistaken?).
>
>So really my question to all of you is: do you believe that I need to do
>any further correction or could I use the original model (using the var-
>covar matrix) even though the distribution of residuals is too broad?
>With the note of caution that I may be underestimating the width of the
>CIs? As far as I understood the former post by Wolfgang the non-normality
>may not be such a big problem regarding the fixed effects, after all.
>
>And then again: I have not yet removed outliers from the dataset which
>may also help to correct the problem. The question is of course if one
>would like to do that, but none the less it could be used as a
>sensitivity analysis.
>
>Given my extremely limited knowledge of Bayesian statistics, JAGS and
>related subjects implementing the last suggestion by Wolfgang is
>unfortunately not a feasible way to go forward for me.
>
>Best
>
>Jonas
>
>Ps. How can I best acknowledge your contribution to my paper, Wolfgang?