[R-meta] large increase of the QM value after applying robust on rma.mv

[Lea BRIARD]

Hi all,

I first want to apologize if my email is too long or if I don't provide the right information. This is my first email to a R community.

I currently running a meta-analysis using the rma.mv function. 

Here are some information that could be useful

I have a dataset with 210 observations, 23 studies and 16 species. I study the link between parasite and host behaviour. 
Each study is on one particular group of individuals, but the different observations are different behaviour and infection measures calculated for the same individuals (sometimes in the same study I have more than one species of parasites, and for each parasites multiple measures of host behaviour).


After reading these two threads
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-August/000130.html
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-November/000351.html

I followed these steps :
1° calculation of effect sizes with escalc ( r converted to Fisher's Z)
2° I have a three-level random factor (1|species/studyID/observation) (I compared different versions with different random factors using anova() to choose the best version).
I use REML, optimizer = optim and optmethod=Nelder-Mead

3° Imputation of the variance-covariance matrix use the impute_covariance_matrix (data$vi, cluster=studyID, r = 0 or r = 0.5) 


4° Use the robust function with cluster=data$studyID. I'm not using the factor with the smallest number of levels (species) because the risk of interdependence between observations is within studyID (one studyID= same group of individuals). I use robust it's the residuals are not normally distributed (the profile likelihood plots of the sigma components indicate strong parameter identifiability.

5° run intercept-only model

6° run 5 univariate model with different moderators (host characteristics, parasite characteristics) and ranking the 6 models using AICc

I'm running two versions of these models, a non-phylogenetically controlled and a phylogenetically controlled one (using a correlation matrix computed with vcv.phylo) with a R=list(species=matrix) and Rscale = cor0.

after struggling for months (it's my first meta-analysis) I thought I was finally out of the woods thanks to all the amazing material that I could find online (thank you so much Wolfgang and James) BUT I have one model with one particular moderator where the QM value goes through the roof after using the robust function and this happens ONLY in the non-phylogenetically controlled (without the R component). 
My QE is between 616 (intercept only) and 523 (best model)
MY QM after using robust is between 2.9 and 0.39 except for that particular moderator where QM = 673. For that troublemaking moderator, I have three levels with the following sample size level 1 = 179, level 2 = 10, level = 21. 
If I use another version of that moderator (more levels, but more evenly distributed) it's even worst I have a QM > 100000000


I tried to read as extensively as possible the metafor project website, as well as James Pustejovsky's website and the threads in the R-sig-meta-analysis mailing list but I can't find an explanation for this. I specially want to know if it means that this moderator is not usable (because of the sample size, with one particular level being overrepresented) or if I'm doing something wrong in general and this moderator is just a symptom.

King Regards
Léa Briard