[R-meta] Outlier and influential case analysis for multilevel meta-analysis with RVE

[Maximilian Steininger]

Dear all,

I am conducting a three-level meta-analysis where I have different dependency structures in my data. I model the dependency by approximating the var-cov matrix, followed by estimating a three-level model and then I apply robust variance estimation to compute my outcome (as suggested e.g. here: https://wviechtb.github.io/metafor/reference/misc-recs.html#general-workflow-for-meta-analyses-involving-complex-dependency-structures <https://wviechtb.github.io/metafor/reference/misc-recs.html#general-workflow-for-meta-analyses-involving-complex-dependency-structures>)

I wanted to do some sensitivity analysis on the model by running outlier and influential diagnostics. However, most of the proposed diagnostics do not work on "robust.rma" objects.

So far I did some model diagnostics by calculating cook's distance and hat values for my robust model (see e.g., https://wviechtb.github.io/metafor/reference/influence.rma.mv.html <https://wviechtb.github.io/metafor/reference/influence.rma.mv.html>). But as far as I am concerned these "only" give me information on influential cases and not outliers.

What is the best approach to check for outliers when using robust models? Are the two options below a sensible approach to check for outliers?

According to this source a possible but rather conservative approach is to label all studies as outliers that have confidence intervals that do not overlap with the confidence interval of the pooled effect. (see: https://cjvanlissa.github.io/Doing-Meta-Analysis-in-R/detecting-outliers-influential-cases.html <https://cjvanlissa.github.io/Doing-Meta-Analysis-in-R/detecting-outliers-influential-cases.html>).
Is it a feasible option to perform outlier diagnostics for the non-robust model as suggested e.g. by Viechtbauer & Cheung (2010; 10.1002/jrsm.11). My approach here would be to identify outliers based on the non-robust model --> exclude the outliers --> rerun the whole analysis without the outliers (i.e., approximate var-cov matrix, estimate three-level model, apply robust variance estimation for the subset of studies).
Or are there other, more elegant ways to do this?

Best and many thanks!
——

Mag. Maximilian Steininger
  PhD candidate

  Social, Cognitive and Affective Neuroscience Unit
  Faculty of Psychology
  University of Vienna

  Liebiggasse 5
  1010 Vienna, Austria

  e: maximilian.steininger using univie.ac.at
  w: http://scan.psy.univie.ac.at


	[[alternative HTML version deleted]]