[R-meta] Plotting rstandard and rstudent values from cluster

[Viechtbauer Wolfgang (SP)]

The cluster level multivariate standardized residuals are computed per cluster, but the fitted values correspond to the estimates, so it isn't directly possible to plot the two against each other. Of course, one could compute a kind of cluster level fitted value (e.g., based on an average of the fitted values within a cluster), but I am not sure how useful that is. Also, assuming that there are no outlying clusters, the multivariate standardized residuals follow (asymptotically) chi^2 distributions with dfs equal to the number of estimates within a cluster. So, the values are all positive, should be right-skewed, and have different distributions (since the number of estimates will probably vary across clusters), so that complicates a comparison. I would just do:

sav <- rstandard(<model>, cluster=<cluster variable>)

or

sav <- rstudent(<model>, cluster=<cluster variable>)

and then:

sav$cluster$pval <- pchisq(sav$cluster$X2, df=sav$cluster$k, lower.tail=FALSE)
sav$cluster

and examine those p-values if any of them are really small. With a large number of clusters, one would expect some small p-values per chance alone, so I would be cautious about overinterpreting those results.

Best,
Wolfgang

-----Original Message-----
From: Martineau, Roger [mailto:Roger.Martineau at AGR.GC.CA] 
Sent: Friday, August 18, 2017 21:55
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: RE : Plotting rstandard and rstudent values from cluster

Dear Wolfgang,

yes, I would like to visualize residuals for the cluster studyID.

May be a scatterplot of standardized (and studentitized) residuals versus fitted values ?

best,

roger ☺

________________________________________
De : Viechtbauer Wolfgang (SP) [wolfgang.viechtbauer at maastrichtuniversity.nl]
Envoyé : 18 août 2017 11:08
À : Martineau, Roger; r-sig-meta-analysis at r-project.org
Objet : RE: Plotting rstandard and rstudent values from cluster

I am not sure I understand your question. If you do:

rstandard(multiplemoderator, cluster=dataset$studyID)
rstudent(multiplemoderator, cluster=dataset$studyID)

you will get a list with two elements. The second element is called 'cluster' and this contains the cluster (in this case, study) level multivariate (internally or externally) standardized residuals (and the number of estimates used to compute them). You want to plot those?

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Martineau, Roger
Sent: Friday, August 18, 2017 16:08
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Plotting rstandard and rstudent values from cluster

Dear metafor users,

I downloaded the data set (i.e., p154-dataset.csv) from  Assinka and Wibbelink (2016) available from http://www.tqmp.org/RegularArticles/vol12-3/p154/p154.pdf

The model is:

> multiplemoderator <- rma.mv(y, v, mods = ~ pyear + typeovert + typecovert,
+                             random =list(~ 1 | effectsizeID, ~ 1 | studyID),
+                             tdist=TRUE, data=dataset)

To plot standardized residuals, I do:

> #### STD residuals ###
> par(mfrow=c(1,1))
> plot(fitted(multiplemoderator), rstandard(multiplemoderator)$z, pch=19)
> abline(h=0)
> which(rstandard(multiplemoderator)$z > 3)
integer(0)

Recently, Dr Viechtbauer reported the following:

By the way, there is now also rstudent() for 'rma.mv' objects (also with a 'cluster' argument, option for parallel processing, and 'reestimate' argument). Also, rstandard() now has a cluster argument. When using the cluster argument with rstandard() and rstudent(), the functions also compute cluster-level multivariate (internally or externally) standardized residuals. So, all of the tools are there for proper outlier diagnostics in 'rma.mv' models (i.e., one can check for outlying estimates and clusters).

What should I write in metafor to plot rstandard and rstudent values for each cluster separately.

Thanks in advance,

Roger ☺