[R-meta] Do the results of rma.mv() depend on how dataframe rows are ordered/arranged?

[Gabriele Midolo]

Dear all,

I noticed that the results of my rma.mv() model strongly changes when the
dataframe rows are arranged in a different order. This is new to me and I
can’t really understand why. I thought it was an issue related to how the V
matrix get computed (?), but they are actually identical independently from
rows order in the dataframe… how is a dataframe supposed to be arranged
before you can trust rma.mv() results then? Thanks.


EXAMPLE:

calc.v <- function(x) {

  v <- matrix((x$sdC_imputed[1]^2 / (x$nC[1] * x$C[1]^2)) , nrow=nrow(x),
ncol=nrow(x))

  diag(v) <- x$vi_Bracken

  v

}

random = ~ 1 | study/ID


If I run the following:



a<-df%>%filter(TRAIT=="LA")

dat<-a%>%mutate(sdC_imputed=sdC)%>%ungroup()

dat<-metagear::impute_SD(dat, "sdC_imputed", "C", method = "Bracken1992")

V <- metafor::bldiag(lapply(split(dat, dat$ctrl_id), calc.v))

m<-rma.mv(yi~dTf,V,data=dat,random = random)

m



Multivariate Meta-Analysis Model (k = 34; method: REML)



Variance Components:



            estim    sqrt  nlvls  fixed    factor

sigma^2.1  0.0000  0.0000      7     no     study

sigma^2.2  0.0607  0.2464     34     no  study/ID



Test for Residual Heterogeneity:

QE(df = 32) = 1024.8720, p-val < .0001



Test of Moderators (coefficient 2):

QM(df = 1) = 1.6018, p-val = 0.2056



Model Results:



         estimate      se     zval    pval    ci.lb   ci.ub

intrcpt   -0.0786  0.0538  -1.4617  0.1438  -0.1841  0.0268

dTf       -0.0233  0.0184  -1.2656  0.2056  -0.0593  0.0128



If I run the same code but arrange the dataframe by e.g. the ID of the
observation via dplyr::arrange(),

a<-df%>%filter(TRAIT=="LA")%>%arrange(ID)


Then I get the following output… i.e. a completely different result:

Multivariate Meta-Analysis Model (k = 34; method: REML)



Variance Components:



            estim    sqrt  nlvls  fixed    factor

sigma^2.1  0.0000  0.0000      7     no     study

sigma^2.2  0.0336  0.1834     34     no  study/ID



Test for Residual Heterogeneity:

QE(df = 32) = 334.3701, p-val < .0001



Test of Moderators (coefficient 2):

QM(df = 1) = 13.9681, p-val = 0.0002



Model Results:



         estimate      se     zval    pval    ci.lb    ci.ub

intrcpt   -0.0740  0.0416  -1.7808  0.0749  -0.1554   0.0074    .

dTf       -0.0668  0.0179  -3.7374  0.0002  -0.1019  -0.0318  ***

	[[alternative HTML version deleted]]