[R-meta] order of effect sizes in data file changes results

[Viechtbauer, Wolfgang (SP)]

Dear Anna,

I do not know how you are creating the V matrix, so I cannot say what the best way is to sort the dataset. Generally speaking though, I would sort it by whatever variable(s) you use in lapply(split()) operations.

As for the negative correlations: Difficult to say without actually seeing the data. But note that there are relatively few pairs (e.g., levels 1 and 2 and levels 1 and 3 only occur 4 times together and levels 2 and 3 only once). So the estimates of these correlations are not going to be very accurate. This aside, it might actually make some sense to see negative correlations here. If more people are classified as Type I, then this could mean that fewer are classified as Type II and vice-versa. But one would rather expect a positive correlation between Type I and All and between Type II and All (we see the latter, but not the former).

Best,
Wolfgang 

>-----Original Message-----
>From: Van Meter, Anna [mailto:avanmeter using northwell.edu]
>Sent: Sunday, 30 December, 2018 23:21
>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>Subject: Re: order of effect sizes in data file changes results
>
>Dear Wolfgang,
>
>Thank you very much for your response to this and to my other question, I
>really appreciate your help!
>
>I have two follow-up questions:
>. In order to ensure that the V matrix is created in the right order for
>the analyses, in which I am estimating three outcomes, what is the best
>way to sort the data file? Currently I have it sorted first by reference
>(outer factor) and then by outcome (threegroup; inner factor).
>. When I run the following code, I get large, negative rho values on the
>off-diagonal. I'm not sure what this means conceptually or if it
>indicates  problems for downstream  analyses.
>
>resmvberkeyhybrid<-rma.mv(yi,   berkeyV, mods = ~ factor(threegroup),
>random = ~ threegroup |   reference, struct="UN",
>method="REML",data=kidtall1, digits=4,  slab =  kidtall1$reference)
>
>Multivariate Meta-Analysis Model (k = 28; method: REML)
>
>Variance Components:
>
>outer factor: reference  (nlvls = 19)
>inner factor: threegroup (nlvls = 3)
>
>            estim    sqrt  k.lvl  fixed  level
>tau^2.1    1.6240  1.2744     13     no      1
>tau^2.2    0.1989  0.4460      7     no      2
>tau^2.3    0.6270  0.7918      8     no      3
>
>     rho.1    rho.2    rho.3    1   2   3
>1        1  -0.8995  -0.7788    -  no  no
>2  -0.8995        1   0.9746    4   -  no
>3  -0.7788   0.9746        1    4   1   -
>
>Thank you!
>Anna
>
>________________________________________
>From: Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Sunday, December 30, 2018 10:27 AM
>To: Van Meter, Anna; r-sig-meta-analysis using r-project.org
>Subject: [EXTERNAL] RE: order of effect sizes in data file changes
>results
>
>External Email. Use Caution.
>
>Dear Anna,
>
>I suspect that the 'V' matrix (in your case, berkeyV) is not aligned with
>the data (yi). For example:
>
>library(metafor)
>dat <- dat.berkey1998
>dat
>
>V <- bldiag(lapply(split(dat[,c("v1i", "v2i")], dat$trial), as.matrix))
>V
>
># one can see here that the 2x2 blocks along the diagonal of V are in
>fact the variances and covariances as given by the variables 'v1i' and
>'v2i' in 'dat'
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial,
>struct="UN", data=dat)
>
># but now let's change the order of the data
>
>myorder <- order(dat$author)
>dat <- dat[myorder,]
>dat
>
># since the V matrix is unchanged, it is now not in the correct order
>anymore, so the following results are nonsense
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial,
>struct="UN", data=dat)
>
># so we have to order the V matrix in the same way as the data
>
>V <- V[myorder, myorder]
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial,
>struct="UN", data=dat)
>
># same results as in the beginning
>
>Also, one has to be careful when using things like split(). It will order
>the splits by the splitting variable:
>
>dat
>split(dat, dat$trial)
>
>Note that the order of the data in 'dat' and the splits are not in the
>same order. This can also lead to a misalignment. So, to be safe, first
>order the data by the variable that will be used in split() (as in the
>very beginning, where the data are already ordered by 'trial').
>
>Best,
>Wolfgang
>
>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>project.org] On Behalf Of Van Meter, Anna
>Sent: Sunday, 30 December, 2018 4:43
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] order of effect sizes in data file changes results
>
>Hello,
>
>As described in a previous post, I am conducting a meta analysis of
>bipolar disorder prevalence rates. Some studies report multiple
>prevalence rates; for example, one study could report the prevalence for
>bipolar I and for the full bipolar spectrum, which would include people
>with bipolar I, plus other people who have other subtypes of bipolar
>disorder. There are three potential prevalence categories: bipolar I,
>bipolar I & II, all bipolar.
>
>I am using the Berkey approach to account for the overlap in effect sizes
>and, based on a helpful response, have set the model up as follows to
>estimate the average prevalence for each subtype (threegroup is a
>dummycode for the subtype, articleno is the study ID):
>
>resmvberkeyhybrid<-rma.mv(yi, berkeyV, mods = ~ threegroup, random = ~
>threegroup | articleno, struct="UN", method="ML",data=kidtall1, digits=4)
>
>My question:
>
>I have noticed that the results of the model change depending on how the
>.csv data file is ordered, why would the order of the data file matter?
>And, what is the correct way to order the data file? My guess would be
>first by articleno and then by threegroup.
>
>Thank you for your help!
>
>Best,
>Anna