[R-meta] order of effect sizes in data file changes results

[Van Meter, Anna]

Dear Wolfgang,

Thank you for this additional clarification. I appreciate it.

Best,
Anna

-----Original Message-----
From: Viechtbauer, Wolfgang (SP) [mailto:wolfgang.viechtbauer using maastrichtuniversity.nl] 
Sent: Monday, January 07, 2019 10:45 AM
To: Van Meter, Anna <avanmeter using northwell.edu>; r-sig-meta-analysis using r-project.org
Subject: [EXTERNAL] RE: order of effect sizes in data file changes results

 External Email. Use Caution.

Dear Anna,

I do not know how you are creating the V matrix, so I cannot say what the best way is to sort the dataset. Generally speaking though, I would sort it by whatever variable(s) you use in lapply(split()) operations.

As for the negative correlations: Difficult to say without actually seeing the data. But note that there are relatively few pairs (e.g., levels 1 and 2 and levels 1 and 3 only occur 4 times together and levels 2 and 3 only once). So the estimates of these correlations are not going to be very accurate. This aside, it might actually make some sense to see negative correlations here. If more people are classified as Type I, then this could mean that fewer are classified as Type II and vice-versa. But one would rather expect a positive correlation between Type I and All and between Type II and All (we see the latter, but not the former).

Best,
Wolfgang 

>-----Original Message-----
>From: Van Meter, Anna [mailto:avanmeter using northwell.edu]
>Sent: Sunday, 30 December, 2018 23:21
>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>Subject: Re: order of effect sizes in data file changes results
>
>Dear Wolfgang,
>
>Thank you very much for your response to this and to my other question, 
>I really appreciate your help!
>
>I have two follow-up questions:
>. In order to ensure that the V matrix is created in the right order 
>for the analyses, in which I am estimating three outcomes, what is the 
>best way to sort the data file? Currently I have it sorted first by 
>reference (outer factor) and then by outcome (threegroup; inner factor).
>. When I run the following code, I get large, negative rho values on 
>the off-diagonal. I'm not sure what this means conceptually or if it 
>indicates  problems for downstream  analyses.
>
>resmvberkeyhybrid<-rma.mv(yi,   berkeyV, mods = ~ factor(threegroup), 
>random = ~ threegroup |   reference, struct="UN", 
>method="REML",data=kidtall1, digits=4,  slab =  kidtall1$reference)
>
>Multivariate Meta-Analysis Model (k = 28; method: REML)
>
>Variance Components:
>
>outer factor: reference  (nlvls = 19)
>inner factor: threegroup (nlvls = 3)
>
>            estim    sqrt  k.lvl  fixed  level
>tau^2.1    1.6240  1.2744     13     no      1
>tau^2.2    0.1989  0.4460      7     no      2
>tau^2.3    0.6270  0.7918      8     no      3
>
>     rho.1    rho.2    rho.3    1   2   3
>1        1  -0.8995  -0.7788    -  no  no
>2  -0.8995        1   0.9746    4   -  no
>3  -0.7788   0.9746        1    4   1   -
>
>Thank you!
>Anna
>
>________________________________________
>From: Viechtbauer, Wolfgang (SP)
><wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Sunday, December 30, 2018 10:27 AM
>To: Van Meter, Anna; r-sig-meta-analysis using r-project.org
>Subject: [EXTERNAL] RE: order of effect sizes in data file changes 
>results
>
>External Email. Use Caution.
>
>Dear Anna,
>
>I suspect that the 'V' matrix (in your case, berkeyV) is not aligned 
>with the data (yi). For example:
>
>library(metafor)
>dat <- dat.berkey1998
>dat
>
>V <- bldiag(lapply(split(dat[,c("v1i", "v2i")], dat$trial), as.matrix)) 
>V
>
># one can see here that the 2x2 blocks along the diagonal of V are in 
>fact the variances and covariances as given by the variables 'v1i' and 
>'v2i' in 'dat'
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial, 
>struct="UN", data=dat)
>
># but now let's change the order of the data
>
>myorder <- order(dat$author)
>dat <- dat[myorder,]
>dat
>
># since the V matrix is unchanged, it is now not in the correct order 
>anymore, so the following results are nonsense
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial, 
>struct="UN", data=dat)
>
># so we have to order the V matrix in the same way as the data
>
>V <- V[myorder, myorder]
>
>rma.mv(yi, V, mods = ~ outcome - 1, random = ~ outcome | trial, 
>struct="UN", data=dat)
>
># same results as in the beginning
>
>Also, one has to be careful when using things like split(). It will 
>order the splits by the splitting variable:
>
>dat
>split(dat, dat$trial)
>
>Note that the order of the data in 'dat' and the splits are not in the 
>same order. This can also lead to a misalignment. So, to be safe, first 
>order the data by the variable that will be used in split() (as in the 
>very beginning, where the data are already ordered by 'trial').
>
>Best,
>Wolfgang
>
>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-
>project.org] On Behalf Of Van Meter, Anna
>Sent: Sunday, 30 December, 2018 4:43
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] order of effect sizes in data file changes results
>
>Hello,
>
>As described in a previous post, I am conducting a meta analysis of 
>bipolar disorder prevalence rates. Some studies report multiple 
>prevalence rates; for example, one study could report the prevalence 
>for bipolar I and for the full bipolar spectrum, which would include 
>people with bipolar I, plus other people who have other subtypes of 
>bipolar disorder. There are three potential prevalence categories: 
>bipolar I, bipolar I & II, all bipolar.
>
>I am using the Berkey approach to account for the overlap in effect 
>sizes and, based on a helpful response, have set the model up as 
>follows to estimate the average prevalence for each subtype (threegroup 
>is a dummycode for the subtype, articleno is the study ID):
>
>resmvberkeyhybrid<-rma.mv(yi, berkeyV, mods = ~ threegroup, random = ~ 
>threegroup | articleno, struct="UN", method="ML",data=kidtall1, 
>digits=4)
>
>My question:
>
>I have noticed that the results of the model change depending on how 
>the .csv data file is ordered, why would the order of the data file matter?
>And, what is the correct way to order the data file? My guess would be 
>first by articleno and then by threegroup.
>
>Thank you for your help!
>
>Best,
>Anna


The information contained in this electronic e-mail transmission and any attachments are intended only for the use of the individual or entity to whom or to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this communication is not the intended recipient, or the employee or agent responsible for delivering this communication to the intended recipient, you are hereby notified that any dissemination, distribution, copying or disclosure of this communication and any attachment is strictly prohibited. If you have received this transmission in error, please notify the sender immediately by telephone and electronic mail, and delete the original communication and any attachment from any computer, server or other electronic recording or storage device or medium. Receipt by anyone other than the intended recipient is not a waiver of any attorney-client, physician-patient or other privilege.