[R-meta] Comparing effect sizes in multivariate meta analysis

[McCutcheon, Robert]

Dear James and Wolfgang,

Thank you very much for these clear and helpful responses

Best Wishes

Rob


On 31 October 2018 at 02:16:15, James Pustejovsky (jepusto using gmail.com<mailto:jepusto using gmail.com>) wrote:

To add to Wolfgang's suggestions, it's also possible to test for differences between positive and negative symptoms using robust variance estimation. The advantage of doing so is that the hypothesis test/confidence interval is not predicated on having accurately imputed the within-study correlations between effect size estimates. Example code below.

James

library(clubSandwich)

# Separate intercepts model
MultiMeta <- rma.mv<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frma.mv%2F&data=01%7C01%7Crobert.mccutcheon%40kcl.ac.uk%7Ccd70f12c7aac46d4608508d63ed6d3e7%7C8370cf1416f34c16b83c724071654356%7C0&sdata=YEmDFUyAjcPto6J0Qu3Y2fxHBQHL5pfnckg%2FPXntqC8%3D&reserved=0>(yi = yi, V = Vlist, mods = ~factor(posneg)-1, random = ~factor(posneg)|studyid, struct = "UN", data = mydata_multi)
Wald_test(MultiMeta, constraints = matrix(c(-1,1), 1), vcov = "CR2")

# With reference level
MultMeta <- rma.mv<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frma.mv%2F&data=01%7C01%7Crobert.mccutcheon%40kcl.ac.uk%7Ccd70f12c7aac46d4608508d63ed6d3e7%7C8370cf1416f34c16b83c724071654356%7C0&sdata=YEmDFUyAjcPto6J0Qu3Y2fxHBQHL5pfnckg%2FPXntqC8%3D&reserved=0>(yi = yi, V = Vlist, mods = ~factor(posneg), random = ~factor(posneg)|studyid, struct = "UN", data = mydata_multi)
coef_test(MultMeta, vcov = "CR2")
conf_int(MultMeta, vcov = "CR2")

On Tue, Oct 30, 2018 at 1:33 PM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl<mailto:wolfgang.viechtbauer using maastrichtuniversity.nl>> wrote:
Hi Rob,

You probably want to use struct="UN" here:

MultiMeta <- rma.mv<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frma.mv&data=01%7C01%7Crobert.mccutcheon%40kcl.ac.uk%7Ccd70f12c7aac46d4608508d63ed6d3e7%7C8370cf1416f34c16b83c724071654356%7C0&sdata=IGLd5%2BjoQfeOwhXoj188PpixTFJOLjcZThqRAyRJI3Y%3D&reserved=0>(yi = yi, V = Vlist, mods = ~factor(posneg)-1, random = ~factor(posneg)|studyid, struct = "UN", data = mydata_multi)

Using struct="CS" assumes that the amount of heterogeneity is the same for positive and negative symptoms, which may not be the case.

To test for a difference between positive and negative symptoms, you can compute the contrast between the two estimates with:

anova(MultiMeta, L=c(-1,1))

Alternatively, fit the model with 'mods = ~factor(posneg)'. Then one of the two levels becomes the reference level (and hence the intercept) and the coefficient for the other level is the *difference* between the two levels. You should get the same p-value for this coefficient as when computing the contrast as shown above.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org<mailto:r-sig-meta-analysis-bounces using r-project.org>] On Behalf Of McCutcheon, Robert
Sent: Tuesday, 30 October, 2018 17:14
To: r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
Subject: [R-meta] Comparing effect sizes in multivariate meta analysis

Dear All,

I have data from a number of randomised placebo controlled trials, each trial reports data on the effect  of the drug on both ‘positive’ and ‘negative’ symptoms. I wish to determine whether the drug has a greater effect on negative or positive symptoms

As I do not know the within-study correlations I have estimated a covariance matrix using the ClubSandwich package, and intend to run the analysis with a range of values for ‘r’:

Vlist <- impute_covariance_matrix(vi = multistudy[[6]]$vi, cluster = mydata_multi$studyid, r=0.5 )

I then conduct the multivariate analysis as follows:

MultiMeta <- rma.mv<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Frma.mv&data=01%7C01%7Crobert.mccutcheon%40kcl.ac.uk%7Ccd70f12c7aac46d4608508d63ed6d3e7%7C8370cf1416f34c16b83c724071654356%7C0&sdata=IGLd5%2BjoQfeOwhXoj188PpixTFJOLjcZThqRAyRJI3Y%3D&reserved=0>(yi = yi, V = Vlist, mods = ~factor(posneg)-1, random = ~factor(posneg)|studyid, struct = "CS", data = mydata_multi)

Where yi is the calculated effect size, ‘posneg’ is the label describing whether the results of that row refers to positive or negative symptoms, and ‘studyid’  is a separate label for each study.

This gives me separate effect sizes for positive and negative symptoms but I wonder if anyone could advise how I test whether these effect sizes are significantly different form one another, i.e. whether the drug has a greater effect on positive as compared to negative symptoms?

Many thanks for your time,

Rob
_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-meta-analysis&data=01%7C01%7Crobert.mccutcheon%40kcl.ac.uk%7Ccd70f12c7aac46d4608508d63ed6d3e7%7C8370cf1416f34c16b83c724071654356%7C0&sdata=AH6px1gpK5X4Oel3d7ah1pgli%2BF6kXV7J2bloBlI9OQ%3D&reserved=0>

	[[alternative HTML version deleted]]