[R-meta] Meta-analysis of multiple treatment studies

[Viechtbauer Wolfgang (SP)]

Hi Jonas,

My preference would be Model 3, with random effects for "ID", "control group", and "experiment" (or only one of the latter two, especially if there are only few experiments with multiple control groups). You have all of the information needed to compute the covariances between ROM values sharing a common control group, so why ignore the covariances and then rely on cluster-robust inference methods to fix things up? You could still use that approach as a sensitivity analysis -- but I would base it on a good working model and that would be something along the lines of Model 3.

As for normality -- yes, if important moderators/predictors are omitted from the model, then this could lead to non-normally distributed standardized residuals.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Jonas Duus Stevens Lekfeldt
Sent: Monday, 06 November, 2017 14:40
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Meta-analysis of multiple treatment studies

Dear all

I am currently conducting a meta-analysis of results from a number (~140) pot and field experiments with plants (wheat, maize and tomato). We are basically analyzing if adding different types of bacteria, fungi, seaweed extracts etc. make the plants grow better (produce larger yields). I am using the log(response ratio) or "ROM" as the effect size.

In this analysis I have a multi-level data structure since in all experiments more than one specific treatment was compared to the same control. Furthermore an experiment may also include treatments which do not share a common control. I want to investigate the effect of a number of moderators (eg. type of organism added, plant species investigated etc.). 

The data structure looks something like this (it is much longer this is just an example and I have omitted the moderator columns here for simplicity):

ID	Treatment	Experiment	ControlName		mean_control	sd_control	mean_treated	sd_treated	n1i		n2i
1	TriP		UHOH_2013	UHOH_2013_A		number		number		number		number		number		number	
2	Pro		UHOH_2013 	UHOH_2013_A		number		number		number		number		number		number
3	RhVi		UHOH_2013	UHOH_2013_A		number		number		number		number		number		number
4	TriP		UCPH_2013	UCPH_2013_A		number		number		number		number		number		number
5	BioDC		UCPH_2013	UCPH_2013_A		number		number		number		number		number		number
6	TriP		UCPH_2013	UCPH_2013_B		number		number		number		number		number		number
7	BioDC		UCPH_2013	UCPH_2013_B		number		number		number		number		number		number
8	E		FiBL_2013	FiBL_2013_A		number		number		number		number		number		number
9	F		FiBL_2013	FiBL_2013_A		number		number		number		number		number		number
10	G		FiBL_2013	FiBL_2013_A		number		number		number		number		number		number

Where 
m1i: mean of control (shared between some of the treatments)
sd1i: standard deviation of control
m2i: mean of the treated
sd2i: standard deviation of treated
n1i: sample size of control
n2i: sample size of control

It should be noted that the sample sizes n1i and n2i normally are 5 or smaller.

I have considered three different models:

Model 1: 
So far I have used the rma.uni() function and then the robust() function with the cluster argument indicating groups of observations that are sharing a common control (cluster="ControlName") to model the data 

However, I am not sure if this is correct since as I see it doing it this way the model does not include the random effects of "control groups" and "experiment" when calculating the overall effect only in the calculation of the CI. Did I understand this correctly?
One thing I just discovered with this model is that the data do not seem to be normally distributed when I use the function qqnorm.rma.uni() when looking at a simple model without moderators. Is this a serious problem? And if moderators are significant (for instance different effect in different crops) would you not expect the data to deviate from normality when running a simple model without moderators. Or am I missing something?

I have considered to model the data using rma.mv() instead. I have considered the following two models:

Model 2: 
Including "ID", "control group" and "experiment" (or only "ID" and "control group") as random variables in rma.mv() and then subsequently use robust() to correct the CI for the dependence on a common control treatment within experiments. 

Model 3: 
Including a calculated variance-covariance matrix (V argument in rma.mv()). Here I ran into problems with the var-covar matrix when all data were included. 

Which of the three Models (if any) would people in the forum advise me to use? Is model 1 completely wrong?

Best regards

Jonas Duus Stevens Lekfeldt