[R-meta] Weighting studies combining inverse variance and quality score in multiple treatment studies

[Michael Dewey]

Dear Vivien

Would it not be better to use quality as a moderator? Admittedly with 
ten studies it might not be too helpful.

Michael

On 19/01/2018 17:04, Vivien Bonnesoeur wrote:
> Dear all,
> I would need some advice in the way to combine quality score and inverse
> variance for weighting studies.
> I'm contrasting the infiltration rate between tree plantation and grassland
> and also tree plantation and native forest (effect size = Log ROM) to know
> if tree plantation on grassland can increase the infiltration and recover
> to level of infiltration of native forest.
> 
> here is the raw data :
> article;trial;Land-use_change;Plantation_N;Plantation_mean;Plantation_sd;Control_N;Control_mean;Control_sd;yi;vi;quality_score
> Gaitan2016;1;Plantation-grassland;32;36;41;32;11;7;1.186;0.053;1
> Gonzalez2015;2;Plantation-grassland;9;76.6;17.6;2;76.6;37.5;0.000;0.126;0.8
> Hoyos2005;3;Plantation-grassland;3;101.3;66;23;2.5;1.6;3.702;0.159;0.5
> Hoyos2005;4;Plantation-Native_forest;3;101.3;66;3;225;271;-0.798;0.625;0.5
> Moreno2012;5;Plantation-grassland;10;8064;7092;10;5004;7092;0.477;0.278;0.3
> Moreno2012;6;Plantation-Native_forest;10;8064;7092;10;34092;7092;-1.442;0.082;0.3
> Sadeghian2001;7;Plantation-grassland;12;210;120;16;30;27;1.946;0.078;1
> Sadeghian2001;8;Plantation-Native_forest;12;210;120;16;760;439;-1.286;0.048;1
> Zimmerman2007;9;Plantation-grassland;30;514;137;30;3;4;5.144;0.062;0.8
> Zimmerman2007;10;Plantation-Native_forest;30;514;137;30;135;51;1.337;0.007;0.6
> 
> If I just used the inverse variance-covariance weighting (to account for
> dependency between the reuse of some plantations) :
> 
> model1 = rma.mv
> (yi,V,mods=~Land-use_change-1,method="REML",slab=article,random=~factor(trial)|article,data=ma.infilt)
> 
> I end with a lot of weight to the studies where there is a reuse of the
> plantation. Actually, those weight are really different from the inverse
> vi. For example
> Gaitan2016 : weight(model1) = 2.98%  ;  weight from inverse vi = 7.8%
> Hoyos2005.1 :weight(model1) = 9.01%  ;  weight from inverse vi = 2.6%
> Hoyos2005.2 :weight(model1) = 7.67%  ;  weight from inverse vi = 0.66%
> Zimmerman2007.1 :weight(model1) = 13.6%  ;  weight from inverse vi = 6.7%
> Zimmerman2007.2 :weight(model1) = 13.9%  ;  weight from inverse vi = 58%
> 
> Here I have a first question :
> -is there a way to reduce the weight of studies where the plantation is
> reused for contrasting with 2 different control? It seems to be an
> artificial over-weighting decision to me?
> 
> Besides, some studies with a low quality score have stronger weights than
> studies with high quality score. To combine the quality score and the
> inverse variance in study weighting, my try is to use the weight from the
> model1 and to multiply it with the quality score in this way :
> 
> model2=rma.mv
> (yi,V,mods=~Land-use_change-1,W=(ma.infilt$quality_score*weights(model1))/sum(ma.infilt$quality_score*weights(model1)),method="REML",slab=article,random=~factor(trial)|article,data=ma.infilt)
> 
> It gives more satisfactory weigths since the studies with very low quality
> score have now a small contribution to the grand mean.
> I would like to know however if this way of combining the quality and
> inverse variance weighting is sound theoretically and won't be rejected by
> reviewer as a "critical flaw"
> 
> Best regards
> 
> 
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html