[R-meta] Questions on partial d-type effect sizes

[Carla Gomez Creutzberg]

Dear fellow meta-analysts:



My apologies for a very long and detailed message, but I am trying to wrap
my head around an analysis that is quite unique and thought it might be
best if I provided a comprehensive explanation of it.


Iam trying to work my way through a meta-analysis aimed at separating the
variance explained by two variables (land use – a factor, and biodiversity
– a continuous variable) in the provision of ecosystem services. For this I
have a number of studies that report both the biodiversity and the
provision of an ecosystem service within two or more land uses.



For my analysis I would like to compare estimates of the provision of
ecosystem services and their corresponding confidence intervals when only
land use is included against estimates of service provision when both
biodiversity and land use are included. I would do this for the different
pairwise combinations of land uses in my studies. I have therefore
considered using the partial d-effect size proposed by Keef & Roberts
(2004) as my effect size metric. This would allow me to quantify the
differences between pairs of land uses in ecosystem service provision under
two regression models a) one with only land use and b) one with land use
and biodiversity.



 To this, per study, I would generate models and extract the regression
coefficients of how pairs of land uses compare to each other from:

a)      An ANOVA on the effect of land use on ecosystem service provision,

b)      An ANCOVA on the effect of land use and biodiversity on ecosystem
service provision.

With these coefficients I would then calculate the corresponding partial
d-effect sizes and their variances and try to follow the approach proposed
by Keef & Roberts (2004) to synthesize partial d-effect sizes across
studies. I am assuming that the variance of Keef & Robert’s partial
d-effect size can also be used to construct confidence intervals for each
estimate which will allow me to compare the overall estimates with and
without biodiversity for each pair of land uses.



My approach is complicated by the fact that each study presents data on a
different set of land uses so, for each study, I will have to run not just
one but several ANOVAs and ANCOVAs with different land uses as the
intercept so that I can extract the coefficients of how all pairs of land
uses compare against each other in the study and from there calculate the
corresponding partial d-effect size estimations. For example if one study
has land uses A, B and C I will have to run an ANOVA and ANCOVA with land
use A as a baseline (to get estimates of how A compares to B and to C) and
with B as a baseline (to get the coefficients that´ll show how B compares
to C, and B to A but that one we already have from the first model).



Upon trying to implement this approach I have come across a few issues I
would like to ask you all about:



1.       The denominator used in the calculation of the partial d-effect
size estimator proposed by Keef & Roberts (2004) is the standard deviation
of the residuals in each ANOVA or ANCOVA. However, these authors warn
against the fact that this value is sensitive to the number of covariates
that are included in each model and that it tends to decrease in size when
more covariates are added to the model. This decrease, in turn, inflates
the value of the estimated effect size. In my case this means that the
effect size estimates of any given pair of land uses would be greater for
studies that have more land uses than those that have fewer. To overcome
this problem, Keef and Roberts (2004) suggest using a standard deviation of
the residuals based on a model that doesn’t include all covariates of a
given study but those that are common across all studies. In my case, this
would mean using a model for the standard deviation of the residuals with
only the land uses in a given study that are common to all studies and
which may be, at most, one pair of land uses.


2.       I am also struggling to follow the method proposed by Keef and
Roberts to calculate the variance of their effect size estimator, since it
is based on using a gamma function. I was wondering if anybody had
attempted to implement these calculations or knew of a more straightforward
way to go about them?


3.       Keef and Roberts only propose a method for summarizing their
effect size within a fixed effects model. Although they present a method
for assessing heterogeneity and determining whether fixed effects can be
assumed, they are not very clear about what can be done when fixed effects
do not apply.



Finally if anybody has any idea of an alternative approach I could follow,
or knows of any further developments that have been made since the
publication of the Keef and Roberts paper on partial d- type effect sizes
and their synthesis with meta-analysis I would really appreciate hearing
about it.



The reference to the Keef and Roberts paper is as follows:

Keef S & Roberts LA, 2004. The meta-analysis of partial effect sizes. *British
Journal of Mathematical and Statistical Psychology*, 57: 97 – 129.



Thanks for your attention.


-- 
*Carla Gómez Creutzberg*
PhD. Candidate - Tylianakis Lab
<http://www.tylianakislab.org/the-group.html>
University of Canterbury - *Te Whare Wānanga o Waitaha*
Christchurch, New Zealand <http://www.tylianakislab.org/the-group.html>
cgomezcre at gmail.com

	[[alternative HTML version deleted]]