[R] metafor package: effect sizes are not fully independent

[Mike Cheung]

Dear Gang,

It seems that it is possible to use a univariate meta-analysis to
handle your multivariate effect sizes. If you want to calculate a
weighted average first, Hedges and Olkin (1985) has discussed this
approach.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for
meta-analysis. Orlando, FL: Academic Press.

Regards,
Mike
-- 
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://courses.nus.edu.sg/course/psycwlm/internet/
---------------------------------------------------------------------

On Mon, Feb 8, 2010 at 6:48 AM, Gang Chen <gangchen6 at gmail.com> wrote:
> Dear Mike,
>
> Thanks a lot for the kind help!
>
> Actually a few months ago I happened to read a couple of your posts on
> the R-help archive when I was exploring the possibility of using lme()
> in R for meta analysis.
>
> First of all, I didn't specify the meta analysis model for my cases
> correctly in my previous message. Currently I'm only interested in
> random- or mixed-effects meta analysis. So what you've suggested is
> directly relevant to what I've been looking for, especially for case
> (2). I'll try to gather those references you listed, and figure out
> the details.
>
> Also I think I didn't state my case (1) clearly in my previous post.
> In that case, all the effect sizes are the same and in the same
> condition too (e.g., happy), but each source has multiple samples of
> the measurement (and also measurement error, or standard error). Could
> this still be handled as a multivariate meta analysis since the
> samples for the the same source are correlated? Or somehow the
> multiple measures from the same source can be somehow summarized
> (weighted average?) before the meta analysis?
>
> Your suggestions are highly appreciated.
>
> Best wishes,
> Gang
>
>
> On Sun, Feb 7, 2010 at 10:39 AM, Mike Cheung <mikewlcheung at gmail.com> wrote:
>> Dear Gang,
>>
>> Here are just some general thoughts. Wolfgang Viechtbauer will be a
>> better position to answer questions related to metafor.
>>
>> For multivariate effect sizes, we first have to estimate the
>> asymptotic sampling covariance matrix among the effect sizes. Formulas
>> for some common effect sizes are provided by Gleser and Olkin (2009).
>>
>> If a fixed-effects model is required, it is quite easy to write your
>> own GLS function to conduct the multivariate meta-analysis (see e.g.,
>> Becker, 1992). If a random-effects model is required, it is more
>> challenging in R. SAS Proc MIXED can do the work (e.g., van
>> Houwelingen, Arends, & Stijnen, 2002).
>>
>> Sometimes, it is possible to transform the multivariate effect sizes
>> into independent effect sizes (Kalaian & Raudenbush, 1996; Raudenbush,
>> Becker, & Kalaian, 1988). Then univariate meta-analysis, e.g.,
>> metafor(), can be performed on the transformed effect sizes. This
>> approach works if it makes sense to pool the multivariate effect sizes
>> as in your case (2)- the effect sizes are the same but in different
>> conditions (happy, sad, and neutral). However, this approach does not
>> work if the multivariate effect sizes are measuring different
>> concepts, e.g., verbal achievement and mathematical achievement.
>>
>> Hope this helps.
>>
>> Becker, B. J. (1992). Using results from replicated studies to
>> estimate linear models. Journal of Educational Statistics, 17,
>> 341-362.
>> Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect
>> sizes. In H. Cooper, L. V. Hedges, and J. C. Valentine (Eds.), The
>> handbook of research synthesis and meta-analysis, 2nd edition (pp.
>> 357-376). New York: Russell Sage Foundation.
>> Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed
>> linear model for meta-analysis. Psychological Methods, 1, 227-235.
>> Raudenbush, S. W., Becker, B. J., & Kalaian, H. (1988). Modeling
>> multivariate effect sizes. Psychological Bulletin, 103, 111-120.
>> van Houwelingen, H.C., Arends, L.R., & Stijnen, T. (2002). Advanced
>> methods in meta-analysis: multivariate approach and meta-regression.
>> Statistics in Medicine, 21, 589-624.
>>
>> Regards,
>> Mike
>> --
>> ---------------------------------------------------------------------
>>  Mike W.L. Cheung               Phone: (65) 6516-3702
>>  Department of Psychology       Fax:   (65) 6773-1843
>>  National University of Singapore
>>  http://courses.nus.edu.sg/course/psycwlm/internet/
>> ---------------------------------------------------------------------
>>
>> On Sat, Feb 6, 2010 at 6:07 AM, Gang Chen <gangchen6 at gmail.com> wrote:
>>> In a classical meta analysis model y_i = X_i * beta_i + e_i, data
>>> {y_i} are assumed to be independent effect sizes. However, I'm
>>> encountering the following two scenarios:
>>>
>>> (1) Each source has multiple effect sizes, thus {y_i} are not fully
>>> independent with each other.
>>> (2) Each source has multiple effect sizes, and each of the effect size
>>> from a source can be categorized as one of a factor levels (e.g.,
>>> happy, sad, and neutral). Maybe better denote the data as y_ij, effect
>>> size at the j-th level from the i-th source. I can code the levels
>>> with dummy variables into the X_i matrix, but apparently the data from
>>> the same source are correlated with each other. In this case, I would
>>> like to run a few tests one of which is, for example, whether there is
>>> any difference across all the levels of the factor.
>>>
>>> Can metafor handle these two cases?
>>>
>>> Thanks,
>>> Gang
>