[R-meta] (partial) eta squared

[Michael Dewey]

Dear Antonia

Just a small piece of advice - if you call the variance "error" then 
when you re-read your code in a few months you may get confused and 
think it was the standard error (even though you put in a comment).

Michael

On 07/08/2018 14:45, Antonia Sudkaemper wrote:
> Hello Wolfgang,
> 
> thank you very much for your advice and your reading suggestions (will take
> me a while to catch up on those...) - based on your instructions I am using
> this code now to run the (mini) meta-analysis.
> 
> slope = c(-0.0478, 0.1307, 0.0941)               #regression coefficient
> error = c(0.0075, 0.0083, 0.0077)                  #standard error squared
> study<-c("Study 1", "Study 2", "Study 3")
> 
> summary(meta <- rma(yi=slope, vi= error, method= "FE", slab=c("Study 1",
> "Study 2", "Study 3")))
> forest(meta, xlab = "regression coefficient")
> 
> All the best, Antonia
> 
> 
> On 7 August 2018 at 10:31, Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
> 
>> I would then meta-analyze the non-standardized regression coefficients.
>> The square of the SEs of the coefficients are the appropriate sampling
>> variances. Computing the correct SEs (and hence sampling variances) for
>> standardized regression coefficients is a bit more tricky. See, for example:
>>
>> Jones, J. A., & Waller, N. G. (2015). The normal-theory and asymptotic
>> distribution-free (adf) covariance matrix of standardized regression
>> coefficients: Theoretical extensions and finite sample behavior.
>> Psychometrika, 80(2), 365-378.
>>
>> Jones, J. A., & Waller, N. G. (2013). Computing confidence intervals for
>> standardized regression coefficients. Psychological Methods, 18(4), 435-453.
>>
>> Yuan, K.-H., & Chan, W. (2011). Biases and standard errors of standardized
>> regression coefficients. Psychometrika, 76(4), 670-690.
>>
>> Alternatively, you could meta-analyze the (semi)partial correlation
>> coefficients. See:
>>
>> Aloe, A. M., & Becker, B. J. (2012). An effect size for regression
>> predictors in meta-analysis. Journal of Educational and Behavioral
>> Statistics, 37(2), 278-297.
>>
>> Aloe, A. M., & Thompson, C. G. (2013). The synthesis of partial effect
>> sizes. Journal of the Society for Social Work and Research, 4(4), 390-405.
>>
>> Aloe, A. M. (2014). An empirical investigation of partial effect sizes in
>> meta-analysis of correlational data. Journal of General Psychology, 141(1),
>> 47-64.
>>
>> See also help(escalc) and then search for "Partial and Semi-Partial
>> Correlations".
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: Antonia Sudkaemper [mailto:a.sudkaemper using gmail.com]
>> Sent: Monday, 06 August, 2018 16:53
>> To: Viechtbauer, Wolfgang (SP)
>> Cc: r-sig-meta-analysis using r-project.org
>> Subject: Re: [R-meta] (partial) eta squared
>>
>> Hello Wolfgang,
>>
>> yes, I noticed that this is a difficult one...
>>
>> Yes, the studies are (almost) equivalent, using the same DV's and IV's.
>> Would you then recommend meta-analyzing the standardized or
>> non-standardized regression coefficients?
>>
>> Thank you so much for your help!
>>
>> All the best, Antonia
>>
>> On 6 August 2018 at 14:47, Viechtbauer, Wolfgang (SP) <
>> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> It's not so much an issue with metafor, but with lack of methodology for
>> this in general.
>>
>> Did all three studies use the same DV and IVs? Then you could meta-analyze
>> the regression coefficient for the interaction directly.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: Antonia Sudkaemper [mailto:a.sudkaemper using gmail.com]
>> Sent: Monday, 06 August, 2018 15:41
>> To: Viechtbauer, Wolfgang (SP)
>> Cc: r-sig-meta-analysis using r-project.org
>> Subject: Re: [R-meta] (partial) eta squared
>>
>> Hello Wolfgang,
>>
>> thank you very much for your reply. It seems like eta squared might not be
>> the most straightforward option then when intending to work with metafor.
>>
>> Would you recommend another effect size for an interaction effect from a
>> linear regression that is more compatible with the analyses metafor runs?
>>
>> All the best, Antonia
>>
>> On 3 August 2018 at 15:09, Viechtbauer, Wolfgang (SP) <
>> wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
>> Dear Antonia,
>>
>> in principle, you could meta-analyze eta^2 values, but there are several
>> issues to consider:
>>
>> 1) The sampling distribution of eta^2 isn't normal. So, one would first
>> have to explore what kind of transformation would be appropriate for eta^2
>> values to normalize their sampling distribution.
>>
>> 2) I do not know off the top of my head an equation for the sampling
>> variance of eta^2 values.
>>
>> 3) eta^2 isn't a directional effect size measure. Two eta^2 values of the
>> same magnitude could imply entirely opposite findings. So, one could
>> question the usefulness of aggregating eta^2 values in the first place.
>>
>> Best,
>> Wolfgang
>>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-
>> bounces using r-project.org] On Behalf Of Antonia Sudkaemper
>> Sent: Friday, 03 August, 2018 12:33
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] (partial) eta squared
>>
>> Hello fellow meta-analysis colleagues,
>>
>> I have recently started using metafor and am still exploring. Currently, I
>> am
>> working on a mini meta-analysis of three studies I ran myself. In recent
>> psychology journal I have seen the use of (partial) eta squared as an
>> indicator of effect size. I was wondering if I can use the rma.uni command
>> to run a meta-analysis on (partial) eta squared? And if so, which error
>> (vi/sei) indicator would I use with it?
>>
>> Hope someone can help!
>>
>> All the best, Antonia
>>
>> --
>> Antonia Sudkämper
>> PhD Candidate in Organizational Psychology/University of Exeter
>> www.antoniasudkaemper.com
>> a.sudkaemper using gmail.com
>>
> 
> 
> 

-- 
Michael
http://www.dewey.myzen.co.uk/home.html