[R] Difference betweeen cor.test() and formula everyone says to use

[peter dalgaard]

This is pretty much standard. I'm quite sure that other stats packages do likewise and I wouldn't know who "everyone" is. It is not unheard of that textbook authors give suboptimal formulas in order not to confuse students, though.

The basic point is that the t transformation gives the exact distribution under the null. Fisher's Z is only approximately normally distributed. 

The t transformation works because if beta is the regression coefficient of y on x, beta==0 iff rho==0, and we have exact theory for testing beta==0 by a t-test.

Off-null, the t-approach does not readily transfer, so confidence intervals tend to be based on the Z-transformation.

-Peter D.



On 17 Oct 2014, at 02:20 , Joshua Wiley <jwiley.psych at gmail.com> wrote:

> Hi Jeremy,
> 
> I don't know about references, but this around.  See for example:
> http://afni.nimh.nih.gov/sscc/gangc/tr.html
> 
> the relevant line in cor.test is:
> 
> STATISTIC <- c(t = sqrt(df) * r/sqrt(1 - r^2))
> 
> You can convert *t*s to *r*s and vice versa.
> 
> Best,
> 
> Josh
> 
> 
> 
> On Fri, Oct 17, 2014 at 10:32 AM, Jeremy Miles <jeremy.miles at gmail.com>
> wrote:
> 
>> I'm trying to understand how cor.test() is calculating the p-value of
>> a correlation. It gives a p-value based on t, but every text I've ever
>> seen gives the calculation based on z.
>> 
>> For example:
>>> data(cars)
>>> with(cars[1:10, ], cor.test(speed, dist))
>> 
>> Pearson's product-moment correlation
>> 
>> data:  speed and dist
>> t = 2.3893, df = 8, p-value = 0.04391
>> alternative hypothesis: true correlation is not equal to 0
>> 95 percent confidence interval:
>> 0.02641348 0.90658582
>> sample estimates:
>>      cor
>> 0.6453079
>> 
>> But when I use the regular formula:
>>> r <- cor(cars[1:10, ])[1, 2]
>>> r.z <- fisherz(r)
>>> se <- se <- 1/sqrt(10 - 3)
>>> z <- r.z / se
>>> (1 - pnorm(z))*2
>> [1] 0.04237039
>> 
>> My p-value is different.  The help file for cor.test doesn't (seem to)
>> have any reference to this, and I can see in the source code that it
>> is doing something different. I'm just not sure what.
>> 
>> Thanks,
>> 
>> Jeremy
>> 
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>> 
> 
> 
> 
> -- 
> Joshua F. Wiley
> Ph.D. Student, UCLA Department of Psychology
> http://joshuawiley.com/
> Senior Analyst, Elkhart Group Ltd.
> http://elkhartgroup.com
> Office: 260.673.5518
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com