[R] T-test to check equality, unable to interpret the results.

[Greg Snow]

It appears that you have a fundamental misunderstanding of what p-values do and do not say (though this misunderstanding is commom).  The following article addresses this issue and could help with a better understanding:

     Murdock, D, Tsai, Y, and Adcock, J (2008) _P-Values are Random
     Variables_. The American Statistician. (62) 242-245.

See also the Pvalue.norm.sim function in the TeachingDemos package for simulation examples demonstrating points from the article.

Hope this helps,


-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at imail.org
801.408.8111

> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
> project.org] On Behalf Of Robert Hall
> Sent: Wednesday, September 16, 2009 12:55 PM
> To: r-help
> Subject: [R] T-test to check equality, unable to interpret the results.
> 
> Hi,
> I have the precision values of a system on two different data sets.
> The snippets of these results are as shown:
> 
> sample1: (total 194 samples)
> 0.6000000238
> 0.8000000119
> 0.6000000238
> 0.2000000030
> 0.6000000238
> ...
> ...
> 
> sample2: (total 188 samples)
> 0.80000001
> 0.20000000
> 0.80000001
> 0.00000000
> 0.80000001
> 0.40000001
> ...
> ...
> 
> I want to check if these results are statistically significant?
> Intuitively,
> the similarity in the two results mean the results are statistically
> significant.
> I am using the t-test t.test(sample1,sample2)to check for similarity
> amongst
> the two results.
> I get the following output:
> 
> -----------------------------------------------
>     Welch Two Sample t-test
> 
> data:  s1p5 and s2p5
> t = 0.9778, df = 374.904, p-value = 0.3288
> alternative hypothesis: true difference in means is not equal to 0
> 95 percent confidence interval:
>  -0.03170059  0.09441172
> sample estimates:
> mean of x mean of y
> 0.5138298 0.4824742
> ------------------------------------------------
> 
> I believe the t-test checks for difference amongst the two sets, and p-
> value
> < 0.05 means both thesets are statistically different. Here while
> checking
> for dissimilarity the p-value is 0.3288, does it mean that higher the
> p-value (while t.test checks for dis-similarity) means more similar the
> results are (which is the case above as the means of the results are
> very
> close!)
> Please help me interpret the results..
> thanks in advance!
> 
> --
> Rob Hall
> Masters Student
> ANU
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.