[R] p-values
Peter Ho
peter at estg.ipvc.pt
Thu Aug 11 15:28:00 CEST 2005
Spencer,
Here is an example from rayner and best 2001 and the script sent by
Felipe. This can be done as follows using the function durbin.grupos()
in the attached file
> ###Ice cream example from Rayner and Best 2001 . Chapter 7
> judge <- rep(c(1:7),rep(3,7))
> variety <- c(1,2,4,2,3,5,3,4,6,4,5,7,1,5,6,2,6,7,1,3,7)
> cream <- c(2,3,1,3,1,2,2,1,3,1,2,3,3,1,2,3,1,2,3,1,2)
> durbin.grupos(judge,variety,cream,k=3,r=3,alpha=0.01)
Prueba de Durbin
..............
Chi Cuadrado : 12
Gl. : 6
P-valor : 0.0619688
..............
Comparación de tratamientos
Alpha : 0.01
Gl. : 8
t-Student : 3.355387
Diferencia minima
para la diferencia entre suma de rangos = 4.109493
Grupos, Tratamientos y la Suma de sus rangos
a 2 9
ab 1 8
abc 7 7
abc 6 6
abc 5 5
bc 3 4
c 4 3
trat prom M
1 2 9 a
2 1 8 ab
3 7 7 abc
4 6 6 abc
5 5 5 abc
6 3 4 bc
7 4 3 c
>
You can see that the p-value is the same with
> pchisq(12, df= 6, lower.tail=F)
[1] 0.0619688
I am hoping that someone, maybe Torsten, might be able to suggest how I
can incorporate Monte-Carlo p-values using pperm(). The statistical
issues are beyond my comprehension and I assume that Rayner and Best
suggestion to use Monte-Carlo p-values instead of Chi-square p-values to
be correct. In the above example the Monte-Carlo p-value is 0.02. This
is a significant difference, resulting in the rejection of the null
hypothesis when using Monte-Carlo p-values.
I hope this example might help. Thanks again for your answer and also to
Felipe for sending the function for Durbin's test.
Peter
Spencer Graves wrote:
> pperm seems reasonable, though I have not looked at the details.
>
> We should be careful about terminology, however. So-called
> "exact p-values" are generally p-values computed assuming a
> distribution over a finite set of possible outcomes assuming some
> constraints to make the outcome space finite. For example, Fisher's
> exact test for a 2x2 table assumes the marginals are fixed.
>
> I don't remember the details now, but I believe there is
> literature claiming that this may not be the best thing to do when,
> for example, when it is reasonable to assume that the number in each
> cell is Poisson. In such cases, you may lose statistical power by
> conditioning on the marginals. I hope someone else will enlighten us
> both, because I'm not current on the literature in this area.
>
> The situation with "exact tests" and "exact p-values" is not
> nearly as bad as with so-called ""exact confidence intervals", which
> promise to deliver at least the indicated coverage probability. With
> discrete distributions, it is known that 'Approximate is better than
> "exact' for interval estimation of binomial proportions', as noted in
> an article of this title by A. Agresti and B. A. Coull (1998) American
> Statistician, 52: 119-126. (For more on this particular issue, see
> Brown, Cai and Dasgupta 2003 "Interval Estimation in Exponential
> Families", Statistica Sinica 13: 19-49).
>
> If this does not answer your question adequately, may I suggest
> you try the posting guide. People report having found answers to
> difficult questions in the process of preparing a question following
> that guide, and when they do post a question, they are much more
> likely to get a useful reply.
>
> spencer graves
>
> Peter Ho wrote:
>
>> Spencer,
>>
>>
>> Thank you for referring me to your other email on Exact
>> goodness-of-fit test. However, I'm not entirely sure if what you
>> mentioned is the same for my case. I'm not a statistician and it
>> would help me if you could explain what you meant in a little more
>> detail. Perhaps I need to explain the problem in more detail.
>>
>> I am looking for a way to calculate exaxt p-values by Monte Carlo
>> Simulation for Durbin's test. Durbin's test statistic is similar to
>> Friedman's statistic, but considers the case of Balanced Incomplete
>> block designs. I have found a function written by Felipe de Mendiburu
>> for calculating Durbin's statistic, which gives the chi-squared
>> p-value. I have also been read an article by Torsten Hothorn "On
>> exact rank Tests in R" (R News 1(1), 11–12.) and he has shown how to
>> calculate Monte Carlo p-values using pperm. In the article by Torsten
>> Hothorn he gives:
>>
>> R> pperm(W, ranks, length(x))
>>
>> He compares his method to that of StatXact, which is the program
>> Rayner and Best suggested using. Is there a way to do this for
>> example for the friedman test.
>>
>> A paper by Joachim Rohmel discusses "The permutation distribution for
>> the friendman test" (Computational Statistics & Data Analysis 1997,
>> 26: 83-99). This seems to be on the lines of what I need, although I
>> am not quite sure. Has anyone tried to recode his APL program for R?
>>
>> I have tried a number of things, all unsucessful. Searching through
>> previous postings have not been very successful either. It seems that
>> pperm is the way to go, but I would need help from someone on this.
>>
>> Any hints on how to continue would be much appreciated.
>>
>>
>> Peter
>>
>>
>> Spencer Graves wrote:
>>
>>> Hi, Peter:
>>>
>>> Please see my reply of a few minutes ago subject: exact
>>> goodness-of-fit test. I don't know Rayner and Best, but the same
>>> method, I think, should apply. spencer graves
>>>
>>> Peter Ho wrote:
>>>
>>>
>>>
>>>> HI R-users,
>>>>
>>>> I am trying to repeat an example from Rayner and Best "A
>>>> contingency table approach to nonparametric testing (Chapter 7, Ice
>>>> cream example).
>>>>
>>>> In their book they calculate Durbin's statistic, D1, a dispersion
>>>> statistics, D2, and a residual. P-values for each statistic is
>>>> calculated from a chi-square distribution and also Monte Carlo
>>>> p-values.
>>>>
>>>> I have found similar p-values based on the chi-square distribution
>>>> by using:
>>>>
>>>> > pchisq(12, df= 6, lower.tail=F)
>>>> [1] 0.0619688
>>>> > pchisq(5.1, df= 6, lower.tail=F)
>>>> [1] 0.5310529
>>>>
>>>> Is there a way to calculate the equivalent Monte Carlo p-values?
>>>>
>>>> The values were 0.02 and 0.138 respectively.
>>>>
>>>> The use of the approximate chi-square probabilities for Durbin's
>>>> test are considered not good enough according to Van der Laan (The
>>>> American Statistician 1988,42,165-166).
>>>>
>>>>
>>>> Peter
>>>> --------------------------------
>>>> ESTG-IPVC
>>>>
>>>> ______________________________________________
>>>> R-help at stat.math.ethz.ch mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide!
>>>> http://www.R-project.org/posting-guide.html
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Durbin.r
Url: https://stat.ethz.ch/pipermail/r-help/attachments/20050811/17206e5f/Durbin.pl
More information about the R-help
mailing list