[R] Genstat into R - Randomisation test

Robert A LaBudde ral at lcfltd.com
Thu Apr 9 17:37:51 CEST 2009


At 04:43 AM 4/9/2009, Tom Backer Johnsen wrote:
>Peter Dalgaard wrote:
> > Mike Lawrence wrote:
> >> Looks like that code implements a non-exhaustive variant of the
> >> randomization test, sometimes called a permutation test.
> >
> > Isn't it the other way around? (Permutation tests can be 
> exhaustive by looking at all permutations, if a randomization test 
> did that, then it wouldn't be random.)
>
>Eugene Edgington wrote an early book (1980) on this subject called 
>"Randomization tests", published by Dekker.  As far as I remember, 
>he differentiates between "Systematic permutation" tests where one 
>looks at all possible permutations.  This is of course prohibitive 
>for anything beyond trivially small samples.  For larger samples he 
>uses what he calls "Random permutations", where a random sample of 
>the possible permutations is used.
>
>Tom
>
>Peter Dalgaard wrote:
>>Mike Lawrence wrote:
>>>Looks like that code implements a non-exhaustive variant of the
>>>randomization test, sometimes called a permutation test.
>>Isn't it the other way around? (Permutation tests can be exhaustive 
>>by looking at all permutations, if a randomization test did that, 
>>then it wouldn't be random.)

Edginton and Onghena make a similar distinction in their book, but I 
think such a distinction is without merit.

Do we distinguish between "exact" definite integrals and 
"approximate" ones obtained by numerical integration, of which Monte 
Carlo sampling is just one class of algorithms? Don't we just say: 
"The integral was evaluated numerically by the [whatever] method to 
an accuracy of [whatever], and the value was found to be [whatever]." 
Ditto for optimization problems.

A randomization test has one correct answer based upon theory. We are 
simply trying to calculate that answer's value when it is difficult 
to do so. Any approximate method that is used must be performed such 
that the error of approximation is trivial with respect to the 
contemplated use.

Doing Monte Carlo sampling to find an approximate answer to a 
randomization test, or to an optimization problem, or to a bootstrap 
distribution should be carried out with enough realizations to make 
sure the residual error is trivial.

As Monte Carlo sampling is a "random" sampling-based approximate 
method. The name does create confusion in terminology for 
"randomization" tests for bootstrapping.

================================================================
Robert A. LaBudde, PhD, PAS, Dpl. ACAFS  e-mail: ral at lcfltd.com
Least Cost Formulations, Ltd.            URL: http://lcfltd.com/
824 Timberlake Drive                     Tel: 757-467-0954
Virginia Beach, VA 23464-3239            Fax: 757-467-2947

"Vere scire est per causas scire"




More information about the R-help mailing list