[R] Fw: Permutations with replacement

Mon Aug 21 19:51:39 CEST 2006

My apologies, I forgot to CC: to the list on my previous communication 
with Daniel.

Jesse

----- Forwarded by Jesse Albert Canchola/EMVL/DIAG/US/BAYER on 08/21/2006 
10:50 AM -----

Jesse Albert Canchola/EMVL/DIAG/US/BAYER
08/21/2006 09:36 AM

To
"Daniel Nordlund" <res90sx5 at verizon.net>
cc

Subject
RE: [R] Permutations with replacement

Thanks, Daniel.  I need to enumerate all possibilities of 8^8 and take a 
random sample of 10,000 from there.  Then I will use the sampled 
possibilities to do a combination of data frames/files then do some math 
on those files and develop the probability distribution from resulting 
sampling statistics (I couldn't get the available bootstrap packages to do 
what I want).  BTW, the preferred solution (however inelegant) is 
reprinted below.   I did have a memory problem with the hypercube for 8^8 
so I did an 8^7 hypercube that worked and concatenated a 1-8 to the 8^7 
matrix that resulted in 8 large matrices which I attempted rbind together 
to create the 8^8 but ran into more memory problems (on matrix number 7 of 
8- it's a Windows problem - I used the --max-mem-size=2G - to no avail). 
The final solution was to take a random sample of 10,000/8=1250 from each 
of the files (doing four of the 8 files at a time, and for which I 
permuted the rows to make it more random), removed the heavy-laden files 
then rbind 'ed the smaller sampled files together to make the 10,000.

Here is the final final code:

# IDEA:  We cannot simply do a permutation of the 8 classes/file id's 
since this will not allow/simulate repeats of numbers
#        as in a bootstrap (e.g., for 3 items - 1,2,3 - we would also want 
the possibility 1,1,1 or 2,2,2 or 3,3,3 etc.
#        The 8!=40,320 permutations with replacement would become an 
8^8=16,777,216 so we would want to take a random
#        sample of 10,000 from the posibilities 
library(combinat)
# THIS WILL NOT WORK DUE TO THE LIMITATIONS OF WINDOWS MEMORY (PROBLEMATIC 
AS IN THE FAQs) SO WE WILL 
# USE A WORKAROUND
# WORKAROUND: 1) Construct the 8 to the 7 power hypercube.  2) For these 
data, create eight additional data sets that include
#             the last position to finish the construction of an 8^8 
hypercube
#x <- rep(8,8) # for partitions of 8 units into classes {1,2,3,4,5,6,7,8} 
#hcube8 <- hcube(x, scale=1, transl=0) 
#hcube8

#step 1: 8^7 = 2,097,152
x1 <- rep(8,7)
x1
hcube87 <- hcube(x1, scale=1, transl=0)
#this will generate 2,097,152 results from 1-8 but for only 7 positions

#step 2: column bind each file with 1-8 in the 8th position 
#x1a <- cbind(hcube87,1)
#x2a <- cbind(hcube87,2)
#x3a <- cbind(hcube87,3)
#x4a <- cbind(hcube87,4)
#x5a <- cbind(hcube87,5)
#x6a <- cbind(hcube87,6)
#x7a <- cbind(hcube87,7)
#x8a <- cbind(hcube87,8)

#turns out this method also chokes with the memory limitations
# Step 2 will be modified as follows: 
# Step 2a: as before; Step2b: Sample 1/8 from each piece and after every 4 
processes, delete the objects to allow for the rest.

#Step 2a
x1a <- cbind(hcube87,1)
x1a <- x1a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x2a <- cbind(hcube87,2)
x2a <- x2a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x3a <- cbind(hcube87,3)
x3a <- x3a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x4a <- cbind(hcube87,4)
x4a <- x4a[sample(1:2097152),] #randomly permute the rows for more 
randomness

x1b <- x1a[sample(1:1250,replace=FALSE),]
x2b <- x2a[sample(1:1250,replace=FALSE),]
x3b <- x3a[sample(1:1250,replace=FALSE),]
x4b <- x4a[sample(1:1250,replace=FALSE),]

rm(x1a,x2a,x3a,x4a) #remove the big files

x5a <- cbind(hcube87,5)
x5a <- x5a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x6a <- cbind(hcube87,6)
x6a <- x6a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x7a <- cbind(hcube87,7)
x7a <- x7a[sample(1:2097152),] #randomly permute the rows for more 
randomness
x8a <- cbind(hcube87,8)
x8a <- x8a[sample(1:2097152),] #randomly permute the rows for more 
randomness

x5b <- x5a[sample(1:1250,replace=FALSE),]
x6b <- x6a[sample(1:1250,replace=FALSE),]
x7b <- x7a[sample(1:1250,replace=FALSE),]
x8b <- x8a[sample(1:1250,replace=FALSE),]

rm(x5a,x6a,x7a,x8a) #remove the big files

#Step 3: combine all the randomly sampled files
m <- rbind(x1b,x2b,x3b,x4b,x5b,x6b,x7b,x8b)

# NOTE:  each number in the matrix represents a file "name" from 1-8. 
# the first pointer should be numeric and then subsequent as character
# since the first time you assign a number to a character in a matrix
# the rest of the numbers in the matrix are coerced to character
m[m==1]='a'; m[m=='2']='b'; m[m=='3']='c' ; m[m=='4']='d'; m[m=='5']='e' ; 
m[m=='6']='f'; m[m=='7']='g' ; m[m=='8']='h'
m
########### end R code ############

Thanks, David.  That worked fabulously! 

Here is the R code for the hypercube test example: 

########## begin R code ############
library(combinat)
x <- rep(3,3)   # for partitions of 3 units into the three classes {1,2,3} 

hcube(x, scale=1, transl=0) 
########### end R code ############

For the larger one I want (i.e., 8^8), I will take a random sample of 
10,000 from the 16,777,216 possibilities.

Regards,
Jesse Canchola

<davidr at rhotrading.com> 
Sent by: r-help-bounces at stat.math.ethz.ch
08/18/2006 01:33 PM

To
"Jesse Albert Canchola" <jesse.canchola.b at bayer.com>, "r-help" 
<r-help at stat.math.ethz.ch>
cc

Subject
Re: [R] Permutations with replacement

If you also want 1,1,1 and so on, the number of these is n^n,
(n choices for each of n slots.)
In that case, you could use hcube from combinat.

David L. Reiner
Rho Trading Securities, LLC
Chicago  IL  60605

"Daniel Nordlund" <res90sx5 at verizon.net> 
08/18/2006 05:16 PM

To
"'Jesse Albert Canchola'" <jesse.canchola.b at bayer.com>, "'r-help'" 
<r-help at stat.math.ethz.ch>
cc

Subject
RE: [R] Permutations with replacement

> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
[mailto:r-help-bounces at stat.math.ethz.ch]
> On Behalf Of Jesse Albert Canchola
> Sent: Friday, August 18, 2006 1:02 PM
> To: r-help
> Subject: [R] Permutations with replacement
> 
> Is there a simple function or process that will create permutations with
> replacement?
> 
> I know that using the combinat package
> 
> ###### begin R code ######
> > library(combinat)
> > m <- t(array(unlist(permn(3)), dim = c(3, 6)))
> 
> # we can get the permutations, for example 3!=6
> # gives us
> 
> > m
>      [,1] [,2] [,3]
> [1,]    1    2    3
> [2,]    1    3    2
> [3,]    3    1    2
> [4,]    3    2    1
> [5,]    2    3    1
> [6,]    2    1    3
> ###### end R code ##########
> 
> I'd like to include the "with replacement possibilities" such as
> 
> 1,1,3
> 1,1,2
> 2,3,3
> 
Isn't what you want just sampling with replacement?

  x <- c(1,2,3)
  sample(x,3,replace=TRUE)

Hope this is helpful,

Dan

Dan Nordlund
Bothell, WA  USA

_______________________________________________________________________________________________

The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged.  Inadvertent disclosure of this message does not constitute a waiver of any privilege.  If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message.  Please also delete this e-mail and all copies and notify the sender.  Thank you.

For alternate languages please go to http://bayerdisclaimer.bayerweb.com