Hi, Charlie,

Thank you for the reply. Maybe I don't need the frequency of each pair. I
only need the top, say 50, pairs with the highest frequency. Is there anyway
which can avoid calculating for all the pairs?

Thanks,

Cindy
On Sun, Nov 15, 2009 at 4:18 PM, cls59 <chuck@sharpsteen.net> wrote:

>
>
>
> cindy Guo wrote:
> >
> > Hi, All,
> >
> > I have an n by m matrix with each entry between 1 and 15000. I want to
> > know
> > the frequency of each pair in 1:15000 that occur together in rows. So for
> > example, if the matrix is
> > 2 5 1 6
> > 1 7 8 2
> > 3 7 6 2
> > 9 8 5 7
> > Pair (2,6) (un-ordered) occurs together in rows 1 and 3. I want to return
> > the value 2 for this pair as well as that for all pairs. Is there a fast
> > way
> > to do this avoiding loops? Loops take too long.
> >
> > Thank you,
> >
> > Cindy
> >
>
> Use %in% to check for the presence of the numbers in a row and apply() to
> efficiently execute the test for each row:
>
>  tstMatrix <- matrix( c(2,5,1,6,
>    1,7,8,2,
>    3,7,6,2,
>    9,8,5,7), nrow=4, byrow=T )
>
>  matches <- apply( tstMatrix, 1, function( row ){
>
>    if( 2 %in% row & 6 %in% row ){
>
>      return( 2 )
>
>    } else {
>
>      return( 0 )
>
>    }
>
>  })
>
>  matches
>  [1] 2 0 2 0
>
> If you have more than one pair, it gets a little tricky.  Say you are also
> looking for the pair (7,8).  Store them as a list:
>
>  pairList <- list( c(2,6), c(7,8) )
>
> Then use sapply() to efficiently iterate over the pair list and execute the
> apply() test:
>
>  matchMatrix <- sapply( pairList, function( pair ){
>
>    matches <- apply( tstMatrix, 1, function( row ){
>
>      if( pair[1] %in% row & pair[2] %in% row ){
>
>        return( pair[1] )
>
>      } else {
>
>        return( 0 )
>
>      }
>
>    })
>
>    return( matches )
>
>  })
>
>  matchMatrix
>
>       [,1] [,2]
>  [1,]    2    0
>  [2,]    0    7
>  [3,]    2    0
>  [4,]    0    7
>
>
>
> If you're looking to apply the above method to every possible permutation
> of
> 2 numbers that may be generated from the range of numbers 1:15000... that's
> 225,000,000 pairs. expand.grid() can generate the required pair list-- but
> that step alone causes a memory allocation of ~6 GB on my machine.
>
> If you don't have a pile of CPU cores and RAM at your disposal, you can
> probably:
>
>  1. Restrict the upper end of your range to the maximal entry present in
> your matrix since all other combinations have zero occurrences.
>
>  2. Break the list of pairs up into several sublists, run the tests, and
> aggregate the results.
>
> Either way, the analysis will take some time despite the efficiencies of
> the
> apply family of functions due to the shear size of the problem.  If you
> have
> more than one CPU, I would recommend taking a look at parallelized apply
> functions, perhaps using a package like snowfall,  as the testing of the
> pairs is an "embarrassingly parallel" problem.
>
> Hopefully I'm misunderstanding the scope of your problem.
>
>
> Good luck!
>
> -Charlie
>
> -----
> Charlie Sharpsteen
> Undergraduate
> Environmental Resources Engineering
> Humboldt State University
> --
> View this message in context:
> http://old.nabble.com/pairs-tp26364801p26365206.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>

	[[alternative HTML version deleted]]


