[R] pairs

[cls59]

cindy Guo wrote:
> 
> Hi, All,
> 
> I have an n by m matrix with each entry between 1 and 15000. I want to
> know
> the frequency of each pair in 1:15000 that occur together in rows. So for
> example, if the matrix is
> 2 5 1 6
> 1 7 8 2
> 3 7 6 2
> 9 8 5 7
> Pair (2,6) (un-ordered) occurs together in rows 1 and 3. I want to return
> the value 2 for this pair as well as that for all pairs. Is there a fast
> way
> to do this avoiding loops? Loops take too long.
> 
> Thank you,
> 
> Cindy
> 

Use %in% to check for the presence of the numbers in a row and apply() to
efficiently execute the test for each row:

 tstMatrix <- matrix( c(2,5,1,6,
    1,7,8,2,
    3,7,6,2,
    9,8,5,7), nrow=4, byrow=T )

  matches <- apply( tstMatrix, 1, function( row ){
   
    if( 2 %in% row & 6 %in% row ){

      return( 2 )

    } else {

      return( 0 )

    }

  })

  matches
  [1] 2 0 2 0

If you have more than one pair, it gets a little tricky.  Say you are also
looking for the pair (7,8).  Store them as a list:

  pairList <- list( c(2,6), c(7,8) )

Then use sapply() to efficiently iterate over the pair list and execute the
apply() test:

  matchMatrix <- sapply( pairList, function( pair ){

    matches <- apply( tstMatrix, 1, function( row ){
    
      if( pair[1] %in% row & pair[2] %in% row ){

        return( pair[1] )

      } else {

        return( 0 )

      }

    })

    return( matches )

  })

  matchMatrix

       [,1] [,2]
  [1,]    2    0
  [2,]    0    7
  [3,]    2    0
  [4,]    0    7



If you're looking to apply the above method to every possible permutation of
2 numbers that may be generated from the range of numbers 1:15000... that's
225,000,000 pairs. expand.grid() can generate the required pair list-- but
that step alone causes a memory allocation of ~6 GB on my machine.

If you don't have a pile of CPU cores and RAM at your disposal, you can
probably:

  1. Restrict the upper end of your range to the maximal entry present in
your matrix since all other combinations have zero occurrences.

  2. Break the list of pairs up into several sublists, run the tests, and
aggregate the results.

Either way, the analysis will take some time despite the efficiencies of the
apply family of functions due to the shear size of the problem.  If you have
more than one CPU, I would recommend taking a look at parallelized apply
functions, perhaps using a package like snowfall,  as the testing of the
pairs is an "embarrassingly parallel" problem.

Hopefully I'm misunderstanding the scope of your problem.


Good luck!

-Charlie

-----
Charlie Sharpsteen
Undergraduate
Environmental Resources Engineering
Humboldt State University
-- 
View this message in context: http://old.nabble.com/pairs-tp26364801p26365206.html
Sent from the R help mailing list archive at Nabble.com.