I have a large file backed big. matrix, with millions of rows and 20
columns.

The columns contain data that I simply need to tabulate. There are a few
dozen unique
values. and I just want a frequency count

Test code with a small "big" matrix.

library(bigmemory)
library(bigtabulate)

  test <- big.matrix(nrow = 100, ncol = 10)
  test[,1:3]<- sample(150)
  test[,4:6]<- sample(100)
  test[,7:10]<- sample(100)

##    so we have  sample big memory matrix. Its not file backed but will do
for testing.
##    the result we want is one that you would get if you could run table()
 on the bigmatrix
##    thats emulated in this example by coercing the bigmatrix to a matrix.
##    in the real application that is not possible, because of RAM limits
  P <- table(as.matrix(test))

##  the package big tabulate has a version of table called  bigtable.
##  you can run table on an individual column.
## I want to run it on all the columns. basically combine the results of
running it on individual columns
## if you try to specify multiple columns, you get a contingency table, and
if you use too many
## columns you will hang your system hard .. so dont try the line below .
Well at least I hung my system

#  Ouch <- bigtable(test, ccols = seq(1,10))

So, is there a simple way to  get the answer as emulated by
 P<-table(as.matrix(test))
without coercing to a matrix.

TIA

	[[alternative HTML version deleted]]

