[R] SVD Memory Issue

[vioravis]

I am trying to perform Singular Value Decomposition (SVD) on a Term Document
Matrix  I created using the 'tm' package. Eventually I want to do a Latent
Semantic Analysis (LSA).

There are 5677 documents with 771 terms (the DTM is 771 x 5677). When I try
to do the SVD, it runs out of memory. I am using a 12GB Dual core Machine
with Windows XP and don't think I can increase the memory anymore. Are there
any other memory efficient methods to find the SVD?

The term document is obtained using:

tdm2 <-
TermDocumentMatrix(tr1,control=list(weighting=weightTf,minWordLength=3))
str(tdm2)

List of 6
 $ i       : int [1:6438] 202 729 737 278 402 621 654 718 157 380 ...
 $ j       : int [1:6438] 1 2 3 7 7 7 7 8 10 10 ...
 $ v       : num [1:6438] 8 5 6 9 5 7 5 6 5 7 ...
 $ nrow    : int 771
 $ ncol    : int 5677
 $ dimnames:List of 2
  ..$ Terms: chr [1:771] "access" "accessori" "accumul" "acoust" ...
  ..$ Docs : chr [1:5677] "1" "2" "3" "4" ...
 - attr(*, "class")= chr [1:2] "TermDocumentMatrix" "simple_triplet_matrix"
 - attr(*, "Weighting")= chr [1:2] "term frequency" "tf"

SVD is calcualted using:

> tdm_matrix <- as.matrix(tdm2)
> svd_out<-svd(tdm_matrix)

Error: cannot allocate vector of size 767.7 Mb
In addition: Warning messages:
1: In matrix(0, n, np) :
  Reached total allocation of 3583Mb: see help(memory.size)
2: In matrix(0, n, np) :
  Reached total allocation of 3583Mb: see help(memory.size)
3: In matrix(0, n, np) :
  Reached total allocation of 3583Mb: see help(memory.size)
4: In matrix(0, n, np) :
  Reached total allocation of 3583Mb: see help(memory.size)


Thank you.

Ravi



--
View this message in context: http://r.789695.n4.nabble.com/SVD-Memory-Issue-tp3809667p3809667.html
Sent from the R help mailing list archive at Nabble.com.