[R] hclust() memory issue

[Ochsner, Scott A]

John,

First, why are you trying to cluster so many rows?  Presumably, if this is a gene expression array dataset, most of the array features are not going to change across treatments/conditions and will be relatively uninformative.  Try using a filter which does not use treatment/condition information to decrease the number or array features you are attempting to cluster.  There are numerous examples in the affycoretools and genefilter packages from Bioconductor http://www.bioconductor.org/.

HTH,

Scott  


Scott A. Ochsner, PhD
One Baylor Plaza BCM130, Houston, TX 77030
Voice: (713) 798-6227  Fax: (713) 790-1275 
-----Original Message-----
From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf Of array chip
Sent: Monday, March 14, 2011 4:03 PM
To: r-help at r-project.org
Subject: [R] hclust() memory issue

Hi, I have a microarray dataset of dimension 25000x30 and try to clustering 
using hclust(). But the clustering on the rows failed due to the size:

> y<-hclust(dist(data),method='average')
Error: cannot allocate vector of size 1.9 Gb

I tried to increase the memory using memory.limit(size=3000), still got the same 
error.

I also tried agnes() from cluster package and pvclust() from pvclust package 
without success.

My computer has 2G memory. Is there a more memory efficient clustering packages 
available?

Thanks

John


> sessionInfo()
R version 2.11.1 (2010-05-31) 
i386-pc-mingw32 

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United 
States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    


attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] pvclust_1.2-1     cluster_1.13.1    rat2302cdf_2.6.0  simpleaffy_2.24.0 
gcrma_2.20.0      genefilter_1.30.0 affy_1.26.1      

[8] Biobase_2.8.0    

loaded via a namespace (and not attached):
 [1] affyio_1.16.0         annotate_1.26.1       AnnotationDbi_1.10.2  
Biostrings_2.16.9     DBI_0.2-5             IRanges_1.6.16       

 [7] preprocessCore_1.10.0 RSQLite_0.9-2         splines_2.11.1        
survival_2.35-8       tools_2.11.1          xtable_1.5-6 


      
	[[alternative HTML version deleted]]

______________________________________________
R-help at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.