[R] Using very large matrix
tony.breyal at googlemail.com
Wed Feb 25 12:58:11 CET 2009
looks like you've run out of memory mate, because that sure is a big
matrix, you'd probably need 64 bit OS/CPU/R and loads of RAM.
see thread: http://www.nabble.com/Error-in-matrix:--too-many-elements-specified-td20457910.html
I know there are some packages on cran which help with large datasets,
but i haven't got around to using them myself (yet).
Large memory and out-of-memory data
* The bigmemory package by Kane and Emerson permits storing large
objects such as matrices in memory and uses external pointer objects
to refer to them. This permits transparent access from R without
bumping against R's internal memory limits. Several R processes on the
same computer can also shared big memory objects.
* A large number of database packages, and database-alike packages
(such as sqldf by Grothendieck) are also of potential interest but not
(yet?) reviewed here.
Just for fun, i just tried (for the first time):
> big.matrix(nrow=80000, ncol=80000, type='double')
but i ended up crashing out of R on my low spec windows XP uni
hope that helps a little,
On 25 Feb, 11:04, Corrado <ct... at york.ac.uk> wrote:
> Dear friends,
> I have to use a very large matrix. Something of the sort of
> matrix(80000,80000,n) .... where n is something numeric of the sort 0.xxxxxx
> I have not found a way of doing it. I keep getting the error
> Error in matrix(nrow = 80000, ncol = 80000, 0.2) : too many elements specified
> Any suggestions? I have searched the mailing list, but to no avail.
> Corrado Topi
> Global Climate Change & Biodiversity Indicators
> Area 18,Department of Biology
> University of York, York, YO10 5YW, UK
> Phone: + 44 (0) 1904 328645, E-mail: ct... at york.ac.uk
> R-h... at r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
More information about the R-help