[R] Running out of memory when using lapply
sfalcon at fhcrc.org
Fri Aug 11 16:29:09 CEST 2006
"Kamila Naxerova" <kamila.naxerova at childrens.harvard.edu> writes:
> Hi all!
> I'm afraid I programmed something totally non-sensical and inefficient,
> but I can't figure out how to do it better.
> I have a list of ~ 40 000 characters. I want to take each element at a
> time, map it to a large data frame with
> hit=which(data.frame$column==elementFromList), then compute some
> statistic on data.frame[hit,] and return a result that consists of
> either 1) a list of integers or 2) a character.
> On a small scale, this works and returns something like
>  "UNIQUE"
>  405 406 407 16351
>  "REMOVE"
>  "REMOVE"
> If I try this with the entire 40 000 character list, though, I get the
> "Reached total allocation of 1022Mb: see help(memory.size)" error message.
> Can someone please give me a hint how to solve this problem correctly?
One thing you might try is not running the entire 40K list at once.
Perhaps try breaking it into 4 10K lists, running each, and combining
the results. This may get you around the allocation problem.
Another thing would be to find a system with more RAM (also read the
FAQ regarding ways to make the most amount of RAM available to R on
Windows, is that where you are?).
More information about the R-help