[R] Memory Size & Allocation in R

[jim holtman]

Try writing out the dataframe in smaller pieces.  If you have 800K row
and 18 columns, then this is about 14M items and if you have 8 bytes
per items (assuming all numeric; more for character), this is 115MB of
space that would probably be required to construct the output data (or
more).  Your error might be due to someplace along the line of getting
the memory, it was too fragmented to get the space required.  Try
writing out 20% at a time and appending to the file, or using a
connection.

On Tue, Jan 13, 2009 at 12:24 PM, Brigid Mooney <bkmooney at gmail.com> wrote:
> My apologies if this is a bit of a 'newbie' question.
>
> I am using R v 2.8.0 in Windows and am a bit confused about the memory
> size/allocation.
>
> A script I wrote faulted out with the error: "Error: cannot allocate vector
> of size 5.6 Mb"
>
> After this error, I still have:
>
>> memory.size()
> [1] 669.3517
>> memory.limit()
> [1] 1535.875
>
> Since the memory size is well under 5.6Mb less than the memory limit, I
> assume there is some limit on object size within R.  Is this correct?  If
> so, is there a way to determine which one of my objects is too large? - So
> that I can remove it or update my script.
>
> Also, is there a way to temporarily increase the limit on vector memory
> allocation above 5.6Mb?  I'm hoping to still retreive the data that was
> calculated in the run prior to the error.
>
> To get at this data, I tried write.csv and got a similar error:
>> write.csv(Results, "TempOutput_011309.csv", row.names=FALSE,
> col.names=TRUE)
> Error: cannot allocate vector of size 5.5 Mb
>
> For reference, 'Results' is a data frame with about 800K rows and 18 columns
> (1 col contains character strings of class factor, 1 col contains Date/Time
> stamps of class factor, the remaining cols are all numeric).
>
> Any help here is greatly appreciated - Thanks!
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?