[R] Out of memory issue

[David Winsemius]

On Apr 24, 2009, at 9:06 PM, Neotropical bat risk assessments wrote:

>
>   Hi all,
>   Julian just reminded me I need to include the lines of code so  
> here it is
>   below.
>   I am trying to run some plots on data, but when loading the CSV  
> data file it
>   doss that but in ggplot R is stopping and I am getting an out of  
> memory
>   error.
>>
>> d <- read.csv batcalls.rda
>   Error: unexpected symbol in "d <- read.csv batcalls.rda"

If you really do have a data file called batcalls.rda, then the proper  
way to get it into memory is with:

load("C:/Rainey//batcalls.rda")  # replace path with the actual path  
on your system and remember not to use "\"'s

>
>> attach(d)
>   Error in attach(d) : object "d" not found

Because of the error ... nothing got done.

>
>> library(ggplot2)
>> print(qplot(Sc, Fc))
>   Error in eval(expr, envir, enclos) : object "Sc" not found

You cannot do anything on something that isn't there
>
>> data(batcalls)
>>    attach(batcalls)
>>    print(qplot(Fc,geom="density"))
>>    b<-kmeans(Fc,c(10,15,18,30))
>>   print(qplot(Sc,Fc,colour=as.factor(b$cluster)))
>>
>>  print(qplot(Dur,geom="density"))
>>    b<-kmeans(Dur,c(10,15,18,30))
>>   print(qplot(Dur,Fc,colour=as.factor(b$cluster)))
>>
>> d <- read.csv ("C:/Rainey/RainyAllbats.csv")

OK, that appears to have run without error, so it fits into memory.  
Now you should run:
str(d)   # and report the results

>>
>> attach(d)
>          The following object(s) are masked from batcalls :
>           Dc Dur Fc Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC Tc Tk
>> library(ggplot2)
>> print(qplot(Sc, Fc))
>   Warning messages:
>   1: In data.frame(..., check.names = FALSE) :
>     Reached total allocation of 1535Mb: see help(memory.size)
>   2: In data.frame(..., check.names = FALSE) :
>     Reached total allocation of 1535Mb: see help(memory.size)
>   3: Removed 1200249 rows containing missing values (geom_point).
>> data(d)
>   Warning message:
>   In data(d) : data set 'd' not found
>>    attach(d)
>          The following object(s) are masked from d ( position 3 ) :
>           Dc Dur Fc Filename Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC  
> Tc Tk
>          The following object(s) are masked from batcalls :
>           Dc Dur Fc Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC Tc Tk
>>    print(qplot(Fc,geom="density"))
>   There were 15 warnings (use warnings() to see them)
>>    b<-kmeans(Fc,c(10,15,18,30))
>   Error in switch(nmeth, { : NA/NaN/Inf in foreign function call  
> (arg 1)
>> f=jpeg(file="Sc x Fc.jpg")
>>   print(qplot(Sc,Fc,colour=as.factor(b$cluster)))
>   Error in data.frame(colour = c(3L, 4L, 4L, 4L, 4L, 1L, 4L, 2L, 1L,  
> 1L, :
>     arguments imply differing number of rows: 9404, 2400489

You seem to be ignoring all the informative error message.
>
>> dev.off()
>   Anyway to tweak this somehow to get it to run?
>   Using WinXP with 4 GB RAM

WinXP in its default state can only use around 2.5GB and that may  
require upping the memory limits:
<http://cran.r-project.org/bin/windows/base/rw-FAQ.html#There-seems-to-be-a-limit-on-the-memory-it-uses_0021 
 >

And read the material in the links from that page as well.

-- 


David Winsemius, MD
Heritage Laboratories
West Hartford, CT