[R] Error: OutOfMemoryError (Java): GC overhead limit exceeded

[Allen Bingham]

Rebecca,

An additional issue might be plots, pivot tables, etc. that you might have in separate sheets on the input Excel workbook (if you have those, just make a separate workbook with only the data you need for R in it).

BTW---I've recently been using the following option for similar use of XLConnect:

	options(java.parameters = "-Xmx4g" )

to set it at 4 Gbytes ... but I have lots of memory on my machine (16 Gbytes).

Another issue is that the above setting is for one instance/session of R --- you can't re-issue it with a larger number later in the same session --- you have to quit R first then re-issue with a larger number.

Good luck-Allen
______________________________________
Allen Bingham
Bingham Statistical Consulting
aebingham2 at gmail.com
LinkedIn Profile: www.linkedin.com/pub/allen-bingham/3b/556/325



-----Original Message-----
From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of jim holtman
Sent: Monday, February 2, 2015 10:23 AM
To: Yuan, Rebecca
Cc: R help (r-help at r-project.org)
Subject: Re: [R] Error: OutOfMemoryError (Java): GC overhead limit exceeded

How big are the worksheets that you are reading in?  Do you have multiple ones open at the same time?  Have to tried to use 'xlcFreeMemory' to see if this helps?  How much RAM to you have on your system?


Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

On Mon, Feb 2, 2015 at 1:11 PM, Yuan, Rebecca < rebecca.yuan at bankofamerica.com> wrote:

>  Hello Jim,
>
>
>
> I already use “.xls” for the loading, but still have the memory issue….
>
>
>
> Thanks,
>
>
>
> Rebecca
>
>
>
>
>
> *From:* jim holtman [mailto:jholtman at gmail.com]
> *Sent:* Monday, February 02, 2015 10:21 AM
> *To:* Jeff Newmiller
> *Cc:* Yuan, Rebecca; R help (r-help at r-project.org)
> *Subject:* Re: [R] Error: OutOfMemoryError (Java): GC overhead limit 
> exceeded
>
>
>
> On the off-chance, are you using XLConnect or xlsx packages that are 
> using Java to access the spreadsheets?  If it is XLConnect, are you 
> access the '.xlsx' style workbooks?  If so, can you try using '.xls' 
> workbooks?  This is a problem that I have had; the '.xlsx' workbooks 
> take a lot more resources (CPU and memory) to process.
>
>
>
> A little more information like what your sessionInfo is would help a 
> lot in responding to your problem.
>
>
>
> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>
>
>
> On Mon, Feb 2, 2015 at 9:27 AM, Jeff Newmiller 
> <jdnewmil at dcn.davis.ca.us>
> wrote:
>
> The memory issue is in Java, not R.
>
> You can either be more parsimonious in your use of Java memory (we 
> have no idea what you are doing with it here, so how you do that is up 
> to you), or you can allocate more memory to Java (you may be able to 
> guess how to do that, or read the Java documentation on the X parameter).
> ---------------------------------------------------------------------------
> Jeff Newmiller                        The     .....       .....  Go Live...
> DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live
> Go...
>                                       Live:   OO#.. Dead: OO#..  Playing
> Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
> /Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
> ----------------------------------------------------------------------
> ----- Sent from my phone. Please excuse my brevity.
>
> On February 2, 2015 5:43:50 AM PST, "Yuan, Rebecca" < 
> rebecca.yuan at bankofamerica.com> wrote:
> >Hello all,
> >
> >When I met this following error message:
> >
> >Error: OutOfMemoryError (Java): GC overhead limit exceeded
> >
> >I usually use the following options to overcome the memory limit:
> >
> >options(java.parameters = "-Xmx1024m") # to reduce the error message
> >"Error: OutOfMemoryError (Java): GC overhead limit exceeded"
> >
> >However, this seems not working any more, is there any other way to 
> >help avoiding the memory error issue in R?
> >
> >Thanks very much!
> >
> >Cheers,
> >
> >Rebecca
> >
> >
> >
> >20
> >
> >---------------------------------------------------------------------
> >- This message, and any attachments, is for the intended 
> >r...{{dropped:5}}
> >
> >______________________________________________
> >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see 
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see 
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
>  ------------------------------
> This message, and any attachments, is for the \ > in...{{dropped:10}}