[R] Error: OutOfMemoryError (Java): GC overhead limit exceeded

Tue Feb 10 21:51:12 CET 2015

Another to try is the 'openxlsx' package if  you have '.xlsx' files; it
does not read '.xls' files.  It seems to be faster than XLConnect and does
not require Java; it uses Rcpp to access the APIs.

Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.

On Tue, Feb 10, 2015 at 2:13 PM, Allen Bingham <aebingham2 at gmail.com> wrote:

> Rebecca,
>
> An additional issue might be plots, pivot tables, etc. that you might have
> in separate sheets on the input Excel workbook (if you have those, just
> make a separate workbook with only the data you need for R in it).
>
> BTW---I've recently been using the following option for similar use of
> XLConnect:
>
>         options(java.parameters = "-Xmx4g" )
>
> to set it at 4 Gbytes ... but I have lots of memory on my machine (16
> Gbytes).
>
> Another issue is that the above setting is for one instance/session of R
> --- you can't re-issue it with a larger number later in the same session
> --- you have to quit R first then re-issue with a larger number.
>
> Good luck-Allen
> ______________________________________
> Allen Bingham
> Bingham Statistical Consulting
> aebingham2 at gmail.com
> LinkedIn Profile: www.linkedin.com/pub/allen-bingham/3b/556/325
>
>
>
> -----Original Message-----
> From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of jim
> holtman
> Sent: Monday, February 2, 2015 10:23 AM
> To: Yuan, Rebecca
> Cc: R help (r-help at r-project.org)
> Subject: Re: [R] Error: OutOfMemoryError (Java): GC overhead limit exceeded
>
> How big are the worksheets that you are reading in?  Do you have multiple
> ones open at the same time?  Have to tried to use 'xlcFreeMemory' to see if
> this helps?  How much RAM to you have on your system?
>
>
> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>
> On Mon, Feb 2, 2015 at 1:11 PM, Yuan, Rebecca <
> rebecca.yuan at bankofamerica.com> wrote:
>
> >  Hello Jim,
> >
> >
> >
> > I already use “.xls” for the loading, but still have the memory issue….
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Rebecca
> >
> >
> >
> >
> >
> > *From:* jim holtman [mailto:jholtman at gmail.com]
> > *Sent:* Monday, February 02, 2015 10:21 AM
> > *To:* Jeff Newmiller
> > *Cc:* Yuan, Rebecca; R help (r-help at r-project.org)
> > *Subject:* Re: [R] Error: OutOfMemoryError (Java): GC overhead limit
> > exceeded
> >
> >
> >
> > On the off-chance, are you using XLConnect or xlsx packages that are
> > using Java to access the spreadsheets?  If it is XLConnect, are you
> > access the '.xlsx' style workbooks?  If so, can you try using '.xls'
> > workbooks?  This is a problem that I have had; the '.xlsx' workbooks
> > take a lot more resources (CPU and memory) to process.
> >
> >
> >
> > A little more information like what your sessionInfo is would help a
> > lot in responding to your problem.
> >
> >
> >
> > Jim Holtman
> > Data Munger Guru
> >
> > What is the problem that you are trying to solve?
> > Tell me what you want to do, not how you want to do it.
> >
> >
> >
> > On Mon, Feb 2, 2015 at 9:27 AM, Jeff Newmiller
> > <jdnewmil at dcn.davis.ca.us>
> > wrote:
> >
> > The memory issue is in Java, not R.
> >
> > You can either be more parsimonious in your use of Java memory (we
> > have no idea what you are doing with it here, so how you do that is up
> > to you), or you can allocate more memory to Java (you may be able to
> > guess how to do that, or read the Java documentation on the X parameter).
> >
> ---------------------------------------------------------------------------
> > Jeff Newmiller                        The     .....       .....  Go
> Live...
> > DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live
> > Go...
> >                                       Live:   OO#.. Dead: OO#..  Playing
> > Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
> > /Software/Embedded Controllers)               .OO#.       .OO#.
> rocks...1k
> > ----------------------------------------------------------------------
> > ----- Sent from my phone. Please excuse my brevity.
> >
> > On February 2, 2015 5:43:50 AM PST, "Yuan, Rebecca" <
> > rebecca.yuan at bankofamerica.com> wrote:
> > >Hello all,
> > >
> > >When I met this following error message:
> > >
> > >Error: OutOfMemoryError (Java): GC overhead limit exceeded
> > >
> > >I usually use the following options to overcome the memory limit:
> > >
> > >options(java.parameters = "-Xmx1024m") # to reduce the error message
> > >"Error: OutOfMemoryError (Java): GC overhead limit exceeded"
> > >
> > >However, this seems not working any more, is there any other way to
> > >help avoiding the memory error issue in R?
> > >
> > >Thanks very much!
> > >
> > >Cheers,
> > >
> > >Rebecca
> > >
> > >
> > >
> > >20
> > >
> > >---------------------------------------------------------------------
> > >- This message, and any attachments, is for the intended
> > >r...{{dropped:5}}
> > >
> > >______________________________________________
> > >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > >https://stat.ethz.ch/mailman/listinfo/r-help
> > >PLEASE do read the posting guide
> > >http://www.R-project.org/posting-guide.html
> > >and provide commented, minimal, self-contained, reproducible code.
> >
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> >
> >  ------------------------------
> > This message, and any attachments, is for the
> > intended...{{dropped:11}}
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>

	[[alternative HTML version deleted]]