[R] Memory management in R

[Mike Marchywka]

----------------------------------------
> Date: Fri, 8 Oct 2010 13:30:59 -0400
> From: jholtman at gmail.com
> To: lorenzo.isella at gmail.com
> CC: r-help at r-project.org
> Subject: Re: [R] Memory management in R
>
> More specificity: how long is the string, what is the pattern you are
> matching against? It sounds like you might have a complex pattern
> that in trying to match the string might be doing a lot of back
> tracking and such. There is an O'Reilly book on Mastering Regular
> Expression that might help you understand what might be happening. So
> if you can provide a better example than just the error message, it
> would be helpful.


This is possibly a stack issue. Error messages are not often literal,
I have seen out of memory for graphic device objects :) Regex suggests
a stack issue but that would be a guess on the mechanism of death but
what you probably really want is a simpler regex :)




>
> On Fri, Oct 8, 2010 at 1:11 PM, Lorenzo Isella  wrote:
> > Dear All,
> > I am experiencing some problems with a script of mine.
> > It crashes with this message
> >
> > Error in grepl(fut_string, past_string) :
> >  invalid regular expression
> > '12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12653a6#12
> > Calls: entropy_estimate_hash -> total_entropy_lz -> entropy_lz -> grepl
> > In addition: Warning message:
> > In grepl(fut_string, past_string) : regcomp error:  'Out of memory'
> > Execution halted
> >
> > To make a long story short, I use some functions which eventually call grepl
> > on very long strings to check whether a certain substring is part of a
> > longer string.
> > Now, the script technically works (it never crashes when I run it on a
> > smaller dataset) and the problem does not seem to be RAM memory (I have
> > several GB of RAM on my machine and its consumption never shoots up so my
> > machine never resorts to swap memory).
> > So (though I am not an expert) it looks like the problem is some limitation
> > of grepl or R memory management.
> > Any idea about how I could tackle this problem or how I can profile my code
> > to fix it (though it really seems to me that I have to find a way to allow R
> > to process longer strings).
> > Any suggestion is appreciated.
> > Cheers
> >
> > Lorenzo
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
>
>
> --
> Jim Holtman
> Cincinnati, OH
> +1 513 646 9390
>
> What is the problem that you are trying to solve?
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.