[R] R Memory Issues
Paul Johnson
pauljohn32 at gmail.com
Wed May 23 04:57:23 CEST 2012
Dear Emiliano:
When they say to read the posting guide, mostly they mean read the
posting guide. But I'll tell you the short version.
1. Include a full runable R program that causes the trouble you are
concerned about. Include the data or a link to the data, usually the
smallest possible example is what they want. They don't want 1000
lines of your dissertation project, they want 10 lines needed to
produce the problem you are concerned about.
The point here is this: Don't make people guess about what commands
you ran or what your data actually was. You are going to get the
attention of these folks one time, and you waste it by not reading the
guide and not giving the full details.
2. Include the output from sessionInfo() whenever you ask a question
of this sort.
> sessionInfo()
R version 2.15.0 (2012-03-30)
Platform: x86_64-pc-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=C LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
I can tell you my best guess about what is wrong: I suspect you have
a corrupted R install. If you had given us the full working code, I
could have tested that theory. But, alas, I can't.
Why do I think so? I've taught a course this term with 45 students and
about 1 time per week, a student would turn up with that "can't
allocate vector..." error you see. On Windows, sometimes it seems the
problem is due to installing R as an administrator and then trying to
update some packages as a non-administrator. In one really
frustrating case, student has installed "car" both as admin and as the
user, and the one that was at the front of the search path was
damaged, but we kept removing and re-installing the other one and
nothing was fixed. Until I noticed there were 2....
pj
On Tue, May 22, 2012 at 11:40 AM, Emiliano Zapata <ezapataika at gmail.com> wrote:
> As a continuation to my original question, here is the massage that I get:
>
> Error in glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, :
> cannot allocate memory block of size 2.1 Gb
>
> The model "glm.fit" is a logistic type (in the family of GLM) model. Maybe
> this is not enough information; again!, but some feedback will be
> appreciated. To me the issues appears to be associated with manipulation of
> large dataset. Howeverl the algorithm runs fine in Unix; but not in Windows
> (64 bits windows 7).
>
> EZ
>
> On Sun, May 20, 2012 at 4:09 PM, Emiliano Zapata <ezapataika at gmail.com>wrote:
>
>> Already then, thank you everyone. This information was extremly useful,
>> and I'll do a better job on the web next time.
>>
>> On Sun, May 20, 2012 at 2:10 PM, Prof Brian Ripley <ripley at stats.ox.ac.uk>wrote:
>>
>>> On 20/05/2012 18:42, jim holtman wrote:
>>>
>>>> At the point in time that you get the error message, how big are the
>>>> objects that you have in memory? What does 'memory.size()' show as
>>>> being used? What does 'memory.limit()' show? Have you tried using
>>>> 'gc()' periodically to do some garbage collection? It might be that
>>>> you memory is fragmented. You need to supply some additional
>>>> information.
>>>>
>>>
>>> Either this is a 32-bit version of R in which case the wrong version is
>>> being used, or your advice is wrong: there are no credible fragmentation
>>> issues (and no need to use gc()) on a 64-bit build of R.
>>>
>>> But, we have a posting guide, we require 'at a minimum information', and
>>> the OP failed to give it to us so we are all guessing, completely
>>> unnecessarily.
>>>
>>>
>>>
>>>> On Sun, May 20, 2012 at 12:09 PM, Emiliano Zapata<ezapataika at gmail.com>
>>>> wrote:
>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Emiliano Zapata<ezapataika at gmail.com>
>>>>> Date: Sun, May 20, 2012 at 12:09 PM
>>>>> Subject:
>>>>> To: R-help at r-project.org
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I have a 64 bits machine (Windows) with a total of 192GB of physical
>>>>> memory
>>>>> (RAM), and total of 8 CPU. I wanted to ask how can I make R make use of
>>>>> all
>>>>> the memory. I recently ran a script requiring approximately 92 GB of
>>>>> memory
>>>>> to run, and got the massage:
>>>>>
>>>>> cannot allocate memory block of size 2.1 Gb
>>>>>
>>>>>
>>>>>
>>>>> I read on the web that if you increase the memory you have to reinstall
>>>>> R;
>>>>> would that be enough. Could I just increase the memory manually.
>>>>>
>>>>>
>>>>> Take you for any comments, or links on the web.
>>>>>
>>>>>
>>>>> EZ
>>>>>
>>>>> [[alternative HTML version deleted]]
>>>>>
>>>>> ______________________________**________________
>>>>> R-help at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>>>>> PLEASE do read the posting guide http://www.R-project.org/**
>>>>> posting-guide.html <http://www.R-project.org/posting-guide.html>
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> --
>>> Brian D. Ripley, ripley at stats.ox.ac.uk
>>> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~**ripley/<http://www.stats.ox.ac.uk/%7Eripley/>
>>> University of Oxford, Tel: +44 1865 272861 (self)
>>> 1 South Parks Road, +44 1865 272866 (PA)
>>> Oxford OX1 3TG, UK Fax: +44 1865 272595
>>>
>>
>>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Paul E. Johnson
Professor, Political Science Assoc. Director
1541 Lilac Lane, Room 504 Center for Research Methods
University of Kansas University of Kansas
http://pj.freefaculty.org http://quant.ku.edu
More information about the R-help
mailing list