[R] Memory(RAM) issues

Fri Jun 24 10:27:52 CEST 2011

...you should find a 64 bit Linux version here:

http://cran.r-project.org/bin/linux/


On Thu, Jun 23, 2011 at 6:04 PM, David Winsemius <dwinsemius at comcast.net> wrote:
>
> On Jun 23, 2011, at 11:57 AM, Abhisek Saha wrote:
>
>> Hi Lui, Anupam and my other R-user friends,
>> Thanks  for your previous suggestions. As for my issue, it is clearly
>> RAM problem as my code is running perfectly as long as my input data
>> size is small and code has been refined a number of times to increase
>> efficiency [ by using matrix more in the context of manipulations and
>> reducing order of time complexity wherever possible]. Yet  it has
>> gotta run 2000 iterations and in each of the iterations it repeatedly
>> modifies  few [0.5 Mn * 15] tables
>>
>> After few basic investigation it seems that the problem MAY be
>> resolved if 64 bit version of R is installed in 64 bit OS as it claims
>> to be able use potentially a  8TB RAM ( for my case I think 4 GB RAM
>> may serve the purpose but 2 GB is not enough) if available.I don't
>> have any domain knowledge in IT since I hail from a statistics
>> background.  I work in a company as a statistical analyst. The IT
>> folks of my company are ready to deploy any free solution but I need
>> to tell them what software needs to be installed and what all OS
>> configurations would be required. They normally run all of the
>> applications on unix servors.
>>
>> So I need to know if any free 64 bit of R version can be installed in
>> unix servor or if not unix, may be on other servor. So hereby I
>> request my R-user friends to please let me know if any free 64 bit
>> version of R is available
>
> Yes.
>
>> that can be installed on any unix servor. If
>> that is not available is there any other FREE  solution and if
>> available, how to get that and what all configuration is required.
>
> There are a bunch of "UNIX-alike" platforms. Read the FAQ:
>
> http://cran.r-project.org/doc/FAQ/R-FAQ.html#What-machines-does-R-run-on_003f
>
> -- David.
>>
>> Awaiting your replies,
>> Regards,
>> Abhisek
>>
>> P.S. Please forward this mail to any other R-mailing list if you deem
>> it fit for any of them.
>>
>> On Sat, Jun 11, 2011 at 4:38 PM, Lui ## <lui.r.project at googlemail.com>
>> wrote:
>>>
>>> Hello Abhisek,
>>>
>>> maybe you wanna try it on just a bigger machine (I guess you are
>>> working at a university, so I hope you do have access to them). In
>>> case getting computing time or the like is a major issue, you may
>>> wanna try Amazon AWS: For a few rupees (about 50-100 per hour) you can
>>> "rent" pretty fast computers (20 Ghz, 8BG of RAM). You may want to try
>>> out the Windows version (little bit more expensive) which is easily
>>> accessible via remote desktop. Installing Revolution (which is free
>>> for academic purposes) (64 Bit Version) might give you a good start.
>>> Maybe its not a viable option in a long term view (pricing), but it
>>> might help to get a clue whether the problem can be solved on a bigger
>>> machine and just trying it out without a bigger hastle...
>>>
>>> Good luck!
>>>
>>> Lui
>>>
>>> On Sat, Jun 11, 2011 at 7:47 AM, Abhisek Saha <tading at gmail.com> wrote:
>>>>
>>>> Thanks Anupam for your inputs. I believe there are two ways to
>>>> circumvent the issue...1> making the code more efficient 1> Increasing
>>>> the memory in some way.The reasons why I did not paste the code are 1>
>>>> It is long and using quite a number of functions that  I created 2>
>>>> Secondly my intention is not to make the code more efficient if that's
>>>> possible. Here landing into a memory problem with 2 GB RAM is natural
>>>> as my analysis entails 1500 simulations from huge multivariate
>>>> distributions that change after every simulation and tomorrow I may
>>>> have to do similar analysis with 10 million observations * 20 columns.
>>>>
>>>> In view of above I shall be needing more memory sometime later and my
>>>> IT friends are ready to support me for that(probably with a sandbox)
>>>> but I am not sure whether I can install probably a servor version of R
>>>> that can be capable of working with 8GB or so RAM. So it is more of
>>>> technical help I need and I have no clue regarding the plausibility of
>>>> the solution mentioned( i.e. a servor version of R that is capable of
>>>> more memory).
>>>>
>>>> Regards,
>>>> Abhisek
>>>>
>>>> On Sat, Jun 11, 2011 at 10:10 AM, Anupam <anupamtg at gmail.com> wrote:
>>>>>
>>>>> It will be helpful on this forum to use metric measures: 12 Lakh is 1.2
>>>>> million, thus your data is 1.2 million observations x 15 variables. I
>>>>> do not
>>>>> know the intricacies of R. You may have to wait for someone with that
>>>>> knowledge to respond.
>>>>>
>>>>> Including some relevant portions of error messages and code in your
>>>>> query
>>>>> can also help someone to respond to your message.
>>>>>
>>>>> Anupam.
>>>>> -----Original Message-----
>>>>> From: r-help-bounces at r-project.org
>>>>> [mailto:r-help-bounces at r-project.org] On
>>>>> Behalf Of Abhisek Saha
>>>>> Sent: Saturday, June 11, 2011 6:25 AM
>>>>> To: r-help at r-project.org
>>>>> Subject: [R] Memory(RAM) issues
>>>>>
>>>>> Dear All,
>>>>> I have been working with R(desktop version) in VISTA. I have the latest
>>>>> version R 2.13.0. I have been working with few data-sets of size 12Lakh
>>>>> * 15
>>>>> and my code is quite computing intensive ( applying MCMC gibbs sampler
>>>>> on a
>>>>> posterior of 144 variables) that often runs into a memory issue such as
>>>>> memory can not allocate the vector ,full size(shows to have reached
>>>>> something like 1.5 GB) reached or something to this effect. I have a
>>>>> RAM of
>>>>> 2GB.  I checked with the option like memory.size and it says a 64 bit R
>>>>> if
>>>>> sat on 64 bit windows then max memory capability is 8TB.
>>>>>
>>>>> Now I don't have  background to understand the definitions and
>>>>> differences
>>>>> between 32 and 64 bit machines and other technical requirements like
>>>>> servor
>>>>> etc but it would be of great help if anyone can let me have a feel of
>>>>> it.
>>>>> Could any of you tell me whether some servor version of R would resolve
>>>>> my
>>>>> issue or not (I am not sure now what kind of servor my company would
>>>>> allow R
>>>>> to be installed at this point ,may be linux type) and if that's the
>>>>> case
>>>>> could any of you guide me about how to go about installing that onto a
>>>>> sevor.
>>>>>
>>>>> Thank you,
>>>>> Abhisek
>>>>>
>>>>>       [[alternative HTML version deleted]]
>>>>>
>>>>> ______________________________________________
>>>>> R-help at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> PLEASE do read the posting guide
>>>>> http://www.R-project.org/posting-guide.html
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius, MD
> West Hartford, CT
>
>