[R] Memory Limits in Ubuntu Linux

[Bos, Roger]

David,

I wouldn't give up on windows so fast.  Many people have gotten the 3Gb
switch to work. One used to have to modify the header of the Rgui.exe
program to use the switch, but now the binary comes ready for that, so
its really quite easy.  I would like to hear more about why its not
working for you.

As for Linux, I use FC5 for which there is a 64-bit binary.  But there
are also 64-bit binaries for other distros.  The 32-bit and 64-bit
binaries are in different directories, so you should have no trouble
telling them apart.  

I have heard good things about Ubuntu--mainly that its very easy to
use--but FC5 has been pretty easy to learn too and I use the KDE desktop
which gives me Kate as a text editor.  You can open a terminal window in
Kate to run R and set up a key like F10 to send the code from the editor
to R.  Its not quite as good as my Windows setup with Tinn-R, but almost
as good.

Thanks,

Roger


-----Original Message-----
From: davidkat at davidkatzconsulting.com
[mailto:davidkat at davidkatzconsulting.com] 
Sent: Tuesday, March 06, 2007 5:37 PM
To: Bos, Roger
Subject: RE: [R] Memory Limits in Ubuntu Linux

Thanks for your prompt reply!

The windows 3GB switch is quite problematic - it was not useable on my
machine, and there are comments about these problems around the net.
Thus, on to Linux. My machine has 4Gig, and some megabytes are grabbed
by my Asus motherboard, leaving some 3.56 Gig. 

So if I understand your suggestion, try the 64-bit version of Ubuntu
(based on Debian but I had better luck with the video part of the
install) and then use the corresponding image from CRAN. My fear is that
the CRAN Ubuntu version might be 32-bit - any idea how to find out
before I embark on another install? Which Linux do you have - you
described some significant success with getting large jobs to run.

And yes, I've worked hard to save memory by tweaking the code.


Thanks again.


On 6 Mar 2007 at 16:51, Bos, Roger wrote:

> David,
> 
> First of all, under Windows you can get about 3GB available to R by 
> using the /3Gb switch in your boot.ini file, assuming you have 4Gb of 
> memory installed on your windows machine.  Using that method, I have 
> seen the memory using of my R process get as big as 2.7Gb in task 
> manager.  What's important, of course, is contiguous space, as you 
> mentioned.  There, you may want to check your code closely and make 
> sure that its memory usage is as efficient as possible and you are 
> storing the minimal amount you need for each run.  If you don't need 
> an object for a while consider writing it to disk and reading it back
in later.
> 
> Second, AFAIK to get any benefit from more memory is Linux you have to

> go to the 64bit version.  I am a Linux newbie too, so I choose to use 
> one of the pre-compiled binaries available on CRAN.  In other words, 
> you shouldn't have to compile anything yourself.  How much memory do 
> you have on your Linux box?  I have 16Gb and I know I have ran stuff 
> that wouldn't run on my 4Gb windows box.
> 
> HTH,
> 
> Roger
> 
> 
> 
> 
> 
>  
> 
> -----Original Message-----
> From: r-help-bounces at stat.math.ethz.ch 
> [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of 
> davidkat at davidkatzconsulting.com
> Sent: Tuesday, March 06, 2007 3:44 PM
> To: r-help at stat.math.ethz.ch
> Subject: [R] Memory Limits in Ubuntu Linux
> 
> I am an R user trying to get around the 2Gig memory limit in Windows, 
> so here I am days later with a working Ubuntu, and R under Ubuntu. But

> - the memory problems seem worse than ever. R code that worked under 
> windows fails, unable to allocate memory.
> 
> Searching around the web, it appears that the problem may be the 
> ability to find contguous memory for my big vectors, but a fresh boot 
> of Ubuntu does not help either.
> 
> Which way to go?
> 
> 1) Try to install 64-bit version for bigger address space. Would this 
> help? Is this workable for my Athlon 64 Dual-core? (the live cd seems 
> to work but I never got it to boot after a disk install, but then the 
> 386 version was no better until I learned more about Grub...I could 
> try again if this might solve the
> problem)
> 
> 2) Recompile R to get bigger memory capability? (I'll have to 
> cross-post to some R forums too) This will be a challenge for a Linux 
> newbie...like me.
> 
> 3) Any other suggestions? My goal is to create a bigger neural network

> than fits in my Windows R version.
> --
> David Katz
>  www.davidkatzconsulting.com
>    541 482-1137
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 
> **********************************************************************

> * This message is for the named person's use only. It may contain 
> confidential, proprietary or legally privileged information. No right 
> to confidential or privileged treatment of this message is waived or 
> lost by any error in transmission. If you have received this message 
> in error, please immediately notify the sender by e-mail, delete the 
> message and all copies from your system and destroy any hard copies. 
> You must not, directly or indirectly, use, disclose, distribute, print

> or copy any part of this message if you are not the intended 
> recipient.
> **********************************************************************
> 

--
David Katz
   www.davidkatzconsulting.com


********************************************************************** * 
This message is for the named person's use only. It may 
contain confidential, proprietary or legally privileged 
information. No right to confidential or privileged treatment 
of this message is waived or lost by any error in 
transmission. If you have received this message in error, 
please immediately notify the sender by e-mail, 
delete the message and all copies from your system and destroy 
any hard copies. You must not, directly or indirectly, use, 
disclose, distribute, print or copy any part of this message 
if you are not the intended recipient.