[R] Memory Limits in Ubuntu Linux

[Bos, Roger]

David,

Here is what my boot.ini file looks like:

[boot loader]
timeout=5
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP
Professional" /noexecute=optin /fastdetect /3gb

The easiest way to edit the boot.ini file is My
Computer/Properties/Advanced/Startup & Recovery/Edit; add the /3gb and
reboot.  I know that a messed up boot.ini file can be a real pain.  I
posted what mine looks like so you can compare yours, but I wouldn't
suggest making any changes to your boot.ini except at the very end of
the last line.

HTH,

Roger


 

-----Original Message-----
From: davidkat at davidkatzconsulting.com
[mailto:davidkat at davidkatzconsulting.com] 
Sent: Wednesday, March 07, 2007 12:27 PM
To: Bos, Roger
Cc: r-help at stat.math.ethz.ch
Subject: RE: [R] Memory Limits in Ubuntu Linux

Thanks for the tips, Roger.

fyi: When I added /3GB to the boot.ini, the resulting desktop was
incomplete and locked - no chance to try starting R. Searching the web
lead me to believe that this was possibly a dead-end, so I abandoned
this effort. Any hints on getting this to work, anyone? 



On 7 Mar 2007 at 8:26, Bos, Roger wrote:

> David,
> 
> I wouldn't give up on windows so fast.  Many people have gotten the 
> 3Gb switch to work. One used to have to modify the header of the 
> Rgui.exe program to use the switch, but now the binary comes ready for

> that, so its really quite easy.  I would like to hear more about why 
> its not working for you.
> 
> As for Linux, I use FC5 for which there is a 64-bit binary.  But there

> are also 64-bit binaries for other distros.  The 32-bit and 64-bit 
> binaries are in different directories, so you should have no trouble 
> telling them apart.
> 
> I have heard good things about Ubuntu--mainly that its very easy to 
> use--but FC5 has been pretty easy to learn too and I use the KDE 
> desktop which gives me Kate as a text editor.  You can open a terminal

> window in Kate to run R and set up a key like F10 to send the code 
> from the editor to R.  Its not quite as good as my Windows setup with 
> Tinn-R, but almost as good.
> 
> Thanks,
> 
> Roger
> 
> 
> -----Original Message-----
> From: davidkat at davidkatzconsulting.com 
> [mailto:davidkat at davidkatzconsulting.com]
> Sent: Tuesday, March 06, 2007 5:37 PM
> To: Bos, Roger
> Subject: RE: [R] Memory Limits in Ubuntu Linux
> 
> Thanks for your prompt reply!
> 
> The windows 3GB switch is quite problematic - it was not useable on my

> machine, and there are comments about these problems around the net.
> Thus, on to Linux. My machine has 4Gig, and some megabytes are grabbed

> by my Asus motherboard, leaving some 3.56 Gig.
> 
> So if I understand your suggestion, try the 64-bit version of Ubuntu 
> (based on Debian but I had better luck with the video part of the
> install) and then use the corresponding image from CRAN. My fear is 
> that the CRAN Ubuntu version might be 32-bit - any idea how to find 
> out before I embark on another install? Which Linux do you have - you 
> described some significant success with getting large jobs to run.
> 
> And yes, I've worked hard to save memory by tweaking the code.
> 
> 
> Thanks again.
> 
> 
> On 6 Mar 2007 at 16:51, Bos, Roger wrote:
> 
> > David,
> > 
> > First of all, under Windows you can get about 3GB available to R by 
> > using the /3Gb switch in your boot.ini file, assuming you have 4Gb 
> > of memory installed on your windows machine.  Using that method, I 
> > have seen the memory using of my R process get as big as 2.7Gb in 
> > task manager.  What's important, of course, is contiguous space, as 
> > you mentioned.  There, you may want to check your code closely and 
> > make sure that its memory usage is as efficient as possible and you 
> > are storing the minimal amount you need for each run.  If you don't 
> > need an object for a while consider writing it to disk and reading 
> > it back
> in later.
> > 
> > Second, AFAIK to get any benefit from more memory is Linux you have 
> > to
> 
> > go to the 64bit version.  I am a Linux newbie too, so I choose to 
> > use one of the pre-compiled binaries available on CRAN.  In other 
> > words, you shouldn't have to compile anything yourself.  How much 
> > memory do you have on your Linux box?  I have 16Gb and I know I have

> > ran stuff that wouldn't run on my 4Gb windows box.
> > 
> > HTH,
> > 
> > Roger
> > 
> > 
> > 
> > 
> > 
> >  
> > 
> > -----Original Message-----
> > From: r-help-bounces at stat.math.ethz.ch 
> > [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of 
> > davidkat at davidkatzconsulting.com
> > Sent: Tuesday, March 06, 2007 3:44 PM
> > To: r-help at stat.math.ethz.ch
> > Subject: [R] Memory Limits in Ubuntu Linux
> > 
> > I am an R user trying to get around the 2Gig memory limit in 
> > Windows, so here I am days later with a working Ubuntu, and R under 
> > Ubuntu. But
> 
> > - the memory problems seem worse than ever. R code that worked under

> > windows fails, unable to allocate memory.
> > 
> > Searching around the web, it appears that the problem may be the 
> > ability to find contguous memory for my big vectors, but a fresh 
> > boot of Ubuntu does not help either.
> > 
> > Which way to go?
> > 
> > 1) Try to install 64-bit version for bigger address space. Would 
> > this help? Is this workable for my Athlon 64 Dual-core? (the live cd

> > seems to work but I never got it to boot after a disk install, but 
> > then the
> > 386 version was no better until I learned more about Grub...I could 
> > try again if this might solve the
> > problem)
> > 
> > 2) Recompile R to get bigger memory capability? (I'll have to 
> > cross-post to some R forums too) This will be a challenge for a 
> > Linux newbie...like me.
> > 
> > 3) Any other suggestions? My goal is to create a bigger neural 
> > network
> 
> > than fits in my Windows R version.
> > --
> > David Katz
> >  www.davidkatzconsulting.com
> >    541 482-1137
> > 
> > 	[[alternative HTML version deleted]]
> > 
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list 
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> > 
> > ********************************************************************
> > **
> 
> > * This message is for the named person's use only. It may contain 
> > confidential, proprietary or legally privileged information. No 
> > right to confidential or privileged treatment of this message is 
> > waived or lost by any error in transmission. If you have received 
> > this message in error, please immediately notify the sender by 
> > e-mail, delete the message and all copies from your system and
destroy any hard copies.
> > You must not, directly or indirectly, use, disclose, distribute, 
> > print
> 
> > or copy any part of this message if you are not the intended 
> > recipient.
> > ********************************************************************
> > **
> > 
> 
> --
> David Katz
>    www.davidkatzconsulting.com
> 
> 
> **********************************************************************

> * This message is for the named person's use only. It may contain 
> confidential, proprietary or legally privileged information. No right 
> to confidential or privileged treatment of this message is waived or 
> lost by any error in transmission. If you have received this message 
> in error, please immediately notify the sender by e-mail, delete the 
> message and all copies from your system and destroy any hard copies. 
> You must not, directly or indirectly, use, disclose, distribute, print

> or copy any part of this message if you are not the intended 
> recipient.
> **********************************************************************
> 

--
David Katz
   www.davidkatzconsulting.com


********************************************************************** * 
This message is for the named person's use only. It may 
contain confidential, proprietary or legally privileged 
information. No right to confidential or privileged treatment 
of this message is waived or lost by any error in 
transmission. If you have received this message in error, 
please immediately notify the sender by e-mail, 
delete the message and all copies from your system and destroy 
any hard copies. You must not, directly or indirectly, use, 
disclose, distribute, print or copy any part of this message 
if you are not the intended recipient.