[R] Memory Limits in Ubuntu Linux

[Dirk Eddelbuettel]

On 6 March 2007 at 12:43, davidkat at davidkatzconsulting.com wrote:
| I am an R user trying to get around the 2Gig memory limit in Windows, so 

The real limit on 32bit systems is a 3gb address space. R under Windows can
get there, see the R-Windows FAQ.

| here I am days later with a working Ubuntu, and R under Ubuntu. But - the 
| memory problems seem worse than ever. R code that worked under 
| windows fails, unable to allocate memory.

Well, maybe you had virtual memory enabled under Windows but not under
Ubuntu. Or maybe you had other memory-hungry applications up under Ubuntu.

There is only so much magic the OS can do.  You easiest remedy will be to
upgrade to 4gb.  And even 8gb can useful on 32bit system, despite the fact
that each individual address space can only max out at 3gb, as you may have
multi-core / multi-cpu systems that allow you to multitask better.  
 
| Which way to go?
| 
| 1) Try to install 64-bit version for bigger address space. Would this help?

Yes, but you'd probably would have to buy more ram to. The main advantage is
that your limit is now way above the 3gb -- and probably set by your hardware
or budget. Maybe it is as high as 16gb.

But again, on the _same_ box with the _same_ amount of ram that is already
constrained under 32bit, you will not see any improvement.  Rather the
opposite as the basic building block is now 8 bytes instead of 4, you will
need more memory for the same tasks.  No free lunch, as they say.

| 2) Recompile R to get bigger memory capability?

Nope. It's what you give your OS in terms of RAM what's binding here.

| 3) Any other suggestions? 

Different algorithms or approaches, tricks like the data.frame-in-sqlite or
biglm, ...

Dirk

-- 
Hell, there are no rules here - we're trying to accomplish something. 
                                                  -- Thomas A. Edison