[R] Best Hardware & OS For Large Data Sets

[kMan]

JD,

I would recommend 64 bit, and win 7 on a quad core system has been
surprisingly stable for me. Allan's points are also spot on. I would suggest
reading the references Allan provided.

Increased memory will increase your ability to handle n vectors of 2^31-1
elements, a limit that was a design decision to make results comparable
between 32 bit and 64 bit systems. This means larger scale data will need to
be handled in chunks, but a 64 bit system lets you handle more of those
chunks in memory simultaneously. 

As far as memory allocation problems, this is just a next-jump in the
learning curve. Technically, you can handle large scale data on a 32 bit
system with 2 G RAM. It would just mean your chunks may be smaller, the
computations would take a little longer (more read calls to the disk, more
careful planning to avoid paging in Windows environments), and your
parameter estimates would be based on perhaps more aggregates. That's not to
say chunk-size could not be made the same on 64 bit systems with loads of
RAM so that the estimates would remain comparable.

Welcome to the world of portions, JD.

Sincerely,

KeithC.

-----Original Message-----
From: J. Daniel [mailto:jdlecy at gmail.com] 
Sent: Saturday, February 27, 2010 10:47 AM
To: r-help at r-project.org
Subject: [R] Best Hardware & OS For Large Data Sets


Greetings,

I am acquiring a new computer in order to conduct data analysis.  I
currently have a 32-bit Vista OS with 3G of RAM and I consistently run into
memory allocation problems.  I will likely be required to run Windows 7 on
the new system, but have flexibility as far as hardware goes.  Can people
recommend the best hardware to minimize memory allocation problems?  I am
leaning towards dual core on a 64-bit system with 8G of RAM.  Given the
Windows constraint, is there anything I am missing here?

I know that Windows limits the RAM that a single application can access. 
Does this fact over-ride many hardware considerations?  Any way around this?

Thanks,

JD


--
View this message in context:
http://n4.nabble.com/Best-Hardware-OS-For-Large-Data-Sets-tp1572129p1572129.
html
Sent from the R help mailing list archive at Nabble.com.