[R-sig-eco] Large data processing in R

[ONKELINX, Thierry]

Dear Noel,

Have a look at the High-Performance and Parallel Computing task view on CRAN.

Best regards,

Thierry

ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest
team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
Kliniekstraat 25
1070 Anderlecht
Belgium
+ 32 2 525 02 51
+ 32 54 43 61 85
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey


-----Oorspronkelijk bericht-----
Van: r-sig-ecology-bounces at r-project.org [mailto:r-sig-ecology-bounces at r-project.org] Namens Noel Aloysius
Verzonden: maandag 12 maart 2012 13:13
Aan: r-sig-ecology at r-project.org
Onderwerp: [R-sig-eco] Large data processing in R

Hi all,

I have a model output file, size 120gb, I want to process. It has about
10 million records written in f8.2 format. Each record has variables in string, integer and double formats.

Are there R packages that are capable of loading this dataset without exhausting the memory? I am trying to use a dual core, 1.6GHz, 4gb RAM machine.

Thank you in advance for your suggestions,

Noel

_______________________________________________
R-sig-ecology mailing list
R-sig-ecology at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology