[R] Large Dataset

[Simon Pickett]

Hi,

I am not very knowledgeable about this kind of stuff but my guess is that if 
you have a fairly slow computer and massive data sets there isnt alot you 
can do except get a better computer, buy more RAM or use something like SAS 
instead?

Hopefully someone else will chip in Edwin, best of luck.

Simon.


----- Original Message ----- 
From: "Edwin Sendjaja" <edwin7 at web.de>
To: "Simon Pickett" <simon.pickett at bto.org>
Cc: <r-help at r-project.org>
Sent: Tuesday, January 06, 2009 2:53 PM
Subject: Re: [R] Large Dataset


> Hi Simon,
>
> My RAM is only 3.2 GB (actually it should be 4 GB, but my Motherboard 
> doesnt
> support it.
>
> R use almost of all my RAM and half of my swap. I think memory.limit will 
> not
> solve my problem.  It seems that I need  RAM.
>
> Unfortunately, I can't buy more RAM.
>
> Why R is slow reading big data set?
>
>
> Edwin
>
>> Only a couple of weeks ago I had to deal with this.
>>
>> adjust the memory limit as follows, although you might not want 4000, 
>> that
>> is quite high....
>>
>> memory.limit(size = 4000)
>>
>> Simon.
>>
>> ----- Original Message -----
>> From: "Edwin Sendjaja" <edwin7 at web.de>
>> To: "Simon Pickett" <simon.pickett at bto.org>
>> Cc: <r-help at r-project.org>
>> Sent: Tuesday, January 06, 2009 12:24 PM
>> Subject: Re: [R] Large Dataset
>>
>> > Hi Simon,
>> >
>> > Thank for your reply.
>> > I have read ?Memory but I dont understand how to use. I am not sure if
>> > that
>> > can solve my problem. Can you tell me more detail?
>> >
>> > Thanks,
>> >
>> > Edwin
>> >
>> >> type
>> >>
>> >> ?memory
>> >>
>> >> into R and that will explain what to do...
>> >>
>> >> S
>> >> ----- Original Message -----
>> >> From: "Edwin Sendjaja" <edwin7 at web.de>
>> >> To: <r-help at r-project.org>
>> >> Sent: Tuesday, January 06, 2009 11:41 AM
>> >> Subject: [R] Large Dataset
>> >>
>> >> > Hi alI,
>> >> >
>> >> > I  have a 3.1 GB Dataset ( with  11 coloumns and lots data in int 
>> >> > and
>> >> > string).
>> >> > If I use read.table; it takes very long. It seems that my RAM is not
>> >> > big
>> >> > enough (overload) I have 3.2 RAM and  7GB SWAP, 64 Bit Ubuntu.
>> >> >
>> >> > Is there a best sultion to read a large data R? I have seen, that
>> >> > people
>> >> > suggest to use bigmemory package, ff. But it seems very complicated.
>> >> > I dont
>> >> > know how to start with that packages.
>> >> >
>> >> > i have tried to use bigmemory. But I got some kind of errors.  Then 
>> >> > I
>> >> > gave up.
>> >> >
>> >> >
>> >> > can someone give me an simple example how ot use ff or bigmemory?or
>> >> > maybe
>> >> > re
>> >> > better sollution?
>> >> >
>> >> >
>> >> >
>> >> > Thank you in advance,
>> >> >
>> >> >
>> >> > Edwin
>> >> >
>> >> > ______________________________________________
>> >> > R-help at r-project.org mailing list
>> >> > https://stat.ethz.ch/mailman/listinfo/r-help
>> >> > PLEASE do read the posting guide
>> >> > http://www.R-project.org/posting-guide.html
>> >> > and provide commented, minimal, self-contained, reproducible code.
>
>
>