Thanks lcn,

I will try to read data from different chunks.

Best,

Kevin


On Fri, Apr 26, 2013 at 3:05 PM, lcn <lcn918@gmail.com> wrote:

> Do you really have the need loading all the data into memory?
>
> Mostly for large data set, people would just read a chunk of it for
> developing analysis pipeline, and when that's done, the ready script would
> just iterate through the entire data set. For example, the read.table
> function has 'nrow' and 'skip' parameters to control the reading of data
> chunks.
>
> read.table(file, nrows = -1, skip = 0, ...)
>
> And another tip here is, you can split the large file into smaller ones.
>
>
>
> On Fri, Apr 26, 2013 at 8:09 AM, Kevin Hao <rfans4chemo@gmail.com> wrote:
>
>> Hi all scientists,
>>
>> Recently, I am dealing with big data ( >3G  txt or csv format ) in my
>> desktop (windows 7 - 64 bit version), but I can not read them faster,
>> thought I search from internet. [define colClasses for read.table, cobycol
>> and limma packages I have use them, but it is not so fast].
>>
>> Could you share your methods to read big data to R faster?
>>
>> Though this is an odd question, but we need it really.
>>
>> Any suggest appreciates.
>>
>> Thank you very much.
>>
>>
>> kevin
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>

	[[alternative HTML version deleted]]

