[R] Conditional read-in of data

[jim holtman]

That does not seem like a "large" data set.  How are you reading it?
How many columns does it have?  What is "a lot of time" by your
definition?  You have provided minimal data for obtaining help.  I
common read in files with 300K rows in under 30 seconds.  Maybe you
need to consider a relational database for storing your data.

On Wed, Nov 4, 2009 at 12:07 AM, mnstn <pavan.namd at gmail.com> wrote:
>
> Hello All,
> I have a 40k rows long data set that is taking a lot of time to be read-in.
> Is there a way to skip reading even/odd numbered rows or read-in only rows
> that are multiples of, say, 10? This way I get the general trend of the data
> w/o actually reading the entire thing. The option 'skip' in read.table
> simply skips the first n rows and reads the rest. I do understand that once
> the full data set (40k rows) is read-in, I can manipulate the data. But the
> bottle-neck here is the first read/scan of data.
>
> I searched in the forum using key words (conditional skip/skip reading
> rows/skip data/conditional data read) etc. but couldn't find relevant
> conversations. I apologize if this has already been discussed since it does
> seem hard to imagine that nobody has come across this problem yet.
>
> Any suggestions/comments are welcome.
> Thanks,
> mnstn
> --
> View this message in context: http://old.nabble.com/Conditional-read-in-of-data-tp26191091p26191091.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?