[R] Data Frame Search Slow
tmdalbey at gmail.com
Tue Nov 22 20:01:44 CET 2011
So - I promise to write a blog post on this topic and post it somewhere on
the internet once I get to the bottom of this. Basically, the set-up to the
problem is like this:
1. I have a data frame with dim (2547290, 4)
2. I need to make SQL like lookups on the dataframe. I have been using the
following sort of syntax:
a.dataframe[a.dataframe[[column_index]] %in% some_value, ]
3. This process takes quite a lot of time (~2 seconds) on m1.small
instances AMIs (AWS)
So, I hope I can get that look-up/search logic quite a lot faster. I have
heard that using matrices is the way to do it but I haven't found any
resources on performing that sort of operation specifically that have
yielded better results.
Thought, feelings and advice are more than welcome.
View this message in context: http://r.789695.n4.nabble.com/Data-Frame-Search-Slow-tp4096906p4096906.html
Sent from the R help mailing list archive at Nabble.com.
More information about the R-help