[R] Data Frame Search Slow

[TimothyDalbey]

Hey All,

So - I promise to write a blog post on this topic and post it somewhere on
the internet once I get to the bottom of this.  Basically, the set-up to the
problem is like this:

1.  I have a data frame with dim (2547290, 4)
2.  I need to make SQL like lookups on the dataframe.  I have been using the
following sort of syntax:

a.dataframe[a.dataframe[[column_index]] %in% some_value, ]

3.  This process takes quite a lot of time (~2 seconds) on m1.small
instances AMIs (AWS)

So, I hope I can get that look-up/search logic quite a lot faster.  I have
heard that using matrices is the way to do it but I haven't found any
resources on performing that sort of operation specifically that have
yielded better results.  

Thought, feelings and advice are more than welcome.

Best,
TMD

--
View this message in context: http://r.789695.n4.nabble.com/Data-Frame-Search-Slow-tp4096906p4096906.html
Sent from the R help mailing list archive at Nabble.com.