[R] Using read.csv.sql() to read in specific columns

[Doran, Harold]

I am dealing with data frames that have thousands of columns and hundreds of thousands of rows and only need a few specific columns from the data. The data take various formats, but normally are tab-delimited.

I have written the following which is working as expected. However, because I’m so new at using sqldf(), just looking for some verification from users that this is in fact efficient and correct in the R-ish sense of the word and generalizable to larger data sets.

Harold

tmp <- data.frame(replicate(50, rnorm(10)))
names(tmp) <- paste('item', 1:50, sep='')
write.table(tmp, 'tmp.txt')
read.csv.sql("tmp.txt", sql = "select item1, item2, item50 from file", sep = ' ')

	[[alternative HTML version deleted]]