[R] how to import several files every day

[Jim Lemon]

On 04/22/2013 05:36 AM, Martin Lavoie wrote:
> Hi All
>
> I want to import several .dat files every day of the week from the same
> folder.
>
> Let say, on day 1, I have about 100 files in the folder.
>
> By using this code, everything works perfectly (maybe there is a more
> efficient way to do it):
>
> filenames<-list.files(path="pathtofile", full.names=TRUE)
> library(plyr)
> import.list<- llply(filenames, read.table, header=TRUE, sep="",
> na.strings="NA", dec=".", strip.white=TRUE)
> #
> #MERGE and RESHAPE some of the files have different columns
> library(reshape)
> data3<- merge_recurse(import.list)
> #
> At the end of each day I will export the file as a cvs file in a different
> folder.
>
> My question is how can I import on day 2 (day3, day4, etc) the new files in
> the folder but without importing the files already imported from the
> previous days.
>
Hi Martin,
Here is one method:

# first set up a dummy file for the first run to avoid an error
system("echo xxx > old.filenames.tab")
# obviously don't do this again
all.filenames<-list.files(path="pathtofile", full.names=TRUE)
old.filenames<-read.table("old.filenames.tab")
filenames<-all.filenames[!(all.filenames %in% old.filenames)]
write.table(all.filenames,file="old.filenames.tab",row.names=FALSE)
library(plyr)
import.list<- llply(filenames, read.table, header=TRUE, sep="",
na.strings="NA", dec=".", strip.white=TRUE)
...

Jim