[R] Importing data by targeting in filenames inside a nested list

[Ivan Calandra]

Hi Mario,

It is at the limit of my skills so I'm not sure this will be a real 
solution. But it might help you anyway (and there will be more competent 
people anyway).

What does not make sense to me is why you set the names of your files 
and then use them to list them with list.files()
What don't you just do
electrical_meas_files <- list.files(path, pattern)
and define a pattern if you don't want to read all of them?

For the next step, I don't think you need to lapply() at all; it is 
already in the loop. I usually prefer loops to lapply() because I find 
it more intuitive. But your lapply() solution would work as well.

So I think that your code is somewhat redundant (but I might have missed 
something). This should do it:

#first define your output data list that will be iteratively filled, 
this will increase speed
electrical_meas_raw_data <- vector(mode="list", 
length=length(electrical_meas_files))

#then read the files (seq_along() is great)
for(i in seq_along(measurement_filenames)){
     electrical_meas_raw_data[[i]] <- 
read.xlsx(file=electrical_meas_files[[i]], sheetName="Data", 
header=TRUE, as.data.frame =TRUE, stringsAsFactors = F)
}

HTH,
Ivan

--
Ivan Calandra, PhD
University of Reims Champagne-Ardenne
GEGENAA - EA 3795
CREA - 2 esplanade Roland Garros
51100 Reims, France
+33(0)3 26 77 36 89
ivan.calandra at univ-reims.fr
https://www.researchgate.net/profile/Ivan_Calandra

Le 08/12/15 12:29, BARLAS Marios 247554 a écrit :
> Hello everyone,
>
> So, rookie me is trying to write a smart code, so here's what I'm doing:
>
> I have a list of a couple of hundrend files, some of which refer to different experiments.
> The naming of the file refers to the experiment and the serial number to the topological reference on my sample.
>
> Performing my data analysis in 1 file class at a time looks OK, so I'm trying to generalize my code.
> I figured out I could group them together by performing a pattern read on a nested list, which works for getting the file names and grouping them but then I'm getting some problems when I try to perform the import from the nested list. My code looks like this:
>
> # Vector containing file name patterns to be read and grouped together
> measurement_filenames <- c("*Q_Read_prist*", "*Quasi_Forming*", "*read_set#1*","*Q_Reset_pForm#1*","*read_reset#2*","*quasistatic_set*", "*read_set#2*", "*quasistatic_reset#2*" )
>
> # Create a list of the files to be read in sorted in a natural fashion
> electrical_meas_files <- lapply(measurement_filenames, function(x) naturalsort(list.files(path, pattern=x)))
> names(electrical_meas_files) <- measurement_filenames
>
> # Perform data import for each element
>
> for(i in 1:length(measurement_filenames))
> {
>    electrical_meas_raw_data[[i]] <- lapply(electrical_meas_files[[i]], function(x) read.xlsx(file=x, sheetName="Data", header=TRUE, as.data.frame =TRUE,  stringsAsFactors = F) )
> }
>
>
>
> My idea is to come up with a nested list of the structure
> {list of different experiments}
> 	{list of all sites where the experiment was run}
> 		{set of dataframes with all data for each site}
>
>
> Do I make sense or am I over-complicating the situation ?
>
>
> Any ideas how I could write this piece of code or improve it ?
>
> Thanks in advance,
> Mario
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>