<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div> </div>
<div>
<div>Thank you very much for the reply.<br/>
<br/>
It's exactly that. My three main folders are "Max_T", Min_T "and" PCP "(for precipitation), they have subfolders per year, with 366 raster files per year. My rasters all have the same structures.<br/>
Thanks for all these elementse, I'll try that.<br/>
<br/>
Having an identical structure for my 30 years and my 3 variables, i will also try the second method. But it will surely be longer because of my memory.</div>
<div>From what i understand, i have to create my empty output array before extract.</div>
<div> </div>
<div>
<div> </div>
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Envoyé:</b> jeudi 17 septembre 2020 à 13:57<br/>
<b>De:</b> "Bede-Fazekas Ákos" <bfalevlist@gmail.com><br/>
<b>À:</b> r-sig-geo@r-project.org<br/>
<b>Objet:</b> Re: [R-sig-Geo] Execute Extract function on several Raster with points layer</div>
<div name="quoted-content">Hello Gaëtan,<br/>
<br/>
so as far as I understand, you have 3 main folders:<br/>
"Max_T", ? and ?<br/>
and in alll the three folders, there are subfolders<br/>
"1961", "1962", ... "1970"<br/>
In each folder, there are 366 raster files, for which the file naming<br/>
conventions are not known by us, but some of the files are called<br/>
"max1961_1.asc", "max1961_2.asc", ... "max1961_366.asc" (in case of<br/>
T_max and year 1961)</div>
<div name="quoted-content"> </div>
<div name="quoted-content"><br/>
In this case, the 10980 layer that belongs to T_max can be read to one<br/>
large RasterStack in this way:<br/>
tmax_filenames <- c(outer(X = as.character(1:366), Y =<br/>
as.character(1961:1970), FUN = function(doy, year) paste0("N:/400074<br/>
Conservation des sols et CC/Data/Climate data/Climate-10km/Max_T/",<br/>
year, "/max", year, "_", doy, ".asc")))<br/>
tmax_raster <- stack(tmax_filenames)</div>
<div name="quoted-content"><br/>
You can give self-explanatory names to the raster layers:<br/>
names(tmax_raster) <- c(outer(X = as.character(1:366), Y =<br/>
as.character(1961:1970), FUN = function(doy, year) paste0(year, "_", doy)))<br/>
<br/>
But if the structure of the rasters are the same (i.e. the cell size,<br/>
extent, projection), then I recommend you to do the raster-vector<br/>
overlay once, save the cell numbers that you are interested in, and then<br/>
in nested for loops (one loop for the climate variable, one for the year<br/>
and one for the day) read the rasters one-by-one, extract the values<br/>
according to the cell numbers, and save the result in a previously<br/>
created data.frame. In this way, you may not encounter memory issues.<br/>
Although, it will take a lot of time...<br/>
<br/>
HTH,<br/>
Ákos Bede-Fazekas<br/>
Hungarian Academy of Sciences<br/>
<br/>
2020.09.17. 19:28 keltezéssel, Gaetan Martinelli írta:<br/>
> Hello everyone, R team,<br/>
><br/>
> Sorry in advance for this long message. Your help will be invaluable.<br/>
><br/>
> For a few days now i have been blocked to execute a task on R. I will<br/>
> try to synthesize my problem.<br/>
><br/>
> I have several raster. I have an ASCII file for each day of a year<br/>
> with a single band. For 30 years, and for three climatic variables on<br/>
> grid 10km/10km (T_min, T_max, Precipitation). So i have a total around<br/>
> of 32 940 raster files (366days*30years*3variables).<br/>
><br/>
> Also, i have a layer of aroud 1000 points.<br/>
><br/>
> I tried to use the Stack function and then make the intersection for<br/>
> each raster files with my 1000 points.<br/>
> I cannot create an independent matrix for all my files where i applied<br/>
> the "extract" function, to then concatenate all my matrices in order<br/>
> to have a single table.<br/>
><br/>
> I tried this, exemple for 10 years et only T_Max (my files are<br/>
> organized the same for my two other variables) :<br/>
> *#Datapoints*<br/>
> Datapoints<-readOGR(dsn="H:/Inventaire/R/final",<br/>
> layer="Centroid_champs")<br/>
> Datapoints<- spTransform (Datapoints, CRS ("+init=epsg:4326") ) # 1022<br/>
> points in the data<br/>
> st_crs(Datapoints)<br/>
> *#Rasters files*<br/>
> folders = list(<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1961'),<br/>
> #Each year includes daily data, the names of my several raster is<br/>
> "max1961_1", "max1961_10", "max1961_100", etc...<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1962'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1963'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1964'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1965'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1966'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1967'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1968'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1969'),<br/>
> file.path('N:','Data','Climate data','Climate-10km','Max_T','1970')<br/>
> )<br/>
> files = unlist(sapply(folders, function(folder) {<br/>
> list.files(folder, full.names=TRUE)<br/>
> }))<br/>
> files<br/>
><br/>
> MET <- lapply(files, raster)<br/>
> s <- raster::stack(MET)<br/>
><br/>
> output <- list()<br/>
> for(i in 1:length(MET)){<br/>
> output[[i]] <- extract(s, Datapoints)<br/>
> names(output)[[i]] <- paste("Année", MET[i], sep = "_")<br/>
> }<br/>
> Also, i tried that :<br/>
> p1 <- 1022 (ID of my DataPoints) ; p2 <- 1 (column where there are the<br/>
> values extracted from my raster) ; p3 <- 3660 # 3660matrix (366<br/>
> day* 10 years)<br/>
> matlist <- list(array(NA,c(p1,p2,p3))) # doing a list of independant<br/>
> matrix<br/>
><br/>
> for(i in seq_along(MET)){<br/>
><br/>
> matlist[[i]] <- extract(s, Datapoints)<br/>
> }<br/>
> But, nothing works...<br/>
> I would like my script to perform these actions :<br/>
> - For each Raster in my Rasterstack, extract the climatic data values<br/>
> and link them to my "Datapoints",<br/>
> - Take the name of my file, take the first three characters of the<br/>
> name to get a column of my weather variable, here, "T_Max" (column<br/>
> with my raster values) ; Take the following four characters then<br/>
> report this information in a new column "Year", and finally, take the<br/>
> last characters of the file name to create a new column "Day".<br/>
> - Concatenate all the independent output matrices corresponding to<br/>
> each intersection made with my different raster files<br/>
> In the end, I would have a huge table, but one that will allow me to<br/>
> do my analysis :<br/>
> Table with 9 attributes (6 attributs of my points + Year + Day +<br/>
> T_Max) like this :<br/>
> ID Datapoint Year Day T_Max<br/>
> 1 1960 1<br/>
> 2 1960 1<br/>
> …... 1960 1<br/>
> 1022 1960 1<br/>
> 1 1960 2<br/>
> 2 1960 2<br/>
> …... 1960 2<br/>
> 1022 1960 2<br/>
> ….. ….. …..<br/>
> 1 1970 1<br/>
> 2 1970 1<br/>
> …... 1970 1<br/>
> 1022 1970 1<br/>
> 1 1970 2<br/>
> 2 1970 2<br/>
> …... 1970 2<br/>
> 1022 1970 2<br/>
> ….. ….. …..<br/>
><br/>
> Could a loop do this task ?<br/>
><br/>
> I'm sorry, i am gradually learning to manipulate R, but this exercise<br/>
> is more difficult than expected...Please feel free to tell me if my<br/>
> question is inappropriate.<br/>
><br/>
> Thank you very much in advance for your answers. Your help or your<br/>
> comments will be really appreciated.<br/>
><br/>
> Have a good day.<br/>
><br/>
> Gaëtan Martinelli<br/>
> Water and Agriculture research professional in Quebec.<br/>
><br/>
> _______________________________________________<br/>
> R-sig-Geo mailing list<br/>
> R-sig-Geo@r-project.org<br/>
> <a href="https://stat.ethz.ch/mailman/listinfo/r-sig-geo" target="_blank">https://stat.ethz.ch/mailman/listinfo/r-sig-geo</a><br/>
<br/>
<br/>
[[alternative HTML version deleted]]<br/>
<br/>
_______________________________________________<br/>
R-sig-Geo mailing list<br/>
R-sig-Geo@r-project.org<br/>
<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-geo" target="_blank">https://stat.ethz.ch/mailman/listinfo/r-sig-geo</a></div>
</div>
</div>
</div></div></body></html>