[R-sig-Geo] Raster in parallel computing?

[Robert J. Hijmans]

Camilo,

The simplest approach is below, but that is inefficient here, at least
in this example, as this is designed to work for large files
that are processes in chunks. The problem here is not that the files
are large, but that they are many.
This uses beginCluster from the raster package. Normally you would
need to use clusterR, which does not work for resample; but resample
has build-in support.

beginCluster()

res <- list()
#for (i in 1:length(v)) {
for (i in 1:10) { # subset for the example
f <- paste0(path, 'r_', i, '.tif')
res[i] <- resample(v[[i]], s, method='bilinear', filename=f, overwrite=TRUE)
}

# However, if the rasters (or groups) have the same dimensions:
ss <- stack(v)
rs <- resample(v[[i]], s, method='bilinear', filename='test.tif',
overwrite=TRUE)

endCluster()


# Alternatively:


library(doParallel)
library(foreach)
registerDoParallel(cl = detectCores()-2) #choose how many cores
getDoParWorkers()
x <- foreach(i = 1:length(v), .packages = "raster") %dopar% {
f <- paste0(path, 'r_', i, '.tif')
resample(v[[i]], s, method='bilinear', filename=f, overwrite=TRUE)
}

# Or in groups

x <- foreach(i = 1:10, .packages = "raster") %dopar% {
f <- paste0(path, 'r_', i, '.tif')
j <- (i-1)*10+1
d <- stack(v[j:(j+9)])
resample(d, s, method='bilinear', filename=f, overwrite=TRUE)
}





On Wed, Jan 8, 2014 at 10:00 AM, Camilo Mora <cmora at dal.ca> wrote:
> Thank you Roger,
>
> Yes I should have mentioned that I have looked extensively over this
> question. Curiously, Jonathan From "spatial.tools" provided me with some
> advice as his tool at the current time does not run the function "resample".
>
> Thanks again,
>
> Camilo
>
>
>
>
> Quoting Roger Bivand <Roger.Bivand at nhh.no>:
>
>> Did you notice the thread started yesterday that appears to meet your
>> need:
>>
>> https://stat.ethz.ch/pipermail/r-sig-geo/2014-January/020156.html
>>
>> It is always a good idea to look at the list archives, a search on:
>>
>> "list:R-sig-geo raster parallel"
>>
>> gives a number of potentially interesting hits. You could then preface
>> your posting by saying that you have already tried some possible solutions,
>> and would like help with them.
>>
>> Roger
>>
>> On Wed, 8 Jan 2014, Camilo Mora wrote:
>>
>>> Hi everyone,
>>>
>>> I am using the package "raster" to interpolate a large number of rasters
>>> (~1million) of different resolutions to a unique 1degree resolution grid and
>>> wonder if you know if it is possible to do this in parallel computer?.
>>>
>>> My code (example below) works like a charm but it will take 30 days to
>>> process all the rasters. Sadly, the process only uses one core of my
>>> computer. I wonder if there is a way to run this code (example below) in
>>> parallel computer?.
>>>
>>> Thanks,
>>>
>>> Camilo
>>>
>>> ####TEST CODE######
>>> library (raster)
>>>
>>> #creates 3 test rasters
>>> a <- raster(nrow=3, ncol=3)
>>> a[] <- 1:9
>>>
>>> b <- raster(nrow=3, ncol=3)
>>> b[] <- 10:18
>>>
>>> c <- raster(nrow=3, ncol=3)
>>> c[] <- 19:27
>>>
>>> #concatenates the rasters
>>> d<-brick(a,b,c)
>>>
>>> #creates a raster at a different resolution
>>> s <- raster(nrow=10, ncol=10)
>>>
>>> #interpolates data from the brick to the new resolution
>>> s <- resample(d, s, method='bilinear')
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; fax +47 55 95 95 43
>> e-mail: Roger.Bivand at nhh.no
>>
>>
>>
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo