[R-sig-Geo] stackApply and clusterR

[Robert J. Hijmans]

Well, you get the same error when you do:

> stackApply(s, indices=c(1:3), fun=my.f1)
Error in FUN(newX[, i], ...) : unused argument (na.rm = TRUE)

The problem is that the function needs to take an na.rm argument,
perhaps via the dots (which it may ignore). So you can do:

> my.f1 <- function(x, ...) length(!is.na(x))
> stackApply(s, indices=c(1:3), fun=my.f1)
class       : RasterBrick
dimensions  : 10, 10, 100, 3  (nrow, ncol, ncell, nlayers)
resolution  : 0.1, 0.1  (x, y)
extent      : 0, 1, 0, 1  (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : in memory
names       : layer.1, layer.2, layer.3
min values  :       1,       1,       1
max values  :       1,       1,       1




On Wed, Oct 2, 2013 at 4:28 AM, joshgray <joshgray at bu.edu> wrote:
> I'd like to use /stackApply/ with /clusterR/ to parallelize /RasterStack/
> operations. For example, to run some time-series analysis function over the
> "z-dimension" of a RasterStack comprised of many large images. I'd
> previously accomplished this with the /parallel/ and /rgdal/ packages along
> with the /parApply/ function. However, the stacks have gotten a lot bigger
> and I can no longer read the entire stack into memory to perform these
> operations. So, I'd like to take a /raster/ approach.
>
> /clusterR/ says that it supports most functions that take a /Raster*/ object
> as the first argument. Since /stackApply/ conforms to this requirement, I
> thought it'd be easy. However, I can't seem to get this to work. My
> computing environment is a Linux cluster running Sun Grid Engine.
>
> Here's a toy example that tries to count the number of non-NA values in the
> z-dimension of a /RasterStack/. This is just for demo purposes, the actual
> operations are more complex.
>
> require(raster)
> beginCluster(n=8)
>
> my.f1 <- function(x) length(!is.na(x))
>
> set.seed(42)
> x1 <- runif(100)
> x2 <- x1
> x3 <- x1
> x1[sample(1:100, 30)] <- NA
> x2[sample(1:100, 30)] <- NA
> x3[sample(1:100, 30)] <- NA
> r1 <- raster(matrix(x1, nrow=10, ncol=10))
> r2 <- raster(matrix(x2, nrow=10, ncol=10))
> r3 <- raster(matrix(x3, nrow=10, ncol=10))
> s <- stack(r1, r2, r3)
>
> s.out <- clusterR(s, stackApply, args=list(indices=c(1:3), fun=my.f1)) #
> fails
> s.out <- clusterR(s, fun=my.f1) # fails
>
> endCluster()
>
>
> The first attempt at creating /s.out/ fails with this message:
>> s.out <- clusterR(s, stackApply, args=list(indices=c(1:3), fun=my.f1)) #
>> fails
> Error in checkForRemoteErrors(lapply(cl, recvResult)) :
>   6 nodes produced errors; first error: unused argument(s) (na.rm = TRUE)
>
> And then running it again:
>> s.out <- clusterR(s, stackApply, args=list(indices=c(1:3), fun=my.f1)) #
>> fails
> Error in cellFromRowCol(out, tr$row[j], 1) : subscript out of bounds
>
> And once more:
>> s.out <- clusterR(s, stackApply, args=list(indices=c(1:3), fun=my.f1)) #
>> fails
> Error in checkForRemoteErrors(lapply(cl, recvResult)) :
>   one node produced an error: unused argument(s) (na.rm = TRUE)
>
> Any ideas?
>
>
>
>
> --
> View this message in context: http://r-sig-geo.2731867.n2.nabble.com/stackApply-and-clusterR-tp7584760.html
> Sent from the R-sig-geo mailing list archive at Nabble.com.
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo