[R-sig-Geo] slow computation progress for calc function

[Sara Shaeri]

Dear Ákos, 
I tried it on a small partition of my data. However, the processing time is the same :(. Any other suggestion? 

-------------------------
Sara Shaeri KarimiPhD Candidate, Department of Environmental SciencesFaculty of Science and EngineeringMacquarie University, NSW 2109 Australia 
Researchgate
Linkedin 





 

    On Tuesday, June 25, 2019, 4:33:34 PM GMT+10, Bede-Fazekas Ákos <bfalevlist using gmail.com> wrote:  
 
 Dear Sara,

it is faster if you first convert the integer vector to logical, and 
then run rle().

set.seed(12345)
random_sample <- sample.int(n = 10, size = 1e5, replace = TRUE) - 1

original <- function(x){
     y <- rle(x)
     return(max(y$lengths[y$values == 0]))
}

faster <- function(x){
     y <- rle(x == 0)
     return(max(y$lengths[y$values]))
}

original(random_sample) == faster(random_sample)

library(microbenchmark)
microbenchmark(
     original(random_sample),
     faster(random_sample)
)

Hence, this may be faster:
interflood <- clusterR(all_predictions, calc, args=list(function(x){y <- 
rle(as.numeric(x) == 0; return(max(y$lengths[y$values]))}))

HTH,
Ákos Bede-Fazekas
Hungarian Academy of Sciences

2019.06.25. 3:32 keltezéssel, Sara Shaeri via R-sig-Geo írta:
> Dear community,
> I’m trying to use the Calc function in a raster stack (8000 binary format rasters). Each raster covers 2,421,090 cells, as such, I’m using parallel coding to make the most of the available cores for this computation, however, this process is extremely slow on a 36 core and 76 Gb RAM. How could I speed up the calculations? This is the code I’m using:
>
> beginCluster(30)
>
> interflood <- clusterR(all_predictions, calc, args=list(function(x){y <- rle(as.numeric(x));return(max( y$lengths[y$values == 0]))}))
>
> endCluster()
>
>
> RegardsSara
>
>
> -------------------------
> Sara Shaeri KarimiPhD Candidate, Department of Environmental SciencesFaculty of Science and EngineeringMacquarie University, NSW 2109 Australia
> Researchgate
> Linkedin
>
>
>
>
>
>
>     [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo using r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
  
	[[alternative HTML version deleted]]