[R-sig-Geo] Error when combining raster and foreach packages

[Lukas Lehnert]

Hi Loïc,

thank you for your reply and the suggestion. Unfortunately, using the  cluster 
functions in raster is not an option, because I am developing a package which 
may run with raster data but not necessarily. Therefore, I wanted to 
parallelize the stuff in the function of the package and not the loop over the 
raster image.

Cheers

Lukas


Am Freitag, 26. August 2016, 13:02:33 schrieb Loïc Dutrieux:
> Hi Lukas,
> 
> What does your function do? Can you use the built-in parallel
> functionalities of the raster package instead? See ?beginCluster
> 
> Cheers,
> Loïc
> 
> On 26/08/2016 12:23, Lukas Lehnert via R-sig-Geo wrote:
> > Dear list members,
> > 
> > I tried to process a large GeoTIFF file in a blockwise manner (following
> > the vignette "Writing functions for large raster files").
> > 
> > I then combined the approach with a "foreeach"-loop to parallelize the
> > calculation on the raster data:
> > 
> > library(raster)
> > library(doMC)
> > library(foreach)
> > registerDoMC(3)
> > 
> > ra <- raster("dem.tif")
> > tr <- blockSize(ra)
> > outfile <- "test_2.tif"
> > res <- writeStart(raster(ra), outfile, overwrite = TRUE)
> > for (i in 1:tr$n)
> > {
> > 
> >   v <- getValuesBlock(ra, row=tr$row[i], nrows=tr$nrows[i])
> >   
> >   v2 <- foreach(i=1:length(v), .combine = 'c') %dopar% {
> >   
> >      i*(-1)
> >   
> >   }
> >   res <- writeValues(res, v2, tr$row[i])
> >   cat(round(i/tr$n*100, 2), "%\n")
> > 
> > }
> > res <- writeStop(res)
> > 
> > However, I get errors and warnings like the following ones:
> > ERROR 1: /tmp/Rtmpz6qEOE/xortest_2.tif:Failed to allocate memory for to
> > read TIFF directory (0 elements of 12 bytes each)
> > ERROR 1: TIFFReadDirectory:Failed to read directory at offset 4639232
> > Warning 1: TIFFFetchNormalTag:ASCII value for tag "GeoASCIIParams"
> > contains
> > null byte in value; value incorrectly truncated during reading due to
> > implementation limitations
> > 
> > The errors are different each time I restart the process. If I do not use
> > foreach but a common for-loop, the errors do not occur. I tried several
> > raster files (large ones and small ones) and two different computers, but
> > the problem remains.
> > 
> > Does somebody has any idea, how to solve this problem?
> > 
> > Thank you in advance
> > 
> > Lukas
> > 
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo