[R-sig-Geo] Analyzing Large Raster Files in R

[mehajain13]

Dear R-sig-geo users,

I’m using the raster package (mainly the calc function) to calculate new
indices for a large MODIS raster that I am using. The code works perfectly
when run over small rasters (e.g. 100 MB), but I now want to run the process
on a 60 GB raster. My computer has 20 GB of RAM and about 500 GB of storage
space.

What is the most efficient and easiest way to run raster calc functions on
extremely large files? I’ve been reading about how to get past memory and
storage limitations in R, but I would love advice on what the easiest and
fastest option might be based on your experience. Some possibilities based
on my reading are:

1)	Somehow use the ff package with the code I’ve written using the raster
package on the full 60 GB file
2)	Using a form of tiling where I break the 60 GB file into smaller
sub-files and run the raster code on each tile using parallels or possibly
multiple computers. I then will piece back together all of these sub-files
to form one large file when they are all done running.

Here is an example of the code that I need to run over the large file (this
is only 1 of the 15 total steps I will eventually run) and a small example
raster is downloadable using the dropbox link below. 

*Example code:*
library(raster)
library(sp)
library(rgdal)

saevi<-brick('D:/Meha_Files/SAFull/SAFull_0013_Final')

spl60fun<-function(x){
	x[x==255]<-NA
	if (sum(is.na(x))>20)
 	{spline<-rep(NA,302)
	return(spline)}
	else
	if (is.na(x[1]))
	{spline<-rep(NA,302)
	return(spline)}
	else
	if (is.na(x[302]))
	{spline<-rep(NA,302)
	return(spline)}
	else
	{x1<-c(1:length(x))
	y1<-x
	intx<-approx(x1,y1,xout=1:length(x1))$y
	spline<-smooth.spline(intx,df=60)$y
	return(spline)}
	}

sasp<-calc(saevi,spl60fun)
writeRaster(sasp,"sasp",format="GTiff")

*Example raster:*
https://www.dropbox.com/s/hh5cl4r02g7havo/Example_Raster.zip

Thanks!
Meha




--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/Analyzing-Large-Raster-Files-in-R-tp7584531.html
Sent from the R-sig-geo mailing list archive at Nabble.com.