[R-sig-Geo] How to write functions with mutiple large raster

[Nagle, Nicholas]

Dear Francisco,

I am working with the same time series of MOD09 data, but I leave it as a stack, rather than create the brick.  
Why do you need it as a brick?  The problem with brick is that it needs the data to be in one file.  That's a big file in this instance (Tb scale?).  And, since you didn't specify a filename, R is going to try and do this in memory, and will need to create virtual memory for this.  11 hours doesn't sound unreasonable for this.

Why not just leave it as a stack?  I've found it to work pretty well.  Otherwise, be sure to specify the filename so that R doesn't try to to this in memory.  But that still might take time, and is probably a needless recopy of a large amount of data.

Nicholas



Nicholas Nagle, Assistant Professor
Department of Geography
Burchfiel Geography Building 307
University of Tennessee
Knoxville, TN 37996


> Date: Tue, 7 May 2013 10:35:00 -0400
> From: Francisco Zambrano <frzambra at gmail.com>
> To: r-sig-geo at r-project.org
> Subject: [R-sig-Geo] How to write functions with mutiple large raster
> 	data	file (MOD13Q1, 2000-2012)
> Message-ID:
> 	<CANE=RDO9N-WmnjKTbqnzkeFN=TSfoGxWwyKQo5Jrx4EJSMgfEA at mail.gmail.com>
> Content-Type: text/plain
> 
> Hi everybody,
> 
> I'm working with MODIS data MOD13Q1 and the NDVI index between 2000- today.
> I have been trying to do a function, but I have a problem when stack the
> 310 files and then I tried to use the writeStart function. I had an error
> because the startFuction work with rasterLayer or RasterBrick. But, when I
> tried to make the brick with the rasterStack (>brick(rasterStack)) the
> operation took too much time (11 hours)
> 
> My question, is what is the most efficiently way to make a rasterBrick from
> multiple raster (geoTIFF) files?
> 
> save the rasterStack and then open it with brick?
> 
> My code is something like:
> 
>> out<-stack(list.files(pattern=*..tif$))
>> out<-brick(mod) #too much time (11 hours appox)
> 
>> out<-writeStart(out,filename,overwrite=TRUE)
>> bs<-blockSize(out)
>> pb<-pbCreate(bs$n)
>> for (i in 1:bs$n){
>    v<-getValues(out,row=bs$row[i],nrows=bs$nrows[i])
>    s<-t(sapply(apply(v,1,lowess,f=7/n,iter=10),"[[","y"))
>    out<-writeValues(out,s,bs$row[i])
>   pbStep(pb,i)
>  }
> 
> Kind Regards
> 
> Francisco Zambrano Bigiarini
> 
> from Chile, latin america