[R-sig-Geo] {Text fixed} Best approach to process a massive ammount of MODIS data

[Thiago Veloso]

Sorry for the two last messages: I have no ideia why the text arrives unformatted and scrambled! Hope it is fixed this time!

Dear R experts,

Recently, I have prepared a comprehensive MODIS imagery database. Ten years of LAI images (50GB of data) encompassing South America were carefully acquired and resampled/reprojected with MRT tool.

In the end, I came up with 8-day composition images like this:

lai200101.tif
lai200109.tif
lai200117.tif
lai200125.tif
        .
        .
        .
lai2011361.tif

following the naming logic of "lai{year}{julian day}.tif". 

As a final step, I need to multiply each image by its respective calibration factor. This can be done in R:

lai[lai>249]<-NA # this eliminates everything which is tagged as non-vegetation
lai_cal<-lai*0.1

After that, I would like to perform monthly integrations and save a netcdf file, something like this:

lai_jan_2001 <- lai200101.tif + lai200109.tif + lai200117.tif + lai200125.tif
lai_feb_2001 <- lai200133.tif + lai200141.tif + lai200149.tif + lai200157.tif
if(require(ncdf)){
writeRaster(lai_jan_2001,filename="/home/thiago/Desktop/lai_jan_2001.nc",format="CDF",overwrite=TRUE)
writeRaster(lai_feb_2001,filename="/home/thiago/Desktop/lai_feb_2001.nc",format="CDF",overwrite=TRUE)
}

and so on, for each month in the 2001~2011 period. I work on a HP Workstation with 6GB RAM and running Ubuntu 11.04 64bits and R 2.15.0. My current attempt to load and process one single image takes long instants:

require(raster)
require(rgdal)
lai<-raster("/home/thiago/MODIS/LAI/200101.tif")
lai[lai>249]<-NA
lai_cal<-lai*0.1
plot(lai) # optional step, but this takes LOTS of seconds to process
#Writing out calibrated image to a netcdf file
if(require(ncdf)){
writeRaster(lai_cal,filename="/home/thiago/Desktop/lai_cal.nc",format="CDF",overwrite=TRUE)
}

Also, I run out of ideas to process all the images in batch. I wonder if anyone can suggest the most effective approach, in terms of reducing both disk usage and time consumption, to process the entire database.

All the best,
Thiago Veloso.