[R-sig-Geo] Applying a simple math on a big stack takes a long time. How to improve it?

[Thiago V. dos Santos]

Ops, by bad, I missed one value when writing the simplified script to the list. The correct vector should be:

rad.add.fact  <- c(-64.2618, -65.8048, -60.6386, 
-51.1339, -31.2914, -28.2876) 


Thanks for pointing that out.

This vanishes the warning message but apparently doesn't affect the calculation since it is still VERY slow.

 
Greetings,
--
Thiago V. dos Santos
PhD student
Land and Atmospheric Science
University of Minnesota
http://www.laas.umn.edu/CurrentStudents/MeettheStudents/ThiagodosSantos/index.htm
Phone: (612) 323 9898



On Friday, February 6, 2015 4:55 PM, "Baldwin, Jim -FS" <jbaldwin at fs.fed.us> wrote:
rad.mult.fact has length 6 and rad.add.fact has length 5.  Does that affect the calculations?

Jim



-----Original Message-----
From: R-sig-Geo [mailto:r-sig-geo-bounces at r-project.org] On Behalf Of Thiago V. dos Santos
Sent: Friday, February 06, 2015 2:07 PM
To: R-SIG list
Subject: [R-sig-Geo] Applying a simple math on a big stack takes a long time. How to improve it?

Hi all,

I am processing some GeoTiff rasters with 60 files in total (might be more), which are relatively big: resolution is about 7800 x 7700 at 30m res and total file size is around 120MB (Landsat 8 images). I am trying to do a simple math equation calculation (please see the end of the code) on a stack of six of those images:



# create some artificial data
# this is used within a loop with all files (~60) in the directory

r <- raster(nrows=7801, ncols=7711)
r[] <- runif(ncell(r),0,65000)
s.dn <- stack(r,r,r,r,r,r)

# Define calibration factors
rad.mult.fact <- c(0.012852, 0.013161, 0.012128, 0.010227, 0.006258, 0.001556)

rad.add.fact  <- c(-64.2618, -65.8048, -60.6386, -51.1339, -31.2914)

#convert DN to TOA radiance
s.rad <- (s.dn * rad.mult.fact) + rad.add.fact

#write file
writeRaster(s.rad, filename='teste.tif', format="GTiff", overwrite=TRUE)



The calculation runs extremely slow and takes about 10 minutes on my dual-core system with 8GB RAM. Three 2.8GB files are written on my temp dir during calculations. Finally, by the end I receive a warning:



Warning message:
In (s.img * rad.mult.fact) + rad.add.fact :
number of items to replace is not a multiple of replacement length


I wonder if there is a way to optimise that script? And why calculation is so slow in my case? What is causing the warning message and how to fix it?

Many thanks,
--
Thiago V. dos Santos
PhD student
Land and Atmospheric Science
University of Minnesota
http://www.laas.umn.edu/CurrentStudents/MeettheStudents/ThiagodosSantos/index.htm
Phone: (612) 323 9898

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-geo




This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.