[R-sig-Geo] issue in using projectRaster in raster library

Jonathan Greenberg jgrn at illinois.edu
Mon Apr 7 22:42:58 CEST 2014


Ping:

Re: maxmemory -- you really shouldn't adjust that, definitely not to
what you set it to -- you've set it to (by my quick calculations) to
be around 7500 GB of needed memory (this is assuming each cell takes
64 bits of space) -- this is probably why you are crashing.  Chunksize
also is usually set correctly, raster processes generally have a
quickly diminishing returns as you increase the chunk size.

You might want to try gdalwarp in my gdalUtils package -- it will be
faster to reproject, and you can even enable parallelized warping.

install.packages("gdalUtils",
repos="http://R-Forge.R-project.org",type="source") # To get the
latest, otherwise just use CRAN

--j

On Mon, Apr 7, 2014 at 3:30 PM, Alex Zvoleff <azvoleff at conservation.org> wrote:
> On Mon, Apr 7, 2014 at 4:05 PM, ping yang <pingyang.whu at gmail.com> wrote:
>
>> Dear r-sig-geo,
>>
>> I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
>> (large) using a R script(I did it successfully for the 30 meter DEM), I
>> used the following code to run:
>>
>> rasterOptions(chunksize = 1e+10, maxmemory = 1e+12) #need to allow bigger
>> chunksize for the processing!!!
>>
>
> Why do you find that you "need to allow bigger chunksize"? I rarely need to
> adjust the chunksize, even for large images.
>
>
>>  #*----Function for detect the UTM Zone-----#
>> long2UTM <- function(long) {
>>   (floor((long + 180)/6) %% 60) + 1
>> }
>>
>>   require(raster)
>>   require(rgdal)
>>   filename <- "grdn41w083_13"
>>   longitude <- paste("-",substr(filename,9,10),sep="")
>>   lon <- as.numeric(longitude)
>>   utm <-paste("+proj=utm +zone=",long2UTM(lon)," +datum=NAD83 +ellps=GRS80
>> +towgs84=0,0,0 +units=m +no_defs",sep="")
>>   r  <- raster(filename)
>>   outputfile <- paste("projected/",substr(filename,4,10),".tif",sep='')
>>   print(paste(outputfile))
>>   pr <- projectRaster(r, crs=utm, res=10.0, method='bilinear')
>>
>> However, the projectRaster takes forever to finish and it almost crashed my
>> computer.
>>
>> The file is kind of big  (file size is 455MB),The file information is
>> class       : RasterLayer
>> dimensions  : 10812, 10812, 116899344  (nrow, ncol, ncell)
>> resolution  : 9.259259e-05, 9.259259e-05  (x, y)
>> extent      : -83.00056, -81.99944, 39.99944, 41.00056  (xmin, xmax, ymin,
>> ymax)
>> coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs
>> data source : c:\temporary\10M\grdn41w083_13
>> names       : grdn41w083_13
>> values      : 212.254, 460.0799  (min, max)
>>
>> My R information:
>> sessionInfo()
>> R version 3.0.1 (2013-05-16)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>> other attached packages:
>> [1] rgdal_0.8-10  raster_2.1-49 sp_1.0-11
>> loaded via a namespace (and not attached):
>> [1] grid_3.0.1      lattice_0.20-27 tools_3.0.1
>>
>> However, this process only took 10 seconds using the existing tool in
>> ArcGIS toolbox. Has someone experienced this before? Any comments will be
>> welcome.
>>
>
> Are you using "Project" or "Define Projection" in ArcGIS? "Define
> projection" just overwrites the existing projection rather than
> reprojecting the data, which is much more calculation intensive.
>
>
>>
>> Thanks,
>>
>> Ping
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
> --
> Alex Zvoleff
> Postdoctoral Associate
> Tropical Ecology Assessment and Monitoring (TEAM) Network
> Conservation International
> 2011 Crystal Dr. Suite 500, Arlington, Virginia 22202, USA
> Tel: +1-703-341-2749, Fax: +1-703-979-0953, Skype: azvoleff
> http://www.teamnetwork.org | http://www.conservation.org
>
>
> On Mon, Apr 7, 2014 at 4:05 PM, ping yang <pingyang.whu at gmail.com> wrote:
>
>> Dear r-sig-geo,
>>
>> I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
>> (large) using a R script(I did it successfully for the 30 meter DEM), I
>> used the following code to run:
>>
>> rasterOptions(chunksize = 1e+10, maxmemory = 1e+12) #need to allow bigger
>> chunksize for the processing!!!
>>  #*----Function for detect the UTM Zone-----#
>> long2UTM <- function(long) {
>>   (floor((long + 180)/6) %% 60) + 1
>> }
>>
>>   require(raster)
>>   require(rgdal)
>>   filename <- "grdn41w083_13"
>>   longitude <- paste("-",substr(filename,9,10),sep="")
>>   lon <- as.numeric(longitude)
>>   utm <-paste("+proj=utm +zone=",long2UTM(lon)," +datum=NAD83 +ellps=GRS80
>> +towgs84=0,0,0 +units=m +no_defs",sep="")
>>   r  <- raster(filename)
>>   outputfile <- paste("projected/",substr(filename,4,10),".tif",sep='')
>>   print(paste(outputfile))
>>   pr <- projectRaster(r, crs=utm, res=10.0, method='bilinear')
>>
>> However, the projectRaster takes forever to finish and it almost crashed my
>> computer.
>>
>> The file is kind of big  (file size is 455MB),The file information is
>> class       : RasterLayer
>> dimensions  : 10812, 10812, 116899344  (nrow, ncol, ncell)
>> resolution  : 9.259259e-05, 9.259259e-05  (x, y)
>> extent      : -83.00056, -81.99944, 39.99944, 41.00056  (xmin, xmax, ymin,
>> ymax)
>> coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs
>> data source : c:\temporary\10M\grdn41w083_13
>> names       : grdn41w083_13
>> values      : 212.254, 460.0799  (min, max)
>>
>> My R information:
>> sessionInfo()
>> R version 3.0.1 (2013-05-16)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>> other attached packages:
>> [1] rgdal_0.8-10  raster_2.1-49 sp_1.0-11
>> loaded via a namespace (and not attached):
>> [1] grid_3.0.1      lattice_0.20-27 tools_3.0.1
>>
>> However, this process only took 10 seconds using the existing tool in
>> ArcGIS toolbox. Has someone experienced this before? Any comments will be
>> welcome.
>>
>> Thanks,
>>
>> Ping
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
>
> --
> Alex Zvoleff
> Postdoctoral Associate
> Tropical Ecology Assessment and Monitoring (TEAM) Network
> Conservation International
> 2011 Crystal Dr. Suite 500, Arlington, Virginia 22202, USA
> Tel: +1-703-341-2749, Fax: +1-703-979-0953, Skype: azvoleff
> http://www.teamnetwork.org | http://www.conservation.org
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo



-- 
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
259 Computing Applications Building, MC-150
605 East Springfield Avenue
Champaign, IL  61820-6371
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007



More information about the R-sig-Geo mailing list