[R-sig-Geo] issue in using projectRaster in raster library
Jonathan Greenberg
jgrn at illinois.edu
Mon Apr 7 22:42:58 CEST 2014
Ping:
Re: maxmemory -- you really shouldn't adjust that, definitely not to
what you set it to -- you've set it to (by my quick calculations) to
be around 7500 GB of needed memory (this is assuming each cell takes
64 bits of space) -- this is probably why you are crashing. Chunksize
also is usually set correctly, raster processes generally have a
quickly diminishing returns as you increase the chunk size.
You might want to try gdalwarp in my gdalUtils package -- it will be
faster to reproject, and you can even enable parallelized warping.
install.packages("gdalUtils",
repos="http://R-Forge.R-project.org",type="source") # To get the
latest, otherwise just use CRAN
--j
On Mon, Apr 7, 2014 at 3:30 PM, Alex Zvoleff <azvoleff at conservation.org> wrote:
> On Mon, Apr 7, 2014 at 4:05 PM, ping yang <pingyang.whu at gmail.com> wrote:
>
>> Dear r-sig-geo,
>>
>> I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
>> (large) using a R script(I did it successfully for the 30 meter DEM), I
>> used the following code to run:
>>
>> rasterOptions(chunksize = 1e+10, maxmemory = 1e+12) #need to allow bigger
>> chunksize for the processing!!!
>>
>
> Why do you find that you "need to allow bigger chunksize"? I rarely need to
> adjust the chunksize, even for large images.
>
>
>> #*----Function for detect the UTM Zone-----#
>> long2UTM <- function(long) {
>> (floor((long + 180)/6) %% 60) + 1
>> }
>>
>> require(raster)
>> require(rgdal)
>> filename <- "grdn41w083_13"
>> longitude <- paste("-",substr(filename,9,10),sep="")
>> lon <- as.numeric(longitude)
>> utm <-paste("+proj=utm +zone=",long2UTM(lon)," +datum=NAD83 +ellps=GRS80
>> +towgs84=0,0,0 +units=m +no_defs",sep="")
>> r <- raster(filename)
>> outputfile <- paste("projected/",substr(filename,4,10),".tif",sep='')
>> print(paste(outputfile))
>> pr <- projectRaster(r, crs=utm, res=10.0, method='bilinear')
>>
>> However, the projectRaster takes forever to finish and it almost crashed my
>> computer.
>>
>> The file is kind of big (file size is 455MB),The file information is
>> class : RasterLayer
>> dimensions : 10812, 10812, 116899344 (nrow, ncol, ncell)
>> resolution : 9.259259e-05, 9.259259e-05 (x, y)
>> extent : -83.00056, -81.99944, 39.99944, 41.00056 (xmin, xmax, ymin,
>> ymax)
>> coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs
>> data source : c:\temporary\10M\grdn41w083_13
>> names : grdn41w083_13
>> values : 212.254, 460.0799 (min, max)
>>
>> My R information:
>> sessionInfo()
>> R version 3.0.1 (2013-05-16)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> attached base packages:
>> [1] stats graphics grDevices utils datasets methods base
>> other attached packages:
>> [1] rgdal_0.8-10 raster_2.1-49 sp_1.0-11
>> loaded via a namespace (and not attached):
>> [1] grid_3.0.1 lattice_0.20-27 tools_3.0.1
>>
>> However, this process only took 10 seconds using the existing tool in
>> ArcGIS toolbox. Has someone experienced this before? Any comments will be
>> welcome.
>>
>
> Are you using "Project" or "Define Projection" in ArcGIS? "Define
> projection" just overwrites the existing projection rather than
> reprojecting the data, which is much more calculation intensive.
>
>
>>
>> Thanks,
>>
>> Ping
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
> --
> Alex Zvoleff
> Postdoctoral Associate
> Tropical Ecology Assessment and Monitoring (TEAM) Network
> Conservation International
> 2011 Crystal Dr. Suite 500, Arlington, Virginia 22202, USA
> Tel: +1-703-341-2749, Fax: +1-703-979-0953, Skype: azvoleff
> http://www.teamnetwork.org | http://www.conservation.org
>
>
> On Mon, Apr 7, 2014 at 4:05 PM, ping yang <pingyang.whu at gmail.com> wrote:
>
>> Dear r-sig-geo,
>>
>> I plan to reproject from lonlat to UTM for a bunch of 10 meter NED files
>> (large) using a R script(I did it successfully for the 30 meter DEM), I
>> used the following code to run:
>>
>> rasterOptions(chunksize = 1e+10, maxmemory = 1e+12) #need to allow bigger
>> chunksize for the processing!!!
>> #*----Function for detect the UTM Zone-----#
>> long2UTM <- function(long) {
>> (floor((long + 180)/6) %% 60) + 1
>> }
>>
>> require(raster)
>> require(rgdal)
>> filename <- "grdn41w083_13"
>> longitude <- paste("-",substr(filename,9,10),sep="")
>> lon <- as.numeric(longitude)
>> utm <-paste("+proj=utm +zone=",long2UTM(lon)," +datum=NAD83 +ellps=GRS80
>> +towgs84=0,0,0 +units=m +no_defs",sep="")
>> r <- raster(filename)
>> outputfile <- paste("projected/",substr(filename,4,10),".tif",sep='')
>> print(paste(outputfile))
>> pr <- projectRaster(r, crs=utm, res=10.0, method='bilinear')
>>
>> However, the projectRaster takes forever to finish and it almost crashed my
>> computer.
>>
>> The file is kind of big (file size is 455MB),The file information is
>> class : RasterLayer
>> dimensions : 10812, 10812, 116899344 (nrow, ncol, ncell)
>> resolution : 9.259259e-05, 9.259259e-05 (x, y)
>> extent : -83.00056, -81.99944, 39.99944, 41.00056 (xmin, xmax, ymin,
>> ymax)
>> coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs
>> data source : c:\temporary\10M\grdn41w083_13
>> names : grdn41w083_13
>> values : 212.254, 460.0799 (min, max)
>>
>> My R information:
>> sessionInfo()
>> R version 3.0.1 (2013-05-16)
>> Platform: x86_64-w64-mingw32/x64 (64-bit)
>> attached base packages:
>> [1] stats graphics grDevices utils datasets methods base
>> other attached packages:
>> [1] rgdal_0.8-10 raster_2.1-49 sp_1.0-11
>> loaded via a namespace (and not attached):
>> [1] grid_3.0.1 lattice_0.20-27 tools_3.0.1
>>
>> However, this process only took 10 seconds using the existing tool in
>> ArcGIS toolbox. Has someone experienced this before? Any comments will be
>> welcome.
>>
>> Thanks,
>>
>> Ping
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
>
> --
> Alex Zvoleff
> Postdoctoral Associate
> Tropical Ecology Assessment and Monitoring (TEAM) Network
> Conservation International
> 2011 Crystal Dr. Suite 500, Arlington, Virginia 22202, USA
> Tel: +1-703-341-2749, Fax: +1-703-979-0953, Skype: azvoleff
> http://www.teamnetwork.org | http://www.conservation.org
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
--
Jonathan A. Greenberg, PhD
Assistant Professor
Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
Department of Geography and Geographic Information Science
University of Illinois at Urbana-Champaign
259 Computing Applications Building, MC-150
605 East Springfield Avenue
Champaign, IL 61820-6371
Phone: 217-300-1924
http://www.geog.illinois.edu/~jgrn/
AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
More information about the R-sig-Geo
mailing list