[R-sig-Geo] Array indices for very large multi-band images

[Edzer Pebesma]

Over the weekend I corrected the bug in rgdal that caused the following
error in Guido's email:

sub1 <- x[14000:14199, 14000:14199,1:3]
> Error: cannot allocate vector of size 2.3 Gb

in rgdal 0.6-26 this should be resolved, and only require the amount of
memory needed for the small selection. Sources are committed to
http://sourceforge.net/projects/rgdal/

It might take some time before this new gdal version reaches CRAN, as
rgdal is trying to hit the win64 and mac-osx walls, which is also not
insignificant.
--
Edzer


Robert J. Hijmans wrote:
> Guido,
> 
> Perhaps the function getValuesBlock in 'raster' makes this easier to do:
> 
> library(raster)
> b <- brick(system.file("external/Rlogo.grd", package="raster"))
> getValuesBlock(b, row=1, nrows=3, col=1, ncols=3)
> 
> Robert
> 
> On Fri, Apr 9, 2010 at 3:05 AM,  <guido.lemoine at jrc.ec.europa.eu> wrote:
>> Dears,
>>
>> I just joined the list, so please bear with me if my problem
>> has been dealt with already. I did an extensive google search,
>> making sure r-gis-geo was included, so am sort of sure my
>> problem is relevant.
>>
>> I have a (very) large 3 band ENVI file (32431 x 47262, 4.5 Gb,
>> i.e. too big for normal GeoTIFF). I would like to generate
>> statistics for 200 by 200 sized cells (including cross-correlation
>> between bands).
>> I am using the rgdal package, trying to use the array indexing
>> syntax of R, running into trouble when trying to subset all
>> 3 bands at the same time.
>>
>> Using array index syntax is useful, because it produces a
>> SpatialGridDataFrame, which makes further processing easier.
>>
>> Here's my trial script:
>>
>> require(rgdal)
>> x<-GDAL.open("verylarge3channelENVIimage")
>> r <- getRasterData(x, offset = c(14000, 14000), region.dim = c(200, 200))
>> str(r)
>> # This works, but produces a simple integer array
>> int [1:200, 1:200, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
>> # I'd rather do this
>> sub0 <- x[14000:14199, 14000:14199,]  # ... but doesn't work
>> Error in x[14000:14199, 14000:14199, ] :
>>  argument is missing, with no default
>> # this then?
>> sub1 <- x[14000:14199, 14000:14199,1:3] # no, running into memory trouble
>> Error: cannot allocate vector of size 2.3 Gb
>> # per band works:
>> sub2 <- x[14000:14199, 14000:14199,1]  # produces a nice spatial object.
>> summary(sub2)
>> Object of class SpatialGridDataFrame
>> ...
>>
>> Is there a way to get all 3 bands with array index syntax, preferably
>> as a SpatialGridDataFrame with band1, band2, band3.
>>
>> Any help greatly appreciated,
>>
>> GL
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at stat.math.ethz.ch
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo

-- 
Edzer Pebesma
Institute for Geoinformatics (ifgi), University of Münster
Weseler Straße 253, 48151 Münster, Germany. Phone: +49 251
8333081, Fax: +49 251 8339763  http://ifgi.uni-muenster.de
http://www.52north.org/geostatistics      e.pebesma at wwu.de