[R-sig-Geo] Help on slow processing of extract fuction

[Roger Bivand]

On Mon, 21 Feb 2011, Rahul Raj wrote:

> Dear All,
>
> I am using the "extract" command to extract the fraction of each class in
> the raster cells, falling in each grid of shape file of polygon type.
>> extract(r,v) ### where 'v' is my polygon grid of 1 km resolution and 'r'
> is my classified raster image.
>
> My polygon grid has more than 2 million polygons and I am running the
> command in 64 bit window xp operating system having quard 4 processor and 8
> GB RAM. But the extract function is taking a long time (more than 2 days) to
> generate results.
> please help me how can I speed up the processing time. Is there any other
> packages in R that can provide the fast processing for this type of
> extraction.
>
> Also, how can I read one polygon at a time in R from the polygon shape file
> stored in disk.

Please, do try to think carefully before beginning work. What are you 
actually doing? Are your polygon objects really irregular multipolygons, 
or are they rectangles? Do they overlap? If I recall, you are looping over 
the polygons - did you time a single polygon and say 100 polygons taken 
together? You are trying to use R/raster/sp for tasks that are very 
different from those they were designed for, in a very demanding way 
(loops are known to be inferior to vectorisation in R).

So you run extract() on your whole raster for each polygon, which 
internally crops your raster to the polygon, rasterises it, turns it into 
points, and extracts the values at those rasterised points from the 
raster. In large scale applications like yours, almost certainly one 
should use different data and process models.

Doing this 2 million times is not efficient, as you are experiencing. I 
don't know why you thought it would be. How to make it more efficient will 
depend on your data structures, and possibly on coding in R, or R and C. 
extract() will work perfectly well on your raster with say 500 polygons, 
but does not seem to have been written for very many polygons.

Roger

>
> Thanks in advance for the valuable suggestions.
>
> With regards
> Rahul Raj
> Indian Institute of Remote Sensing, india.
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no