On Mon, Feb 21, 2011 at 3:18 AM, Roger Bivand <Roger.Bivand@nhh.no> wrote:

> On Mon, 21 Feb 2011, Rahul Raj wrote:
>
>  Dear All,
>>
>> I am using the "extract" command to extract the fraction of each class in
>> the raster cells, falling in each grid of shape file of polygon type.
>>
>>> extract(r,v) ### where 'v' is my polygon grid of 1 km resolution and 'r'
>>>
>> is my classified raster image.
>>
>> My polygon grid has more than 2 million polygons and I am running the
>> command in 64 bit window xp operating system having quard 4 processor and
>> 8
>> GB RAM. But the extract function is taking a long time (more than 2 days)
>> to
>> generate results.
>> please help me how can I speed up the processing time. Is there any other
>> packages in R that can provide the fast processing for this type of
>> extraction.
>>
>> Also, how can I read one polygon at a time in R from the polygon shape
>> file
>> stored in disk.
>>
>
> Please, do try to think carefully before beginning work. What are you
> actually doing? Are your polygon objects really irregular multipolygons, or
> are they rectangles? Do they overlap? If I recall, you are looping over the
> polygons - did you time a single polygon and say 100 polygons taken
> together? You are trying to use R/raster/sp for tasks that are very
> different from those they were designed for, in a very demanding way (loops
> are known to be inferior to vectorisation in R).
>
> So you run extract() on your whole raster for each polygon, which
> internally crops your raster to the polygon, rasterises it, turns it into
> points, and extracts the values at those rasterised points from the raster.
> In large scale applications like yours, almost certainly one should use
> different data and process models.
>
> Doing this 2 million times is not efficient, as you are experiencing. I
> don't know why you thought it would be. How to make it more efficient will
> depend on your data structures, and possibly on coding in R, or R and C.
> extract() will work perfectly well on your raster with say 500 polygons, but
> does not seem to have been written for very many polygons.
>
> Roger
>
>
>> Thanks in advance for the valuable suggestions.
>>
>> With regards
>> Rahul Raj
>> Indian Institute of Remote Sensing, india.
>>
>>        [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>
> --
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand@nhh.no
>
> Dear Roger

I have a classified image of whole India (India measures 3,214 km (1,997 mi)
from north to south and 2,993 km (1,860 mi) from east to west) at 56m
resolution. I have generate a square grid  of type polygon (area of each
square polygon in grid is 1sq km) that covers whole India. Now I have to
extract the fraction of crop classes from classified image for each square
polygon in the grid.

For this purpose I thought to use the extract command. As the computer
memory can not handle the such a large grid, therefore I divided my grid
according to the state of India (India has 28 states) .I have used the clip
operation of ARC GIS to clip the state wise grid (from my large grid) using
vector boundary of state. At the boundary of state grid (still have 0.1 to 2
million polygons) , I am losing the square shape of polygon.
 I have used rgdal package to import this subset grid (state wise). I am
running the 'extract' command in loop so that the extraction of class
fraction (from classified map) runs on one polygon at a time in each loop (I
did it to avoid the out of memory problem of computer).

I knew by reading your mail that my approach is not appropriate.
I am also getting problems like " Error in gc(): caught access violation -
continue with care", when the looping on  extract function starts.

I am a beginner in R programming and I have joined R sig geo just before few
days. Please help me to short out my problem in handling large grid in R.

Please let me know if still you have doubt in understanding my data.

With regards
Rahul Raj
Indian Institute of Remote Sensing, India.

	[[alternative HTML version deleted]]

