[R-sig-Geo] How to reduce the buffering time with the function "buffer" (package raster)

Thu Apr 14 07:37:01 CEST 2016

So you're correct, the buffer() and distance() functions you're using will
take a long period of time.  The code below illustrates the shortcut that
I've used in the past when buffering only by the limit of one pixel, since
it uses queens-case connectance and distance between cell-centers based on
that connectance rule. This uses the gridDistance() function which is much
faster than calculating straight line distances. You can see that the
distance-based calculation takes under 40 seconds. But do notice that the
buffer it creates by setting the threshold is around both classes and is
indiscriminate of the nearest class:

library(raster)

clip_buff <- raster("clip_study_area_2010.tif")

system.time(d <- gridDistance(clip_buff, c(2,3)))
#   user  system elapsed
#  37.36   11.00   48.41

rf[(rf != 2 & rf != 3)] <- NA

## Set buffer to 10m just for fun and assign
## value of 4 to that buffered region:
rf[ d > 0 & d <= 10 ] <- 4
plot(rf)

If you want to buffer by more than one pixel dimension then you'll probably
want straight line distances and I'd suggest using GDAL and using the
gdal_proximity utility via a system() call. Alternatively, you could look
into other utilities that have better-optimized raster-based distance
tools. Hopefully this gives you some ideas on how to proceed?

Sincerely,
Forrest

On Wed, Apr 13, 2016 at 9:18 PM Nelly Reduan <nell.redu at hotmail.fr> wrote:

> I put a cropped raster with 3581676 cells in the link below:
>
>
> https://www.dropbox.com/sh/ewaz4yncdqt7kxj/AAAEHyuWyHzBE7gSINA1oayTa?dl=0
>
> Here is my code to built the buffers:
>
> rf <- clip_buff
>
> rf [(rf != 2 & rf != 3)] <- NA
>
> plot(rf)
>
> system.time(corr <- buffer(rf, 10))
>
> system.time(dist_rf <- distance(rf))
>
>
> I stopped the code with the functions "buffer" and "distance" after two
> hours.
>
>
> Thanks a lot for your time.
>
> Have a nice day.
>
> Nell
>
>
>
>
> ------------------------------
> *De :* Michael Sumner <mdsumner at gmail.com>
> *Envoyé :* mardi 12 avril 2016 20:36
>
> *À :* Nelly Reduan; Forrest Stevens; r-sig-geo at r-project.org
> *Objet :* Re: [R-sig-Geo] How to reduce the buffering time with the
> function "buffer" (package raster)
>
> I think we still haven't seen a print of the raster object?
>
> It may be a tiled file on disk? Try readAll to pull into memory, if it is
> tiled and read in on demand via gdal tools like extract will be very slow
> since access is done line by line.
>
> But just guessing, please give full details and ideally a reproducible
> example.
>
> Cheers, Mije
>
> On Wed, 13 Apr 2016, 02:33 Nelly Reduan <nell.redu at hotmail.fr> wrote:
>
>> Yes, I tested the functions "distance" and "buffer" by using a cropped
>> raster with around 5 million cells. But my original raster has 48096864
>> cells. Ideally, I would like to build the buffers from my original raster.
>> Here are some characteristics of the original raster:
>>
>>
>> class       : RasterLayer
>>
>> dimensions  : 4876, 9864, 48096864  (nrow, ncol, ncell)
>>
>> extent      : 188384.5, 484304.5, 4914481, 5060761  (xmin, xmax, ymin,
>> ymax)
>>
>> coord. ref. : +proj=tmerc +lat_0=0 +lon_0=-73.5 +k=0.9999 +x_0=304800
>> +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs
>>
>>
>> Thanks a lot for your help.
>>
>> Have a nice day.
>>
>> Nell
>>
>>
>>
>> ________________________________
>> De : Forrest Stevens <r-sig-geo at forreststevens.com>
>> Envoyé : lundi 11 avril 2016 18:55
>> À : Nelly Reduan; Forrest Stevens; r-sig-geo at r-project.org
>> Objet : Re: [R-sig-Geo] How to reduce the buffering time with the
>> function "buffer" (package raster)
>>
>> And this is for a raster with around 5 milliion cells? Your results
>> surprise me a bit, especially if the bottleneck is at the call to
>> buffer()/distance().  Your best bet is to wait for a response from Robert
>> Hijmans or someone else working on the package, they might have some other
>> tricks up their sleeves. In any case, I agree, 5+ hours is extremely
>> excessive.
>>
>> Sincerely,
>> Forrest
>>
>> On Mon, Apr 11, 2016 at 6:57 PM Nelly Reduan <nell.redu at hotmail.fr
>> <mailto:nell.redu at hotmail.fr>> wrote:
>>
>> Thank you very much Forrest for your answer. I don't think I have memory
>> problems (my computer has a 32 GB memory).
>>
>>
>> Here is my code to draw the 100-m buffers with the function "buffer"
>> (package raster):
>>
>> r1 <- r ## r is the original raster and has a resolution of 10m
>> r1[(r1[]!=2 & r1[]!=5)]=NA
>> plot(r1)
>> r2 <- buffer(r1, 10)
>>
>>
>> I also tested the function distance (package raster):
>>
>> r1 <- r ## r is the original raster and has a resolution of 10m
>> r1[(r1[]!=2 & r1[]!=5)]=NA
>> plot(r1)
>> r3 <- distance(r1)
>>
>>
>> In the two cases, I stopped the functions afer 5 hours as this was too
>> long ! For the moment, I haven't find solutions.
>>
>>
>> Thanks a lot for your time.
>>
>> Have a nice day.
>>
>> Nell
>>
>>
>>
>> ________________________________
>> De : Forrest Stevens <r-sig-geo at forreststevens.com<mailto:
>> r-sig-geo at forreststevens.com>>
>> Envoyé : lundi 11 avril 2016 09:09
>> À : Nelly Reduan; r-sig-geo at r-project.org<mailto:r-sig-geo at r-project.org>
>> Objet : Re: [R-sig-Geo] How to reduce the buffering time with the
>> function "buffer" (package raster)
>>
>> Five million cells isn't all that many, how slow is too slow? Buffering a
>> raster of about 10 million cells on my laptop takes on the order of 20
>> seconds or so for a binary raster. Is it possible that you're fighting
>> memory problems?
>>
>> In the past when doing multiple ring buffers I've found it faster to
>> calculate a distance-to raster first, and then apply multiple logical
>> comparisons on the distance raster.  I don't know if this applies to your
>> situation or not but it's one trick that might help.
>>
>> Sincerely,
>> Forrest
>>
>> On Mon, Apr 11, 2016 at 10:58 AM Nelly Reduan <nell.redu at hotmail.fr
>> <mailto:nell.redu at hotmail.fr>> wrote:
>>
>> Hello,
>>
>>
>> I would like to build 100-m buffers (representing ecological corridors)
>> around land cover attributes in a raster layer and to assign a given value
>> (i.e., value of 13) to buffer cells.
>>
>>
>> To do this, I'm using the function "buffer" (package raster) to draw the
>> buffers around particular raster cells (i.e., cell values of 2 and 5). For
>> example, in the image below, the buffers are represented in blue (cell
>> values of 13) and the particular raster cells are represented in yellow
>> (cell values of 2).
>>
>>
>> The problem is that the time to run the function "buffer" is very low
>> because I have a raster of 5000000 cells. Does anyone know how I can reduce
>> the buffering time ?
>>
>>
>> Thanks a lot for your time.
>>
>> Have a nice day.
>>
>> Nell
>>
>>
>> [Capture.PNG]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org<mailto:R-sig-Geo at r-project.org>
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
> --
> Dr. Michael Sumner
> Software and Database Engineer
> Australian Antarctic Division
> 203 Channel Highway
> Kingston Tasmania 7050 Australia
>
>

	[[alternative HTML version deleted]]