[R-sig-Geo] Using 'raster' package functions with rasterEngine

[Dave Marvin]

Thank you for the comments Jonathan. I coerced the output from my
function to an array, and that got it working. As you said the
dimensions need to be ordered as (col, row). However, as you will see,
there is an issue with the output in parallel mode:

test_input=raster(system.file("external/tahoe_lidar_highesthit.tif",
package="spatial.tools"))
test_input=test_input>2535 #create distinct tree canopy objects.
plot(test_input)

clump_fun=function(raster_layer,...){
     clump_raster=clump(raster_layer,directions=8)
     clump_array=array(getValues(clump_raster),dim=c(dim(raster_layer)[c(2:1)],1))
     return(clump_array)
}

cpus=3
cl=makeCluster(spec = cpus, type = "PSOCK", methods = FALSE)
registerDoParallel(cl)

test_raster_out_a=rasterEngine(raster_layer=test_input,fun=clump_fun,chunk_format="raster",debugmode=FALSE)

stopCluster(cl)

**While this gives me an output, it is quite different from running
the same function in either sequential mode or outside rasterEngine
entirely:

registerDoSEQ()
test_raster_out_b=rasterEngine(raster_layer=test_input,fun=clump_fun,chunk_format="raster",debugmode=FALSE)
test_raster_out_c=clump(test_input,directions=8)

dim(freq(test_raster_out_a)$layer) #parallel output
dim(freq(test_raster_out_b)$layer) #sequential mode output
dim(freq(test_raster_out_c)) #non-rasterEngine output (same as "b")

In fact, the number of clumps varies depending on the number of cores
I use in parallel mode. Any idea what is happening here?

Thank you,
-Dave

----
Dave Marvin | davidcmarvin.org
Postdoctoral Research Fellow
Department of Global Ecology
Carnegie Institution for Science


On Mon, Aug 11, 2014 at 8:55 AM, Jonathan Greenberg <jgrn at illinois.edu> wrote:
> Hi Dave:
>
> If you can provide some code examples that would be easier, but I can give
> you a couple of hints.  First, use debugmode=TRUE when you are testing your
> function -- this will save you a lot of headaches down the line.  R will
> enter its debugmode inside of the rasterEngine function so you can track
> what is happening.
>
> The first issue is that clump only works on raster objects.  rasterEngine,
> in its default mode, sends "chunks" of the image to your function formatted
> as *arrays*, not raster objects.  When you did chunk_format to raster, it
> did, indeed, send the chunk as a small, cropped raster so clump worked, BUT:
> all *outputs* from rasterEngine (at present) MUST be coerced to an array
> that has the same number of columns and rows as the input (in that order).
> I might mod this down the road to allow raster outputs, but for now you'll
> need to make sure the output is of type array.  I suspect you are trying to
> return the chunk from your function as a raster, which is why it isn't
> working.
>
> Again: debugmode=TRUE, and track your function down to just before your
> return the output -- check the type of the to-be-returned object and confirm
> it is an array with dim()[1:2] equal to the input raster.
>
> Good luck!
>
> --j
>
>
> On Thu, Aug 7, 2014 at 12:02 PM, Dave Marvin <dmarvin at carnegiescience.edu>
> wrote:
>>
>> Hi,
>> I am able to set up functions and apply them to a raster dataset in
>> parallel using the rasterEngine function of the spatial.tools package, but
>> only if the function performs just a numerical operation on the raster.
>> When I try to incorporate a function from the raster package, such as
>> clump(), the rasterEngine fails and I get an error:
>>
>> > Error in (function (classes, fdef, mtable): unable to find an inherited
>> method for function ‘clump’ for signature ‘"array"’
>>
>> If I change "chunk_format" to "raster" in the rasterEngine function, then
>> i
>> get an error:
>>
>> > chunk processing units require array vector outputs.  Please check your
>> function.
>>
>> 1) Am I doing doing something wrong, or does this particular function just
>> not work with rasterEngine?
>>
>> 2) If it is the latter, which of the raster package functions are
>> available
>> for use in parallel processing with rasterEngine?
>>
>> Thank you.
>> -Dave
>>
>>         [[alternative HTML version deleted]]
>>
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
>
> --
> Jonathan A. Greenberg, PhD
> Assistant Professor
> Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
> Department of Geography and Geographic Information Science
> University of Illinois at Urbana-Champaign
> 259 Computing Applications Building, MC-150
> 605 East Springfield Avenue
> Champaign, IL  61820-6371
> Phone: 217-300-1924
> http://www.geog.illinois.edu/~jgrn/
> AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007