[R-sig-Geo] rgrass7 and snow

[Rainer M Krug]

"Dr Didier G. Leibovici" <didier.leibovici at nottingham.ac.uk> writes:

> On 22/04/2016 16:21, Rainer M Krug wrote:
>
>
>  Le vendredi 22 avril 2016, Dr Didier G. Leibovici <didier.leibovici at nottingham.ac.uk> a écrit :
>
>
>  Hi,
>
>  we are trying to use 'grass' in parallel programming with 'snow'
>
>  The code does multiple simulations of a r.viewshed resampling the points
>  generating the viewshed.
>  (I guess the other solution would be to have an equivalent of r.viewshed
>  in an R library or script) ...
>  I think the problems are to do with the gisDbase ...
>
>  Testing with 2 clusters we start with:
>  > clusterCall(cl,initGRASS,"/usr/lib/grass70",home=getwd(),
>  gisDbase="GRASS_TEMP", override=TRUE )
>  [[1]]
>  gisdbase GRASS_TEMP
>  location file5edb6bf06d70
>  mapset file5edb4b0cda81
>  rows 1
>  columns 1
>  north 1
>  south 0
>  west 0
>  east 1
>  nsres 1
>  ewres 1
>  projection NA
>
>  [[2]]
>  gisdbase GRASS_TEMP
>  location file5edb6bf06d70
>  mapset file5edb4b0cda81
>
>  then read a DEM
>  clusterCall(cl,execGRASS,"r.in.gdal", flags="o",
>  parameters=list(input=baseDemFilename, output="DEM"))
>  clusterCall(cl,execGRASS, "g.region", parameters=list(raster="DEM"))
>
>  and then loop on the sampled points ... involving
>  execGRASS("r.viewshed", parameters = list(input = "DEM", output =
>  "cumulativeViewshed", max_distance=maxDistance, coordinates =
>  as.integer(coords[i,])), flags = c("overwrite" , "b","quiet"))
>
>  and cumulating the viewshed (reading the ouput using
>  readRAST("cumulativeViewshed")), all this (loop over the points)within a
>  function simul() called
>  by a
>  parSapply(cl,1:nDsimul,simul)
>
>  Here is the error
>  Error in checkForRemoteErrors(val) :
>  2 nodes produced errors; first error: no such file:
>  GRASS_TEMP/file5edb6bf06d70/file5edb4b0cda81/.tmp/geoprocessing/cumulativeViewshed
>  Calls: parSapply ... clusterApply -> staticClusterApply ->
>  checkForRemoteErrors
>
>  any idea?
>
>  The problem is likely that you are working in parallel in the same map set. I would suggest to use a separate Mauser for each parallel task to write to,
>  read from a different map set which no parallel task is writing to, and finally, after all threads are finished, you can collect the results from each thread
>  in one map set and delete the temporary map sets. 
>
>
>  Cheers, 
>
>
>  Rainer 
>
>
> Yes all the problem is there for each cluster to work on a separate GRASS_TEMP
> ( we have verified that it was running with I cl only!)
> Can we generate randomly the location or mapset so they will not collapse onto the same?

I used a setup for parallel processing (simulating spread under
different scenarios), where I had

1) one locaction in which everything happened, which contained:
2) one mapset which was read-only for each parallel task
3) one mapset for each task in which the results were written; this was
a folder, which was created for each parallel task
4) one mapset into which the individual simulation tasks were analysed into.

So finally, I had a mapset for each simulation, and one in which the
analysis was.

So yes, why shouldn't you be able to create a mapset per task? You can
do it during the initialization by using GRASS.

Hope this helps,

Rainer



>
> DIdier
>
>
>  thanks,
>
>  DIdier
>
>  --
>  Dr Didier G. Leibovici
>  d-d'ye ley-bow-v-c
>  Senior Research Fellow
>  Geocomputational Modelling & Geospatial Statistics
>  Nottingham Geospatial Institute
>  University of Nottingham, UK
>  +44 (0)115 84 13924
>  http://www.nottingham.ac.uk/ngi/people/didier.leibovici
>  Google+ didier.leibovici at gmail.com
>  Skype didierleibovici
>
>  [[alternative HTML version deleted]]
>
>  _______________________________________________
>  R-sig-Geo mailing list
>  R-sig-Geo at r-project.org
>  https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
>
>  -- 
>  Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)
>
>  Centre of Excellence for Invasion Biology
>  Stellenbosch University
>  South Africa
>
>  Tel : +33 - (0)9 53 10 27 44
>  Cell: +33 - (0)6 85 62 59 98
>  Fax (F): +33 - (0)9 58 10 27 44
>
>  Fax (D): +49 - (0)3 21 21 25 22 44
>
>  email: Rainer at krugs.de
>
>  Skype: RMkrug

-- 
Rainer M. Krug
email: Rainer<at>krugs<dot>de
PGP: 0x0F52F982
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 454 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-geo/attachments/20160422/4a387e4b/attachment.bin>