[R-sig-Geo] Intermittent error using clusterR

[Guillaume Drolet]

Hi,

I wrote a code which uses clusterR to run a function on multiple CPUs [see
this thread showing part of the code, that has fixed based on Robert's help:
http://r-sig-geo.2731867.n2.nabble.com/cluster-error-with-clusterR-and-calc-td7585148.html]. 

Intermittently, my code stops with this error (along with its traceback): 

Error in serialize(data, node$con) : error writing to connection 
serialize(data, node$con) 
sendData.SOCKnode(con, list(type = type, data = value, tag = tag)) 
sendData(con, list(type = type, data = value, tag = tag)) 
postNode(con, "EXEC", list(fun = fun, args = args, return = return,  
sendCall(cl[[i]], clusfun, list(fun, i), tag = i) 
clusterR(s, fun1, filename = f) at calculateCloudH.R#33
calculateCloudH(current.h, t.cloudbase, t.cloud, file.cloudH,  at
fmask.R#332
fmask(mtl, filename = ofile, nodes = 8, overwrite = TRUE) 

Sometimes it doesn't stop, some other times it does. It may be similar to
this same problem in which they attribute it to a call to a C function
through .Call()
[http://stackoverflow.com/questions/17015598/error-calling-serialize-r-function].
As in that thread, showConnections() show my connections are still opened
and ready for read/write (but, there are 6 connections instead of 8: maybe
this is due to this call I use when I initialize the cluster,
on.exit(endCluster()), and which didn't do is job completely...?)

Has anyone else ever experienced this same problem with clusterR? If so, I
would appreciate your advice on how you fixed it if you did. Thanks!

>sessionInfo():
R version 3.0.2 (2013-09-25)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=French_Canada.1252  LC_CTYPE=French_Canada.1252   
LC_MONETARY=French_Canada.1252
[4] LC_NUMERIC=C                   LC_TIME=French_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] igraph_0.6.6      snow_0.3-13       fmask_0.1         RSAGA_0.93-6     
plyr_1.8         
 [6] shapefiles_0.7    foreign_0.8-55    gstat_1.0-18      data.table_1.8.10
rgdal_0.8-14     
[11] raster_2.1-68     sp_1.0-14         roxygen2_3.0.0   

loaded via a namespace (and not attached):
 [1] brew_1.0-6       codetools_0.2-8  digest_0.6.3     grid_3.0.2      
intervals_0.14.0 lattice_0.20-23 
 [7] spacetime_1.0-9  stringr_0.6.2    tools_3.0.2      xts_0.9-7       
zoo_1.7-10   




--
View this message in context: http://r-sig-geo.2731867.n2.nabble.com/Intermittent-error-using-clusterR-tp7585360.html
Sent from the R-sig-geo mailing list archive at Nabble.com.