[R] Error in clusterApply(): recursive default argument reference

[Martin Morgan]

Hi Nicolas --

I think your code is assuming that all nodes have access to the same
set of variables. One solution is to write in a more completely
'functional' style

parNullDistribIntersection <- function(g1, g2, perm=1000, cl=cl) {
    n1 = nodes(g1)                      
    parSapply(cl, 1:perm,
              function(i, g1, g2, n1) {
                  nodes(g1) <- sample(n1)
                  numEdges(intersection(g1,g2))
              },
              g1, g2, n1)
}              

Another possibility is to ensure that the relevant variables are
exported to the cluster before parSapply. With both of these you'll
want to pay some attention to the costs of communicating the data to
the nodes, which could easily be overwhelming (Rmpi's version of
parSapply is smarter at doing the data transfer -- log n time rather
than linear time, where n is the number of nodes -- and more flexible
in distributing the work).

Hope that helps,

Martin


nicolas bertin <nbertin at gsc.riken.jp> writes:

> Hi,
>
> I want to compute a distribution of the intersection of a graph and
> 'randomized' graphs induced by the permutations of node labels (to
> preserve the graph topology).
> Since I ll have many permutations to perform, I was thinking of using
> the snow package and in particular "parSapply" to divide the work
> between my 4 CPUs.
> But I get the following error message :
>
> Error in clusterApply(cl, splitList(x, length(cl)), lapply, fun, ...) : 
>         recursive default argument reference
>
> What am i doing wrong ?
>
> Some details about my platform and R version :
> -----------------------------------------------
> $platform
> [1] "x86_64-redhat-linux-gnu"
> $version.string
> [1] "R version 2.4.1 (2006-12-18)"
>
>
> Below is the snippet of code generating the error message :
> -----------------------------------------------------------
> ### libraries ###
> library(graph)
> library(snow)
>
> ### functions ###
> nullDistribIntersection <- function(g1, g2, perm=1000)
> {
>   n1 <- nodes(g1)
>   sapply(1:perm,
> 	  function(i)
> 	  {
>             nodes(g1) <- sample(n1)
> 	    numEdges(intersection(g1,g2))
> 	  }
> 	 )
> }
>
>
> parNullDistribIntersection <- function(g1, g2, perm=1000, cl=cl)
> {
>   n1 <- nodes(g1)
>   parSapply(cl, 1:perm,
> 	  function(i)
> 	  {
>             nodes(g1) <- sample(n1)
> 	    numEdges(intersection(g1,g2))
> 	  }
> 	 )
> }
>
>
> ## initialize the graph and the rand seed
> set.seed(123)
> g <- randomEGraph(LETTERS[1:15], edges = 100)
>
> ## compute a distribution of the intersection of the graph 
> ## and a 'randomized' graph induced by the permutations of
> ## node labels (to preserve the graph topology)
> cat("1CPU sys time:",
>     system.time(
>                  dist <- nullDistribIntersection(g,g)
>                ),
>     "\n"
>     )
>
> cl <- makeCluster(c("localhost", "localhost"), type = "SOCK")
> cat("Cluster sys time:",
>     system.time(
>                  dist <- parNullDistribIntersection(g,g)
>                ),
>     "\n"
>     )
> ----------------------------------------------------------------
>
> Thanks in advance.
>
> Nicolas Bertin 
> GSC / RIKEN Yokohama
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Martin Morgan
Bioconductor / Computational Biology
http://bioconductor.org