[R-sig-hpc] Error in UseMethod("stopCluster") : no applicable method for 'stopCluster' ...
Marius Hofert
marius.hofert at math.ethz.ch
Sat Sep 22 18:41:57 CEST 2012
Dear Martin,
On http://stat.ethz.ch/R-manual/R-patched/NEWS it says
,----
| parallel::stopCluster() now works with MPI clusters without snow
| being on the search path.
`----
so that clarifies it :-)
Cheers,
Marius
Marius Hofert <marius.hofert at math.ethz.ch> writes:
> That's interesting, the same script (with the same call) fails for me. I had to
> put sessionInfo() earlier in the script so that it is printed. Below is the
> output of the script.
>
> I further experimented with it, it runs if I additionally put in
> 'library(snow)'. I thought I don't need that if I use 'parallel' (?). Hmmm...
>
> Cheers,
>
> Marius
>
>> library(parallel)
>>
>> sessionInfo()
> R version 2.15.1 (2012-06-22)
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=C LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] parallel stats graphics grDevices utils datasets methods
> [8] base
>>
>> ex <- function() {
> + cl <- makeCluster(detectCores(), type="MPI")
> + stopCluster(cl)
> + }
>> res <- ex()
> Loading required package: Rmpi
> 4 slaves are spawned successfully. 0 failed.
> Error in NextMethod() : 'NextMethod' called from an anonymous function
> Calls: ex ... stopCluster.spawnedMPIcluster -> <Anonymous> -> NextMethod
> Execution halted
>
>
> Martin Morgan <mtmorgan at fhcrc.org> writes:
>
>> On 09/22/2012 08:52 AM, Marius Hofert wrote:
>>> Dear Martin,
>>>
>>> Thanks for your help. I tried to put in library(parallel) in your example (see
>>> below), but I still obtain
>>>
>>> ,----
>>> | Error in NextMethod() : 'NextMethod' called from an anonymous function
>>> `----
>>>
>>> Do you know why?
>>>
>>> Cheers,
>>>
>>> Marius
>>>
>>> grid <- expand.grid(a=1:4, b=2:10)
>>> ngrid <- nrow(grid)
>>> f <- function(x) sum(x)
>>>
>>> library(parallel)
>>>
>>> ex <- function() {
>>> cl <- makeCluster(detectCores(), type="MPI")
>>> stopCluster(cl)
>>> }
>>>
>>> res <- ex()
>>
>> For the file script.R
>>
>> library(parallel)
>> ex <- function() {
>> cl <- makeCluster(detectCores(), type="MPI")
>> stopCluster(cl)
>> }
>> res <- ex()
>> sessionInfo()
>>
>> The command
>>
>> R --vanilla -f script.R
>>
>> 'works for me' with
>>
>>> sessionInfo()
>> R version 2.15.1 Patched (2012-08-26 r60438)
>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>
>> locale:
>> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>> [7] LC_PAPER=C LC_NAME=C
>> [9] LC_ADDRESS=C LC_TELEPHONE=C
>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>
>> attached base packages:
>> [1] parallel stats graphics grDevices utils datasets methods
>> [8] base
>>
>> other attached packages:
>> [1] Rmpi_0.6-1
>>
>> loaded via a namespace (and not attached):
>> [1] snow_0.3-10
>>
>>>
>>>
>>>
>>> Martin Morgan <mtmorgan at fhcrc.org> writes:
>>>
>>>> On 9/21/2012 12:16 PM, Marius Hofert wrote:
>>>>> Dear Martin,
>>>>>
>>>>> thanks for helping. Is there a solution without explicitly loading snow/parallel
>>>>> with library()/require()? I forgot to say, I was hoping for such a solution.
>>>>
>>>> Not that I know of. Write a package and importFrom(parallel, stopCluster)?
>>>> Martin
>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Marius
>>>>>
>>>>> Martin Morgan <mtmorgan at fhcrc.org> writes:
>>>>>
>>>>>> On 09/21/2012 12:47 AM, Marius Hofert wrote:
>>>>>>> Dear expeRts,
>>>>>>>
>>>>>>> If I run the minimal example below, I obtain:
>>>>>>>
>>>>>>> Loading required package: Rmpi
>>>>>>> Loading required package: grDevices
>>>>>>> Loading required package: grDevices
>>>>>>> Loading required package: grDevices
>>>>>>> Loading required package: grDevices
>>>>>>> 4 slaves are spawned successfully. 0 failed.
>>>>>>> Error in UseMethod("stopCluster") :
>>>>>>> no applicable method for 'stopCluster' applied to an object of class "c('spawnedMPIcluster', 'MPIcluster', 'cluster')"
>>>>>>>
>>>>>>> However, if I run the body of the function (linewise; without calling the
>>>>>>> function), everything works. Why is that so?
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Marius
>>>>>>>
>>>>>>>
>>>>>>> grid <- expand.grid(a=1:4, b=2:10)
>>>>>>> ngrid <- nrow(grid)
>>>>>>> f <- function(x) sum(x)
>>>>>>>
>>>>>>> ex <- function() {
>>>>>>> cl <- snow::makeCluster(parallel::detectCores(), type="MPI")
>>>>>>> on.exit(snow::stopCluster(cl))
>>>>>>> RNGkind("L'Ecuyer-CMRG")
>>>>>>> parallel::clusterExport(cl, varlist=c("f", "grid"))
>>>>>>> parallel::clusterApply(cl, seq_len(ngrid), function(i) f(grid[i,]))
>>>>>>> }
>>>>>>>
>>>>>>> res <- ex()
>>>>>>
>>>>>> An even more minimal example is
>>>>>>
>>>>>> ex <- function() {
>>>>>> cl <- snow::makeCluster(parallel::detectCores(), type="MPI")
>>>>>> snow::stopCluster(cl)
>>>>>> }
>>>>>> res <- ex()
>>>>>>
>>>>>> To be clear, there is no library(snow) in the script, and on.exit() is not
>>>>>> relevant. The reason is that snow::stopCluster is an S3 generic, and on the help
>>>>>> page ?NextMethod we have
>>>>>>
>>>>>>
>>>>>> Namespaces can register methods for generic functions. To support
>>>>>> this, 'UseMethod' and 'NextMethod' search for methods in two
>>>>>> places: first in the environment in which the generic function is
>>>>>> called, and then in the registration data base for the environment
>>>>>> in which the generic is defined (typically a namespace). So
>>>>>> methods for a generic function need to be available in the
>>>>>> environment of the call to the generic, or they must be
>>>>>> registered. (It does not matter whether they are visible in the
>>>>>> environment in which the generic is defined.)
>>>>>>
>>>>>> There are no stopCluster methods defined in the calling namespace (i.e., your
>>>>>> script) and since there has been no call to library(snow), there is no snow
>>>>>> namespace to search and hence no method to be found.
>>>>>>
>>>>>> The solution is to add
>>>>>>
>>>>>> library(snow)
>>>>>>
>>>>>> to the script (or better library(parallel) and use the 'parallel' library
>>>>>> throughout).
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> R-sig-hpc mailing list
>>>>>>> R-sig-hpc at r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>>>>>>
More information about the R-sig-hpc
mailing list