[R-pkg-devel] Use of `:::` in a package for code run in a parallel cluster

[Martin Morgan]

At least in the 'parallel' package

library(parallel)
cl = makePSOCKcluster(2)

and because of the nature of the R language, the entire namespace is exported, analogous to

baz <- local({
    foo <- function() 2
    function(...) foo()
})

so making a package function baz available makes all functions in the package available -- a function in the package already has access to other functions in the namespace, whether those functions are exported or not, so there is no need to use :::.

> parSapply(1:2, baz)
[1] 2 2

This is in contrast to what one might expect from exploring things on the command line, where foo is defined in the global environment and, by convention, the global environment is not serialized to the workers

> foo <- function() 1
> bar <- function(...) foo()
> parLapply(cl, 1:2, bar)
Error in checkForRemoteErrors(val) : 
  2 nodes produced errors; first error: could not find function "foo"

Do you really need to use `:::`?

Martin Morgan



On 9/13/20, 3:52 PM, "R-package-devel on behalf of David Kepplinger" <r-package-devel-bounces using r-project.org on behalf of david.kepplinger using gmail.com> wrote:

    Dear list members,

    I submitted an update for my package and got automatically rejected by the
    incoming checks (as expected from my own checks) for using `:::` calls to
    access the package's namespace.
    "There are ::: calls to the package's namespace in its code. A package
    *almost* never needs to use ::: for its own objects:…" (emphasis mine)

    This was a conscious decision on my part as the package runs code on a
    user-supplied parallel cluster and I consider cluster-exporting the
    required functions a no-go as it would potentially overwrite objects in the
    clusters R sessions. The package code does not own the cluster and hence
    the R sessions. Therefore overwriting objects could potentially lead to
    unintended behaviour which is opaque to the user and difficult to debug.

    Another solution to circumvent the R CMD check note is to export the
    functions to the public namespace but mark them as internal. This was also
    suggested in another thread on this mailing list (c.f. "Etiquette for
    package submissions that do not automatically pass checks?"). I do not
    agree with this work-around as the methods are indeed internal and should
    never be used by users. Exporting truly internal functions for the sake of
    satisfying R CMD check is a bad argument, in particular if there is a
    clean, well-documented, solution by using `:::`.

    I argue `:::` is the only clean solution to this problem and no dirty
    work-arounds are necessary. This is a prime example of where `:::` is
    actually useful and needed inside a package. If the R community disagrees,
    I think R CMD check should at least emit a WARNING instead of a NOTE and
    elaborate on the problem and accepted work-arounds in "Writing R
    extensions". Or keep emitting a NOTE but listing those nebulous reasons
    where `:::` would be tolerated inside a package. Having more transparent
    criteria for submitting to CRAN would be really helpful to the entire R
    community and probably also reduce the traffic on this mailing list.

    Best,
    David

    	[[alternative HTML version deleted]]

    ______________________________________________
    R-package-devel using r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/r-package-devel