[R-pkg-devel] Use of `:::` in a package for code run in a parallel cluster

Duncan Murdoch murdoch@dunc@n @end|ng |rom gm@||@com
Mon Sep 14 17:49:06 CEST 2020


On 14/09/2020 10:30 a.m., Wang, Zhu wrote:
> In mypkg, I want to call a function foo from pkg,  and foo is not exported. I thought I should use pkg:: or pkg:::, but R CMD check provided a warning.

I'm assuming that mypkg is not the same as pkg; Jeff Newmiller's answer 
assumes the opposite.

In that case where they are different, there is only one circumstance 
where you should be calling foo, and you'll have to do it using 
foo:::pkg.  That circumstance is that you are the maintainer of both 
packages.  You should explain this in your submission message, and ask 
CRAN to ignore the warning if there is one.

The reason for this is the following.  If someone else is maintaining 
pkg, then they are free to change the behaviour of foo without any 
consideration for you, because as an internal function, they have no 
contract with you to maintain its behaviour.  On the other hand, if you 
maintain both packages, then you should be ready to modify mypkg as soon 
as you modify pkg:::foo.

Duncan Murdoch


> 
> Thanks,
> Zhu
> 
>> You don't need either pkg:: or pkg::: if you are calling the function from within the package.  You may need one of those if the call is coming from a user script.
> 
> -----Original Message-----
> From: Duncan Murdoch <murdoch.duncan using gmail.com>
> Sent: Monday, September 14, 2020 7:17 AM
> To: Wang, Zhu <wangz1 using uthscsa.edu>; David Kepplinger <david.kepplinger using gmail.com>; R Package Devel <r-package-devel using r-project.org>
> Subject: Re: [R-pkg-devel] Use of `:::` in a package for code run in a parallel cluster
> 
> On 13/09/2020 8:47 p.m., Wang, Zhu wrote:
>> Apologize if I hijack this thread, but the use of ::: is something I was puzzled.
>>
>> I tried Duncan's solution in my R package mypkg, something like:
>>
>> pkg::callInternal("foo", args)
>>
>> R CMD check mypkg
>>
>> * checking dependencies in R code ... WARNING '::' or ':::' import not
>> declared from: ‘pkg'
>>
>> I probably missed something here.
> 
> You don't need either pkg:: or pkg::: if you are calling the function from within the package.  You may need one of those if the call is coming from a user script.
> 
> If you use pkg:: from mypkg, you need to declare that mypkg Imports pkg.
>    (This is a line in its DESCRIPTION file.)  I think that's what the WARNING is telling you.
> 
> Duncan Murdoch
> 
>>
>> Thanks,
>> Zhu
>>
>> -----Original Message-----
>> From: R-package-devel <r-package-devel-bounces using r-project.org> On
>> Behalf Of Duncan Murdoch
>> Sent: Sunday, September 13, 2020 3:20 PM
>> To: David Kepplinger <david.kepplinger using gmail.com>; R Package Devel
>> <r-package-devel using r-project.org>
>> Subject: Re: [R-pkg-devel] Use of `:::` in a package for code run in a
>> parallel cluster
>>
>> On 13/09/2020 3:51 p.m., David Kepplinger wrote:
>>> Dear list members,
>>>
>>> I submitted an update for my package and got automatically rejected
>>> by the incoming checks (as expected from my own checks) for using
>>> `:::` calls to access the package's namespace.
>>> "There are ::: calls to the package's namespace in its code. A
>>> package
>>> *almost* never needs to use ::: for its own objects:…" (emphasis
>>> mine)
>>>
>>> This was a conscious decision on my part as the package runs code on
>>> a user-supplied parallel cluster and I consider cluster-exporting the
>>> required functions a no-go as it would potentially overwrite objects
>>> in the clusters R sessions. The package code does not own the cluster
>>> and hence the R sessions. Therefore overwriting objects could
>>> potentially lead to unintended behaviour which is opaque to the user and difficult to debug.
>>>
>>> Another solution to circumvent the R CMD check note is to export the
>>> functions to the public namespace but mark them as internal. This was
>>> also suggested in another thread on this mailing list (c.f.
>>> "Etiquette for package submissions that do not automatically pass
>>> checks?"). I do not agree with this work-around as the methods are
>>> indeed internal and should never be used by users. Exporting truly
>>> internal functions for the sake of satisfying R CMD check is a bad
>>> argument, in particular if there is a clean, well-documented,
>>> solution by using `:::`
>>
>> Who is calling this function:  package code or user code?  I assume
>> it's a bit of a mix:  your package writes a script that calls the
>> function when it runs in user space.  (It would help if you gave an
>> explicit example of when you need to use this technique.)
>>
>> If my assumption is correct, there are other simple workarounds
>> besides exporting the functions.  Instead of putting
>>
>>       pkg:::foo(args)
>>
>> into your script, put
>>
>>       pkg::callInternal("foo", args)
>>
>> where pkg::callInternal is an exported function that can look up unexported functions in the namespace.
>>
>> You may argue that you prefer pkg:::foo for some reason:  to which I'd respond that you are being rude to the CRAN volunteers.  I've offered two options (one in the previous thread, a different one here), and there was a third one in that thread offered by Ivan Krylov.  Surely one of these is good enough for your needs, and you shouldn't force CRAN to handle you specially.
>>
>> Duncan
>>
>>>
>>> I argue `:::` is the only clean solution to this problem and no dirty
>>> work-arounds are necessary. This is a prime example of where `:::` is
>>> actually useful and needed inside a package. If the R community
>>> disagrees, I think R CMD check should at least emit a WARNING instead
>>> of a NOTE and elaborate on the problem and accepted work-arounds in
>>> "Writing R extensions". Or keep emitting a NOTE but listing those
>>> nebulous reasons where `:::` would be tolerated inside a package.
>>> Having more transparent criteria for submitting to CRAN would be
>>> really helpful to the entire R community and probably also reduce the traffic on this mailing list.
>>>
>>> Best,
>>> David
>>>
>>> 	[[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> R-package-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>>
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>



More information about the R-package-devel mailing list