[R-pkg-devel] Using parallel:::sendMaster &c. despite warnings

[Duncan Murdoch]

On 23/07/2021 10:59 a.m., Hugh Parsonage wrote:
> Happily, package parallel is open-source. If you would like to make
> use of the unexported functions, you could copy the source code with
> acknowledgement into your package.

There's a good chance that wouldn't work, because parallel is a base 
package.  Base packages can work with R internals and user-written 
packages can't.

Probably David's best course of action is to submit patches where 
necessary so that he doesn't need ::: access.  The original motivation 
appeared to be avoiding inefficiency in a contributed package; if 
there's a patch that can fix that, it could be available in a few days. 
  If it needs changes to the parallel package (e.g. exposing functions 
that are currently internal), that will take much longer, and needs a 
strong argument why the current API isn't sufficient.

Alternatively, he can go ahead and use :::, but just not expect his 
package to be on CRAN.  There are other ways to distribute packages.

Duncan Murdoch

  Naturally this might be a lot of
> work and risks different behaviour between your package and parallel,
> but neither might be that terrible for you.  And certainly would be
> better than trying to pretend that the functions you want are exported
> from parallel.
> 


> On Fri, 23 Jul 2021 at 23:09, David Norris <david using precisionmethods.guru> wrote:
>>
>> I can confirm that getFromNamespace() does indeed circumvent the WARNING.
>> I might otherwise prefer ':::', however, for making its 'bad intentions' so clear.
>> I was very much aware of the cautionary language under `?readChild`, but wondered whether *in practice* more liberal community policies might be in force pending the maturation of the futureverse.
>> Although the sparse history of the relevant file hardly supports "might-break-at-any-time" alarmism, I do note that the last commit (in May 2020) introduced an argument sendMaster(raw.asis=TRUE) that I am actually using.
>> https://github.com/wch/r-source/commits/trunk/src/library/parallel/R/unix/mcfork.R
>>
>> (Also agree with Yohann that ideally one might support Windows with a warning such as data.table issues regarding my Mac's non-support for OpenMP.)
>>
>> Best, David
>>
>> From: Sebastian Meyer <seb.meyer using fau.de>
>> Organization: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
>> Date: Friday, July 23, 2021 at 8:05 AM
>> To: David Norris <david using precisionmethods.guru>, "r-package-devel using r-project.org" <r-package-devel using r-project.org>
>> Subject: Re: [R-pkg-devel] Using parallel:::sendMaster &c. despite warnings
>>
>> Am 23.07.21 um 13:19 schrieb David Norris:
>> Because parallelized progress reporting in the futureverse.org incurs latencies too great for my application (https://github.com/HenrikBengtsson/progressr/issues/118), I have found it necessary to implement my own progress reporting using some of the non-exported functionality from `parallel`. (I do appreciate that Windows lacks the fork() system call, and will not support this. But am willing to make this an OS_type: unix-only package.)
>>
>> Of course, I get a WARNING for this:
>>
>> ── R CMD check results ──────────────────────────────── precautionary 0.2.6 ────
>> Duration: 6m 41.8s
>>
>> ❯ checking dependencies in R code ... WARNING
>>      Unexported objects imported by ':::' calls:
>>        ‘parallel:::readChild’ ‘parallel:::selectChildren’
>>        ‘parallel:::sendMaster’
>>        See the note in ?`:::` about the use of this operator.
>>        Including base/recommended package(s):
>>        ‘parallel’
>>
>> Is this warning an absolute deal-killer on CRAN? Is there a 'correct' way to do `:::` that avoids the warning altogether?
>>
>> The 'parallel' functions your package intends to access seem to be
>> intentionally unexported. Their help page says: "They are not available
>> on Windows, and not exported from the namespace", and "This is a very
>> low-level interface for expert use only: it not regarded as part of the
>> R API and subject to change without notice."
>>
>> Correspondingly, the CRAN Repository Policy says
>>
>> Also, ::: should not be used to access undocumented/internal objects in base packages (nor should other means of access be employed). Such usages can cause packages to break at any time, even in patched versions of R.
>>
>> which kind of answers both of your questions. The policy thus implicitly
>> advises against using getFromNamespace().
>>
>> Best regards,
>>
>>          Sebastian Meyer
>>
>>
>> Kind regards,
>> David Norris
>>
>> ______________________________________________
>> mailto:R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>>
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>