[Rd] declaring package dependencies

[Paul Gilbert]

On 13-09-14 09:04 AM, Duncan Murdoch wrote:
> On 13-09-13 12:00 PM, Dirk Eddelbuettel wrote:
>>
>> On 13 September 2013 at 11:42, Paul Gilbert wrote:
>> | On 13-09-13 11:02 AM, Dirk Eddelbuettel wrote:
>> | > It's not so much Rcpp itself or my 20-ish packages but the fact
>> that we (as
>> | > in the Rcpp authors) now stand behind an API that also has to
>> accomodate
>> | > changes in R CMD check. Case in point is current (unannounced)
>> change that
>> | > makes all Depends: Rcpp become Imports: Rcpp because of the
>> NAMESPACE checks.
>> |
>> | I am a bit confused by this Dirk, so maybe I am missing something. I
>> | think this is still a "Note" in R-devel so you do have some time to
>> make
>> | the change, at least several months, maybe more. It is not quite what I
>> | think of as an "announcement", more like a shot across the bow, but it
>> | is also not "unannounced".
>>
>> One package author [as in user of Rcpp and not an author of it] was
>> told by
>> CRAN this week to change his package and came to me for help -- so in
>> that
>> small way the CRAN "non-communication policy" is already creating more
>> work
>> for me, and makes me look silly as "I don't document what Rcpp-using
>> packages
>> need" as I sadly still lack the time machine or psychic powers to
>> infer what
>> may get changed this weekend.
>>
>> | More importantly, I don't think that the requirement is necessarily to
>> | change Depends: Rcpp to Imports: Rcpp, the requirement is to put
>> | imports(Rcpp) in the NAMESPACE file. I think this is so that the
>> package
>> | continues to work even if the user does something with the search path.
>> | The decision to change Depends: Rcpp to Imports: Rcpp really depends on
>> | whether the package author wants Rcpp functions to be available
>> directly
>>
>> Rcpp is a bit of an odd-ball as you mostly need it at compile-time,
>> and you
>> require very few R-level functions (but there is package
>> initialization etc
>> pp).  We also only about two handful of functions, and those are for
>> functionality not all 135 packages use (eg Modules etc).
>>
>> But the focus here should not be on my hobby package. The focus needs
>> to be
>> on how four CRAN maintainers (who do a boatload of amazing work which is
>> _truly_ appreciated in its thoroughness and reach) could make the life of
>> authors of 4800+ packages easier by communicating and planning a tad
>> more.
>
> Let me paraphrase that:  "The CRAN maintainers do a lot of work, and it
> helps me a lot, but if they only did a little bit more work it would
> help me even more."
>
> I suspect they'd be more receptive to suggestions that had them doing
> less work, not more.

Actually, this is one of the parts that I do not understand. It seems to 
me that it would be a lot less work for CRAN maintainers if the 
implications and necessary changes to packages were explained a bit more 
clearly in a forum like R-devel that many package developers actually 
read regularly. I many not fully understand how much of the response to 
package submission gets done automatically, but I do get the sense that 
there is a fairly large amount of actual human time spent dealing with 
just my submissions alone. If that is representative of all developers, 
then CRAN maintainers don't have time to do much else. (The fact that 
they do much more suggests I may not be representative.)

Two specific points have already been mentioned implicitly. CRAN 
submission testing is often done at a higher/newer level using the 
latest devel version. This results in lots of rejections for things that 
I would fix before submission, if I knew about them. If the tests were 
rolled out with R, and only later incorporated into CRAN submission 
testing, I think there would be a lot less work for the CRAN 
maintainers. (This is ignoring the possibility that CRAN submission is 
really the testing ground for the tests, and to prove the tests requires 
a fair amount of manual involvement. I'm happy to continue contributing 
to this -- I've often felt my many contribution is an endless supply of 
bugs for the checkers to catch.)

The second point is that a facility like R-forge that runs the latest 
checks, on many platforms, is really useful in order to reduce work for 
both package developers and CRAN maintainers. With R-forge broken, the 
implication for additional work for CRAN maintainers seems enormous. But 
even with it working, not all packages are kept on R-forge, and with 
package version dependencies R-forge does not really work. (i.e. I have 
to get new versions of some packages onto CRAN before the new versions 
of other packages will build on R-forge.)  Perhaps the package checking 
part of R-forge should be separated into a pre-submission clearing house 
to which packages are submitted. If they pass checks there then the 
package developer could click on a submit button to do the actual 
submission to CRAN. (Of course there needs to be a mechanism to plead 
for the fact that the test systems do not have needed resources.) 
Something like the daily, but with new pre-release versions of packages 
might actually be better than the R-forge approach, for two reasons. One 
is that package maintainers would only put packages there that they 
think are actually working. (R-forge tries to build my svn copy even 
when I know it is broken.) The second is that it would automatically 
handle the version dependency problem, since package maintainers would 
have the ability to put in place versions that should work together. 
However, this does not need to be run daily. It only needs to be run 
when the checks change, or for a package when the package changes.

Paul

>
> Duncan Murdoch
>
>>
>> | by users without them needing to specifically attach Rcpp. They are
>> | available with Depends but with Imports they are just used
>> internally in
>> | the package.
>> |
>> | So, one of us is confused. Usually it is me.
>>
>> No, no, I usually keep you company.
>>
>> Dirk
>>
>