[Rd] if(--as-cran)?
Duncan Murdoch
murdoch.duncan at gmail.com
Thu Sep 6 13:41:06 CEST 2012
On 12-09-05 9:51 PM, Kasper Daniel Hansen wrote:
> Duncan,
>
> you are right that from the perspective of a single developer: it is
> entirely possible to make tests conditional and ensure that 'long'
> tests does not get run by CRAN.
>
> What I, and several others, advocate is a uniform set of conventions,
> so we avoid having multiple package specific solutions.
>
> Martin has explained one reason for why using _R_CHECK_TIMINGS_ is not
> ideal (the fact that we - at least I and Martin - are most interested
> in the timings for the long tests).
You do realize that his objections were wrong, don't you?
Another reason is that I (and I
> suspect most others) would prefer something that looks a bit more
> official and supported.
The _R_CHECK_TIMINGS_ environment variable is documented and official.
High-jacking an environment variable set for
> (what appears to be) a different reason does not seem like something
> that could be considered supported long-term, and which have a uniform
> interface (across packages).
Using it to detect reports about long-running examples is exactly what
it is designed for. It is not "high-jacking".
>
> Of course, adding an extra environment variable or an extra function
> (as suggested by Henrik) is additional work, and I know it is easy for
> us to suggest things. My perspective is that I would like something
> that will continue to be supported in the future, and where I can use
> the same interface across the packages I maintain and others outside
> my control that I occasionally test. Testing is not just being done
> by the package author(s) or CRAN. I suspect many packages already
> have half-baked solutions to the 'long' test vs 'short' test issue,
> and I don't think we will see any movement toward a unified interface
> without support in base R.
If Martin or I added a new environment variable _R_CHECK_LEVEL_, and
perhaps support code to let you set it in R CMD check or elsewhere, and
support functions to let you read it without the work of calling
Sys.getenv(), and documentation for its purpose -- that doesn't mean it
would be ever be used. I would have no objection whatsoever if he wants
to do that work, but I won't, as I think everyone who would ever use it
has already invented their own method for doing the same thing. It
would waste my time to implement it and would waste their time making
their code compatible with the new system.
> And let me end by saying that I do agree with Martin: having numerical
> levels is better (together with a clear statement what --as-cran sets
> the level to). But I would rather have a binary state than nothing.
It was pointed out to me by a member of CRAN that detecting --as-cran is
insufficient. CRAN only runs incoming tests with settings equivalent to
that. The daily tests use different options. I don't know what those
are, and CRAN is unwilling to discuss their internal policies on this
list. (I would guess they find the tantrums and false claims here to be
frustrating. I do.)
So if Martin wants to implement something, I'd suggest that he
coordinate offline with CRAN.
Duncan Murdoch
>
> Kasper
>
>
> On Wed, Sep 5, 2012 at 9:10 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
>> This post contains two incorrect statements. Since Martin is too busy to
>> post a retraction, I will point them out:
>>
>> --timings and --as-cran do not set _R_CHECK_TIMINGS_ in the same way.
>>
>> As far as I recall, nobody has suggested that package writers be limited to
>> two choices for test suites. I certainly haven't.
>>
>> Duncan Murdoch
>>
>>
>> On 05/09/2012 6:25 AM, Martin Maechler wrote:
>>>
>>>>>>>> Deepayan Sarkar <deepayan.sarkar at gmail.com>
>>>>>>>> on Wed, 5 Sep 2012 11:49:37 +0530 writes:
>>>
>>> > On Wed, Sep 5, 2012 at 11:41 AM, Duncan Murdoch
>>> > <murdoch.duncan at gmail.com> wrote:
>>> >> On 12-09-04 8:19 PM, Kasper Daniel Hansen wrote:
>>> >>>
>>> >>> On Tue, Sep 4, 2012 at 6:02 PM, Duncan Murdoch
>>> <murdoch.duncan at gmail.com>
>>> >>> wrote:
>>> >>>>
>>> >>>> On 04/09/2012 5:42 PM, Dirk Eddelbuettel wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>> On 4 September 2012 at 17:26, Duncan Murdoch wrote:
>>> >>>>> | On 04/09/2012 5:14 PM, Dirk Eddelbuettel wrote:
>>> >>>>> | > An add-on argument to the already established option
>>> --as-cran may
>>> >>>>> be
>>> >>>>> the
>>> >>>>> | > best.
>>> >>>>> | >
>>> >>>>> | > And to iterate, what bugs me is that for _me_ on _my_
>>> machine
>>> >>>>> developing _my_
>>> >>>>> | > package I have remember how to enable what is now (as per
>>> CRAN's
>>> >>>>> decree)
>>> >>>>> | > "non-standard behaviour" of full testing. I fully agree
>>> with what
>>> >>>>> Terry had
>>> >>>>> | > said: more tests are better (when we develop). I want the
>>> full
>>> >>>>> suite
>>> >>>>> at my
>>> >>>>> | > end; that is after all why we wrote it!
>>> >>>>> |
>>> >>>>> | You don't have to remember that, you need to figure it out
>>> once, write
>>> >>>>> a
>>> >>>>> | script that sets the environment variables that enable it,
>>> and then
>>> >>>>> you
>>> >>>>> | can forget it.
>>> >>>>>
>>> >>>>> "In theory, theory and practice are the same. In practice, they
>>> are
>>> >>>>> not."
>>> >>>>>
>>> >>>>> The main test script long had exactly such a setting; I wrote
>>> what I
>>> >>>>> wrote
>>> >>>>> because it is _still the wrong way around_ and as I happen to
>>> have added
>>> >>>>> to
>>> >>>>> unit tests this weekend _having suffered through precisely this
>>> >>>>> setting_.
>>> >>>>>
>>> >>>>> But we are on different wavelengths here and I evidently do not
>>> get my
>>> >>>>> point
>>> >>>>> across to you. And as you are the one who could make a change
>>> where it
>>> >>>>> matters, I have no choice but to rest my case in frustration.
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> If you want to give up, then give up, but then don't complain
>>> about the
>>> >>>> current behaviour. If you want to fix it, then continue the
>>> discussion.
>>> >>>>
>>> >>>> You're right that we're on different wavelengths. If you want
>>> some tests
>>> >>>> to
>>> >>>> run at home but not on CRAN, then somewhere there has to be a
>>> >>>> conditional.
>>> >>>> I'm suggesting that the conditional should be "if there's a
>>> tight time
>>> >>>> limit, skip this".
>>> >>>>
>>> >>>> I don't remember if this was your suggestion, but someone has
>>> suggested
>>> >>>> "if
>>> >>>> we're running with the --as-cran option, skip this" and others
>>> have
>>> >>>> suggested "if we're running on CRAN, skip this". I don't see
>>> why you
>>> >>>> find
>>> >>>> my suggestion so objectionable. If you want, I'll repeat why I
>>> find the
>>> >>>> other two suggestions objectionable.
>>> >>>
>>> >>>
>>> >>> I agree with Duncan that having an option long/short makes more
>>> sense
>>> >>> than with/without cran, as long as cran sets that option to be
>>> short.
>>> >>
>>> >>>
>>> >>>
>>> >>> I would also prefer a command line switch to R CMD check to an
>>> >>> environment variable, but I'll be very happy with a standardized
>>> >>> environment variable.
>>> >>
>>> >>
>>> >> I honestly don't see the need for a standardized variable. I've
>>> told you
>>> >> how to detect that you are running --as-cran; if that isn't
>>> sufficient
>>> >> information, then you, the package author, need to set up
>>> something more
>>> >> elaborate, and assume that if it's not set up, then someone else
>>> (maybe
>>> >> CRAN) is running the test.
>>>
>>> > So maybe documenting that (_R_CHECK_TIMINGS_) more formally in
>>> R-exts
>>> > would be sufficient?
>>>
>>> > -Deepayan
>>>
>>> yes and no.
>>> As Duncan said very early, --as-cran is just turning on several
>>> already existing options, one of them being the
>>> _R_CHECK_TIMINGS_ -- which you can *also* enable by --timings.
>>>
>>> So, checking for _R_CHECK_TIMINGS_ is *not* checking for
>>> the presence of --as-cran !
>>> .. in the sense that it is necessary but not sufficient.
>>> Also, if I run the "long checks" I may still want to use
>>> --timings. For me as package developer, the timings may be even more
>>> important in the "long" than in the "short" checks case.
>>>
>>> So, one solution that is little work would be that --as-cran
>>> sets an *extra* environment variable that is *only* set by
>>> --as-cran, but by no other command line switch.
>>>
>>> Still a pity that it seems people want to live in a 0/1
>>> setting when it would be very natural to adopt a 0/1/2/3
>>> (or so) one.
>>> It's a bit like prefering verbose = FALSE/TRUE as argument to
>>> an R function where eventually I often found that using a
>>> tracing = 0/1/2 (or then 'verbose' = 0/1/2 which is almost
>>> back compatible to FALSE/TRUE)
>>> was more useful.
>>>
>>>
>>>
>>> >> I asked the CRAN powers-that-be about the possibility of querying
>>> the amount
>>> >> of time remaining before a timeout; since the different platforms
>>> all use
>>> >> different mechanisms to enforce a timeout, that's not really
>>> practical. So
>>> >> the best you could hope for is to know that a timeout is in
>>> effect. Before
>>> >> I wrote any code, I'd need to hear why --as-cran detection isn't
>>> sufficient.
>>>
>>> I agree that *reliable* --as-cran detection solves the OP's
>>> and most other correspondents problem.
>>> But as argued above, _R_CHECK_TIMINGS_ is not sufficient.
>>>
>>> Martin
>>
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
More information about the R-devel
mailing list