[Rd] if(--as-cran)?
Kasper Daniel Hansen
kasperdanielhansen at gmail.com
Thu Sep 6 03:51:01 CEST 2012
Duncan,
you are right that from the perspective of a single developer: it is
entirely possible to make tests conditional and ensure that 'long'
tests does not get run by CRAN.
What I, and several others, advocate is a uniform set of conventions,
so we avoid having multiple package specific solutions.
Martin has explained one reason for why using _R_CHECK_TIMINGS_ is not
ideal (the fact that we - at least I and Martin - are most interested
in the timings for the long tests). Another reason is that I (and I
suspect most others) would prefer something that looks a bit more
official and supported. High-jacking an environment variable set for
(what appears to be) a different reason does not seem like something
that could be considered supported long-term, and which have a uniform
interface (across packages).
Of course, adding an extra environment variable or an extra function
(as suggested by Henrik) is additional work, and I know it is easy for
us to suggest things. My perspective is that I would like something
that will continue to be supported in the future, and where I can use
the same interface across the packages I maintain and others outside
my control that I occasionally test. Testing is not just being done
by the package author(s) or CRAN. I suspect many packages already
have half-baked solutions to the 'long' test vs 'short' test issue,
and I don't think we will see any movement toward a unified interface
without support in base R.
And let me end by saying that I do agree with Martin: having numerical
levels is better (together with a clear statement what --as-cran sets
the level to). But I would rather have a binary state than nothing.
Kasper
On Wed, Sep 5, 2012 at 9:10 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
> This post contains two incorrect statements. Since Martin is too busy to
> post a retraction, I will point them out:
>
> --timings and --as-cran do not set _R_CHECK_TIMINGS_ in the same way.
>
> As far as I recall, nobody has suggested that package writers be limited to
> two choices for test suites. I certainly haven't.
>
> Duncan Murdoch
>
>
> On 05/09/2012 6:25 AM, Martin Maechler wrote:
>>
>> >>>>> Deepayan Sarkar <deepayan.sarkar at gmail.com>
>> >>>>> on Wed, 5 Sep 2012 11:49:37 +0530 writes:
>>
>> > On Wed, Sep 5, 2012 at 11:41 AM, Duncan Murdoch
>> > <murdoch.duncan at gmail.com> wrote:
>> >> On 12-09-04 8:19 PM, Kasper Daniel Hansen wrote:
>> >>>
>> >>> On Tue, Sep 4, 2012 at 6:02 PM, Duncan Murdoch
>> <murdoch.duncan at gmail.com>
>> >>> wrote:
>> >>>>
>> >>>> On 04/09/2012 5:42 PM, Dirk Eddelbuettel wrote:
>> >>>>>
>> >>>>>
>> >>>>> On 4 September 2012 at 17:26, Duncan Murdoch wrote:
>> >>>>> | On 04/09/2012 5:14 PM, Dirk Eddelbuettel wrote:
>> >>>>> | > An add-on argument to the already established option
>> --as-cran may
>> >>>>> be
>> >>>>> the
>> >>>>> | > best.
>> >>>>> | >
>> >>>>> | > And to iterate, what bugs me is that for _me_ on _my_
>> machine
>> >>>>> developing _my_
>> >>>>> | > package I have remember how to enable what is now (as per
>> CRAN's
>> >>>>> decree)
>> >>>>> | > "non-standard behaviour" of full testing. I fully agree
>> with what
>> >>>>> Terry had
>> >>>>> | > said: more tests are better (when we develop). I want the
>> full
>> >>>>> suite
>> >>>>> at my
>> >>>>> | > end; that is after all why we wrote it!
>> >>>>> |
>> >>>>> | You don't have to remember that, you need to figure it out
>> once, write
>> >>>>> a
>> >>>>> | script that sets the environment variables that enable it,
>> and then
>> >>>>> you
>> >>>>> | can forget it.
>> >>>>>
>> >>>>> "In theory, theory and practice are the same. In practice, they
>> are
>> >>>>> not."
>> >>>>>
>> >>>>> The main test script long had exactly such a setting; I wrote
>> what I
>> >>>>> wrote
>> >>>>> because it is _still the wrong way around_ and as I happen to
>> have added
>> >>>>> to
>> >>>>> unit tests this weekend _having suffered through precisely this
>> >>>>> setting_.
>> >>>>>
>> >>>>> But we are on different wavelengths here and I evidently do not
>> get my
>> >>>>> point
>> >>>>> across to you. And as you are the one who could make a change
>> where it
>> >>>>> matters, I have no choice but to rest my case in frustration.
>> >>>>
>> >>>>
>> >>>>
>> >>>> If you want to give up, then give up, but then don't complain
>> about the
>> >>>> current behaviour. If you want to fix it, then continue the
>> discussion.
>> >>>>
>> >>>> You're right that we're on different wavelengths. If you want
>> some tests
>> >>>> to
>> >>>> run at home but not on CRAN, then somewhere there has to be a
>> >>>> conditional.
>> >>>> I'm suggesting that the conditional should be "if there's a
>> tight time
>> >>>> limit, skip this".
>> >>>>
>> >>>> I don't remember if this was your suggestion, but someone has
>> suggested
>> >>>> "if
>> >>>> we're running with the --as-cran option, skip this" and others
>> have
>> >>>> suggested "if we're running on CRAN, skip this". I don't see
>> why you
>> >>>> find
>> >>>> my suggestion so objectionable. If you want, I'll repeat why I
>> find the
>> >>>> other two suggestions objectionable.
>> >>>
>> >>>
>> >>> I agree with Duncan that having an option long/short makes more
>> sense
>> >>> than with/without cran, as long as cran sets that option to be
>> short.
>> >>
>> >>>
>> >>>
>> >>> I would also prefer a command line switch to R CMD check to an
>> >>> environment variable, but I'll be very happy with a standardized
>> >>> environment variable.
>> >>
>> >>
>> >> I honestly don't see the need for a standardized variable. I've
>> told you
>> >> how to detect that you are running --as-cran; if that isn't
>> sufficient
>> >> information, then you, the package author, need to set up
>> something more
>> >> elaborate, and assume that if it's not set up, then someone else
>> (maybe
>> >> CRAN) is running the test.
>>
>> > So maybe documenting that (_R_CHECK_TIMINGS_) more formally in
>> R-exts
>> > would be sufficient?
>>
>> > -Deepayan
>>
>> yes and no.
>> As Duncan said very early, --as-cran is just turning on several
>> already existing options, one of them being the
>> _R_CHECK_TIMINGS_ -- which you can *also* enable by --timings.
>>
>> So, checking for _R_CHECK_TIMINGS_ is *not* checking for
>> the presence of --as-cran !
>> .. in the sense that it is necessary but not sufficient.
>> Also, if I run the "long checks" I may still want to use
>> --timings. For me as package developer, the timings may be even more
>> important in the "long" than in the "short" checks case.
>>
>> So, one solution that is little work would be that --as-cran
>> sets an *extra* environment variable that is *only* set by
>> --as-cran, but by no other command line switch.
>>
>> Still a pity that it seems people want to live in a 0/1
>> setting when it would be very natural to adopt a 0/1/2/3
>> (or so) one.
>> It's a bit like prefering verbose = FALSE/TRUE as argument to
>> an R function where eventually I often found that using a
>> tracing = 0/1/2 (or then 'verbose' = 0/1/2 which is almost
>> back compatible to FALSE/TRUE)
>> was more useful.
>>
>>
>>
>> >> I asked the CRAN powers-that-be about the possibility of querying
>> the amount
>> >> of time remaining before a timeout; since the different platforms
>> all use
>> >> different mechanisms to enforce a timeout, that's not really
>> practical. So
>> >> the best you could hope for is to know that a timeout is in
>> effect. Before
>> >> I wrote any code, I'd need to hear why --as-cran detection isn't
>> sufficient.
>>
>> I agree that *reliable* --as-cran detection solves the OP's
>> and most other correspondents problem.
>> But as argued above, _R_CHECK_TIMINGS_ is not sufficient.
>>
>> Martin
>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
More information about the R-devel
mailing list