[Rd] R 3.1.1 and 3.1.2 both fail their test suites

[Duncan Murdoch]

On 05/11/2014, 6:48 AM, Martin Maechler wrote:
>>>>>> Duncan Murdoch <murdoch.duncan at gmail.com>
>>>>>>     on Mon, 3 Nov 2014 06:28:19 -0500 writes:
> 
>     > On 03/11/2014, 4:17 AM, Martin Maechler wrote:
>     >>>>>>> Duncan Murdoch <murdoch.duncan at gmail.com>
>     >>>>>>> on Sat, 1 Nov 2014 13:17:56 -0400 writes:
>     >> 
>     >> > On 01/11/2014, 11:33 AM, Peter Simons wrote:
>     >> >> Hi Uwe,
>     >> >> 
>     >> >> > Nobody in R core runs NixOS and can reproduce
>     >> >> this. This passes on most > other platforms,
>     >> >> apparently. If you can point us to a problem or send >
>     >> >> patches, we'd appreciate it.
>     >> >> 
>     >> >> have tried running the test suite in a build that's
>     >> >> configured with '--without-recommended-packages'? That's
>     >> >> about the only unusual thing we do when building with
>     >> >> Nix. Other than that, our build runs on a perfectly
>     >> >> ordinary Linux -- and it used to succeed fine in earlier
>     >> >> versions of R.
>     >> 
>     >> > The tests "make check-devel" and "make check-all" are
>     >> > documented to require the recommended packages, and will
>     >> > fail without them.  On Windows, "make check" also needs
>     >> > them, so this may be true on other systems as well.
>     >> 
>     >> Thank you Duncan, for clarifying (above and later in the thread).
>     >> 
>     >> Would it be hard to strive for
>     >> 
>     >> 1)  'make check' should pass without-rec....
>     >> 2)  'make check-devel' etc do require the recommended packages.
>     >> 
>     >> That would be ideal I think - and correspond to the fact that
>     >> we call the recommended packages 'recommended' only.
> 
>     > I think we could avoid errors in make check, but not warnings.  People
>     > need to understand what the tests are testing, and recognize that some
>     > warnings are ignorable.
> 
>     > To do this, we'd need to make sure that no examples in base packages
>     > require the use of recommended packages.  Currently the failure happens
>     > in capture.output, because it runs the glm example which needs MASS.
>     > (The glm example is marked not to need MASS during testing, but the
>     > capture.output example runs everything.)  
> 
> aah.. that's interesting in itself: Maybe  example() should also
> get 'run.dontcheck' argument in addition to its  'run.dontrun'
> and Rd2ex() a similar enhancement.... I'm looking into that.
> 
>     > Fixing that one causes the error to happen later.
> 
> "fascinating", as Kurt may say ..
> 
>     >> OTOH, if '1)' is too much work for us, we could add this as a
>     >> 'wishlist' item  and wait for someone to send patches..
> 
>     > Alternatively, we could require the recommended packages for all tests.
> 
>     > Duncan Murdoch
> 
> which seems too extreme. If some people really only want to test
> something like "the R base engine", they should be easily able
> to do so, and I still think that 'make check' should do exactly that.
> In the tests/Makefile.{common|in} this is even called 
> "test-all-basics"
> 
> One thing to consider might remove 'Examples' from the "all-basics"
> and use 'Examples' only in a new make target between "basics"
> and "devel".
> But personally, I'd strive for fixing the few (I hope) cases in
> the Examples we currently have.
> Using the \dontcheck{..}  tag should really help there.

I don't think we should be removing tests for everybody to allow a few
people to test a build of R that none of us actually use.

The choice of name "recommended" is unfortunate, because it suggests
that these packages are not necessary in order to get R to run:  but a
build that doesn't contain them won't work properly.  The test is giving
correct results:  R "without-recommended" is broken.

We might be able to get it to pass "make check" by removing tests, but
example(capture.output) and example(glm) will still fail.

Duncan Murdoch