[R-pkg-devel] What to do when you can't reproduce CRAN R CMD check unit test failures?

[boB Rudis]

Of course the Great And Powerful Ushey did the super-smart thing and
actually tested the code locally. Apologies for not thinking of doing
the same on my end (though I'm also OS X):

    > getwd()
    [1] "/Users/bob/Development/pathological"

    > test()
    Loading pathological
    Loading required package: testthat
    Testing pathological
    ..................1..2..W...................................................................................................................
    Warnings -----------------------------------------------------------------------
    1. copy_dir works with recursive = TRUE (@test_dir_copy.R#34) -
The files ‘javaconf’, ‘ldpaths’, ‘Makeconf’, ‘Renviron’,
‘repositories’ were not copied successfully.

    Failed -------------------------------------------------------------------------
    1. Failure: decompose_path handles missing paths.
(@test_decompose_path.R#279) -
    `warnings` does not match "Coercing .+ to class
[[:punct:]]character[[:punct:]]\\.".
    Actual value: "Coercing x to class ‘character’."


    2. Failure: decompose_path works with a factor input.
(@test_decompose_path.R#329)
    `warnings` does not match "Coercing .+ to class
[[:punct:]]character[[:punct:]]\\.".
    Actual value: "Coercing x to class ‘character’."


    DONE ===========================================================================

I'm using R version 3.3.0 RC (2016-05-01 r70572) & testthat 1.0.2
(2016-04-23 CRAN (R 3.3.0)

I also get those same two errors on a slightly out-of-date R-devel on Ubuntu.

On Thu, May 12, 2016 at 2:09 AM, Kevin Ushey <kevinushey at gmail.com> wrote:
> Is it possible that `getwd()` is reporting something on the CRAN build
> servers that your `decompose_path()` doesn't handle? For example, your
> tests fail for me if I run them while in the root directory (on OS X).
>
> Kevin
>
> On Thu, May 12, 2016 at 12:37 AM, Richard Cotton <richierocks at gmail.com> wrote:
>> Thanks Bob,
>>
>> I'd forgotten about the option to run win-builder against R-devel.
>>
>> Though it seems that in this case, all the tests are passing there too.
>>
>> http://win-builder.r-project.org/21kc9XGKdANF/examples_and_tests/tests_i386/testthat.Rout
>> http://win-builder.r-project.org/21kc9XGKdANF/examples_and_tests/tests_x64/testthat.Rout
>>
>> I'm now completely out of ideas as to where the problem could be.
>>
>>
>> On 11 May 2016 at 15:46, boB Rudis <bob at rudis.net> wrote:
>>> did you try winbuilder R-devel? I occasionally (ok, often) forget to
>>> do that and it sometimes kick out different errors than R-devel on
>>> local systems.
>>>
>>> On Wed, May 11, 2016 at 2:40 AM, Richard Cotton <richierocks at gmail.com> wrote:
>>>> Yesterday I uploaded a new version of the pathological package to
>>>> CRAN.  It was initially accepted but today I got a message saying that
>>>> some of the tests for the decompose_path function are failing under
>>>> Windows.
>>>>
>>>> I've tested the package on a local Windows 10 machine with R-3.3.0 and
>>>> R-devel, and in Win 7 and Win 8 compatibility modes, and on Windows
>>>> Server 2012 via AppVeyor.  In all cases, all the tests pass.
>>>>
>>>> I tried the CRAN win builder service today, and again, all tests pass.
>>>> http://win-builder.r-project.org/0ZrX4I75nvzM/examples_and_tests/tests_i386/testthat.Rout
>>>> http://win-builder.r-project.org/0ZrX4I75nvzM/examples_and_tests/tests_x64/testthat.Rout
>>>>
>>>> The failure portion of the output I got from Uwe is as follows:
>>>>
>>>> * checking tests ... ERROR
>>>> Running the tests in 'tests/testthat.R' failed.
>>>> Last 13 lines of output:
>>>>   1. Failure: decompose_path handles paths with no directory and a
>>>> single extension in the filename. (@test_decompose_path.R#42)
>>>>   2. Failure: decompose_path handles paths with no directory and a
>>>> double extension in the filename. (@test_decompose_path.R#72)
>>>>   3. Failure: decompose_path handles paths with no directory and no
>>>> extension in the filename. (@test_decompose_path.R#103)
>>>>   4. Failure: decompose_path handles filenames containing a '.' and an
>>>> extension. (@test_decompose_path.R#133)
>>>>   5. Failure: decompose_path handles the current directory as '.'.
>>>> (@test_decompose_path.R#221)
>>>>   6. Failure: decompose_path handles the parent directory as '..'.
>>>> (@test_decompose_path.R#236)
>>>>   7. Failure: decompose_path handles files inside '.'.
>>>> (@test_decompose_path.R#251)
>>>>   8. Failure: decompose_path works with a character vector input.
>>>> (@test_decompose_path.R#321)
>>>>   9. Failure: decompose_path works with a factor input.
>>>> (@test_decompose_path.R#333)
>>>>   1. ...
>>>>
>>>>
>>>> Since this isn't hugely informative as to the problem, I'd like
>>>> general advice on how I can diagnose a failing test when I can't
>>>> reproduce it.
>>>>
>>>> It would also be useful if anyone can run R CMD check and see if they
>>>> find test failures.
>>>>
>>>> The CRAN release is here (the problematic version is 0.0-8):
>>>> https://cran.r-project.org/web/packages/pathological
>>>>
>>>> and the github version is currently in the same state, available here:
>>>> https://github.com/richierocks/pathological
>>>>
>>>> --
>>>> Regards,
>>>> Richie
>>>>
>>>> Learning R
>>>> 4dpiecharts.com
>>>>
>>>> ______________________________________________
>>>> R-package-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>>
>>
>> --
>> Regards,
>> Richie
>>
>> Learning R
>> 4dpiecharts.com
>>
>> ______________________________________________
>> R-package-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel