[R-pkg-devel] Fwd: Reproducing CRAN pre-tests

[Tomas Kalibera]

On 9/20/19 2:43 AM, Aravind J. wrote:
> Hi,
>
> I was able to fix the issue and submit my package successfully to CRAN.
>
> However, the question remains, is there any way to replicate the CRAN
> pre-tests locally for future debugging, particularly in a windows
> environment?
>
> I had tried rhub platforms without any success.

As this is in a recurring question I am offering a bit more detailed 
advice. In summary the setups of the CRAN systems are documented, so one 
can create easily a similar setup in a container or use those say from 
r-hub. But if that doesn't help to reproduce for whatever reason, my 
advice is don't focus too much on trying to replicate exactly the CRAN 
checking system, often it does not help to find the problem. The CRAN 
outputs have been tuned to include all necessary information.

https://cran.r-project.org/web/checks/check_flavors.html describes the 
architectures/compilers used and more information in "Details".

https://cran.r-project.org/web/checks/check_issue_kinds.html describes 
setups for additional checks (valgrind, asan, ubsan, ..).

If you can't reproduce in a container that tries to mimic some of these 
configurations, it does not necessarily mean that the container is not 
configured properly - some errors are non-deterministic or depend 
(deterministically) on things you can't control. This should not be the 
case of compiler warnings, though, I usually can reproduce these with 
the specific compiler with the same or newer version, usually even on a 
different Linux distribution or even on Windows, but rarely the 
distribution matters where it also depends say on system headers. With 
most warnings from the compiler one should be able to fix without being 
able to reproduce, only in a few cases (including -Wstringop-overflow 
though) it can be hard. Of course one has to enable the warnings that 
are not normally enabled by default.

With other kinds of errors one gets sufficiently detailed output that 
there is no need trying to reproduce the checking process. E.g. for 
memory errors with asan/ubsan/valgrind/rchk, one always has to read the 
code around the locations mentioned, no matter what, until one finds the 
error manually and fixes it. It is easy to see that something is an 
error in this case once found, and as soon as it is plausible the error 
fixed is the one reported in the CRAN outputs, I'd just check the 
package using usual means and submit a new version of the package. 
Iterating on the code, making inconsequential changes until the tool is 
happy, might lead and often does to hiding the error instead of fixing 
it, also one does not learn much from that. When one does not run the 
tool locally, the temptation is not there.

With say segfaults being able to reproduce is very important, but there 
running in a container even carefully set up to mimic the check system 
often will not help, the result is often non-deterministic. One needs to 
provoke the error on whatever system, gctorture often helped me in such 
cases, but often one has to experiment with different variants of say 
the R code example during which the segfault happened, or with different 
external libraries (say LAPACK), or with slightly different inputs. This 
is hard work, success rate increases with experience - there is no 
simple trick around it. Trying to literally mimic the test system beyond 
some point to which it is easy is often a waste of time here, I often 
instead intentionally run on many different platforms from the tested 
one, often getting in the end a reproducible crash in a different 
configuration from the one used by CRAN.  With segfaults one can also 
already switch to debug-friendly build (debug symbols, I prefer with no 
compiler optimizations) before trying the process of provoking the bug; 
if you provoke it in an optimized build, but then switch to a 
debug-friendly build, it often scares the bug away, especially the type 
of bugs that are hard to find. With segfaults and other bugs that 
require debugging, running in an external service, modifying the code to 
add checks and assertions, or even in a stripped down container, is not 
always helpful (one has to install tools for the debugging, sometimes 
this means also upgrading packages, one cannot easily do some kind of 
debugging in Docker in the default security setup). It is best to try to 
provoke the error first on the system where one normally does all 
development and debugging.

Tomas

>
> Warm Regards
>
>
> On Thu, 19 Sep 2019 at 16:45, Duncan Murdoch <murdoch.duncan using gmail.com>
> wrote:
>
>> On 19/09/2019 6:51 a.m., Aravind J. wrote:
>>> Hi,
>>>
>>> I am trying to submit an updated version of my package PGRdup to CRAN.
>> The
>>> package has compiled code in C.
>>>
>>> I do development in a windows environment.
>>>
>>> The update was to fix issues in
>>> https://cran.r-project.org/web/checks/check_results_PGRdup.html.
>>>
>>> I have successfully fixed PCRE2 compatability issues and the issue with "
>>> ‘strncpy’ specified bound depends on the length of the source argument".
>>>
>>> For all instances of of strncpy I had changed the dependency on the
>> length
>>> of the source argument to that of destination argument.
>>>
>>> On submission, now one issue remains with debian pre-test.
>>>
>>>     /usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10:
>>> warning: ‘__builtin_strncpy’ specified bound depends on the length of
>>> the source argument [-Wstringop-overflow=]
>>>
>>>
>> https://win-builder.r-project.org/incoming_pretest/PGRdup_0.2.3.4_20190919_090630/Debian/00check.log
>>> How to reproduce this error. I have tried rhub platforms
>>> windows-x86_64-devel, ubuntu-gcc-release, fedora-clang-devel,
>>> linux-x86_64-rocker-gcc-san, linux-x86_64-rocker-gcc-san and ubuntu-rchk
>>> without any success.
>>>
>>> How to reproduce the pre-test results in a windows environment? and how
>> to
>>> fix this specific error?
>>>
>> The install.out file points to the error here:
>>
>> fdouble_metaphone.c: In function ‘DoubleMetaphone’:
>> fdouble_metaphone.c:271:33: note: length computed here
>>     271 |     strncpy(original->str, str, strlen(str) + 1);
>>
>> Presumably you need to limit the copy to the destination length.
>>
>> Duncan Murdoch
>>
>