[R-pkg-devel] new package submission: "check log incomplete, web timeout?"

[Henrik Bengtsson]

Your package uses parallel processing (foreach w/ doParallel).
Peeking at the code, it looks like the vignette is using single-core
processing via doParallel::registerDoParallel(np) where np = 1.  On
Unix-like system, this *forks* off a single R process, and on Windows
it *launches* a single background R session ("PSOCK" cluster) via a
system() call.  Forking works pretty much all the time, so you rarely
see problems there.  However, with PSOCK cluster there are a few
things that can go wrong, e.g. the master R process and the background
worker(s) communicate via a socket on a random port in 11000:11999.

Now, my future package (https://cran.r-project.org/package=future)
runs lots and lots of parallel tests on various backends, including
PSOCK clusters.  I monitor the CRAN R CMD check:s for the future
package, and I do indeed (rather) occasionally see similar silent
timeouts on the Windows CRAN tests which disappear the next round of
CRAN tests.  I've also occasionally seen it when using win-builder.
It's never been a clear pattern and it appears to hit at random times.
The win-builder machine and the CRAN Windows servers have equal setups
(or even are on the same machine).  I'm pretty sure that if the
parallel package itself would run PSOCK tests regularly on CRAN, it
would also see these random silent timeouts.

I've emailed Uwe Ligges about this when I've seen it, but it's a
really hard problem to troubleshoot, especially since it most likely
won't show if you rerun the test.  It could be a CRAN Windows specific
thing, an R Windows specific thing, or something else.   My guess is
that you hit that problem at submission.   I would ask Uwe to recheck
the package again.  If the problem goes away, then you're in the same
boat.

A completely different alternative, is to have a way to disable
parallel processing in your examples / tests / vignettes, e.g. if (np
== 1) registerDoSEQ() else registerDoParallel(np).  Personally, I'm
not a big fan of that approach because it's an ad-hoc workaround and
you're end up not testing all of your package.

Hope this helps

Henrik

On Sun, Apr 23, 2017 at 9:17 AM, Jeremy Volkening <jdv at base2bio.com> wrote:
> On Sun, Apr 23, 2017 at 10:17:01AM +0200, Alexandre Courtiol wrote:
>>
>> Could it be that one your computer you can build vignettes quickly thanks
>> to some caching of the knitr chunk output but that it takes too much time
>> building that without the cache...? ... CRAN does not like when
>> anything takes more than a few seconds...
>
>
> Thanks for the reply. The vignettes do take a little time to build, as the
> knitr code is doing some modeling, but I wouldn't have thought it excessive.
> According to the logs from the win-builder service, it took 28s (about the
> same as each of the i386 and x64 tests). Is this too long for the CRAN
> submission service? I've already pruned down my example data to just a
> couple of samples instead of the thousands normally modeled in a session.
>
> If this is indeed the issue, do I need to remove the knitr code and just
> include static images of the output plots in the vignette? This would seem
> to go against most of the recommendations I have read from various sources
> on best practice.
>
> Regards,
> Jeremy
>
>> On 23 April 2017 at 05:08, Jeremy Volkening <jdv at base2bio.com> wrote:
>>
>>> Hello,
>>>
>>> I am trying to submit a package to CRAN for the first time. I have gone
>>> through testing on my own machines (Linux, Windows x64, Windows i386), on
>>> Travis CI and using win-builder. All tests are now passing on all of
>>> these
>>> platforms. However, when I try to submit the package to CRAN I get an
>>> email
>>> that the pre-test has failed, with the following line:
>>>
>>> Status: NA (check log incomplete, web timeout?)
>>>
>>> The pre-test link is here: https://win-builder.r-project.
>>> org/incoming_pretest/170423_044521_mstherm_047/
>>>
>>> If I check the '00check.log' file I can see that it ends with the line "*
>>> checking re-building of vignette outputs ...", so apparently the test is
>>> quitting before it finishes this step. I'm not sure how to proceed, since
>>> everything is working for all of my own manual tests (as well as
>>> win-builder). See the log for my manual submission of exactly the same
>>> package to win-builder, which passes:
>>>
>>> https://win-builder.r-project.org/4tvH1D2TdMep/
>>>
>>> Any help would be greatly appreciated. The package itself can be found
>>> here:
>>>
>>> https://github.com/jvolkening/r-mstherm
>>>
>>> Regards,
>>> Jeremy
>>>
>>> ______________________________________________
>>> R-package-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>>
>>
>>
>>
>> --
>> Alexandre Courtiol
>>
>> http://sites.google.com/site/alexandrecourtiol/home
>>
>> *"Science is the belief in the ignorance of experts"*, R. Feynman
>
>
> --
> Nothing so needs reforming as other people's habits.
>                 -- Mark Twain, "Pudd'nhead Wilson's Calendar"
>
>
> ______________________________________________
> R-package-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel