[R-pkg-devel] Submitting a package whose unit tests sometimes fail because of server connections

[Will]

Hello everyone,

Thank you very much for your help with this! These are some excellent
ideas; I think we will go with either the mocking approach or a variant of
Dirk's suggestion to use a test threshold.

Thanks again!

Will

---

Need a phylogeny? Try phyloGenerator: original
<http://willpearse.github.io/phyloGenerator/> or new version
<http://willpearse.github.io/phyloGenerator2/>
Measuring phylogenetic structure? Try install.packages('pez')

Will Pearse <http://www.pearselab.com/>
Assistant Professor of Biology, Utah State University
Office: +1-435-797-0831; Room LSB-333
Skype: will.pearse


On Tue, 16 Apr 2019 at 12:20, Iñaki Ucar <iucar using fedoraproject.org> wrote:

> On Tue, 16 Apr 2019 at 19:57, Will <will.pearse using gmail.com> wrote:
> >
> > Hello everyone,
> >
> > I'm sorry to bother you, but I would like some help getting a package (
> > *suppdata*; https://github.com/ropensci/suppdata) available through
> CRAN.
> > The package downloads supplementary materials from papers and data
> > repositories on the basis of their DOI, making reproducible analyses a
> > little easier to share and create.
> >
> > The package's unit tests involve downloading data, and when multiple
> > requests for the same resource are sent from a single machine (as seems
> to
> > be done by CRAN's testing servers) the data hosts will sometimes refuse
> the
> > connection and so the package's tests will fail. I emphasise that the
> tests
> > pass at Travis-CI (https://travis-ci.org/ropensci/suppdata) and OpenCPU
> (
> > https://ropensci.ocpu.io/suppdata/info); I am confident this is a
> > connection issue but I would be grateful to be shown I am wrong. I am not
> > sure how to solve this problem, and was hoping you might have some
> advice.
> > Some things I have considered include:
> >
> >
> >    1. Skipping all unit tests on CRAN (using something like
> >    *testtht::skip_on_cran*). This would immediately fix the problem, and
> as
> >    a mitigating factor we report automated test result and coverage on
> the
> >    package's GitHub page (https://github.com/ropensci/suppdata).
>
> This doesn't seem a good idea.
>
> >    2. Using HTTP-mocking to avoid requiring a call to a server during
> tests
> >    at all. I would be uncomfortable relying solely on this for all tests,
> >    since if the data hosters changed things we wouldn't know. Thus I
> would
> >    still want the Internet-enabled tests, which would also have to be
> turned
> >    off for CRAN (see 1 above). This would also be a lot of additional
> work.
>
> What about mocking on CRAN and run Internet-enabled tests otherwise?
>
> >    3. Somehow bypassing the requirement for the unit tests to all pass
> >    before the package is checked by the CRAN maintainers. I have no idea
> if
> >    this is something CRAN would be willing to do, or if it is even
> possible.
> >    It would be the easiest option for me, but I don't want to create
> extra
> >    work for other people!
>
> I think it's acceptable to skip a test if something is not available,
> but not the majority of them, for packages like this which basically
> is about downloading stuff from APIs. Mocking on CRAN, as said before,
> is the way to go here.
>
> >    4. Slowing the tests with something like *Sys.sleep*. This might work,
> >    but would slow the tests massively and so might that cause problems
> for
> >    CRAN?
>
> This is not a good idea either.
>
> --
> Iñaki Úcar
>

	[[alternative HTML version deleted]]