[R-pkg-devel] R CMD Check: Tests running infinite

Patrick Schratz patrick.schratz at gmail.com
Tue Feb 7 23:16:28 CET 2017 

>
> I would be curious
> to see what file.path(R.home("share"), "R", "tests-startup.R")
> contains on the system where the problem occurs.


Contents of mine on macOS are the same as on your Ubuntu:

## A custom startup file for tests
## Run as if a system Rprofile, so no packages, no assignments
options(useFancyQuotes = FALSE)

By disabling this, i.e. Sys.setenv(R_TESTS="")


Note that the fix from Gabor is Sys.unsetenv("R_TESTS"). What´s the
difference between both?

My interest in this issue is because I haven't yet experienced this
> myself and I run lots and lots of package tests in future that
> utilizes the parallel package.


So if you use `foreach` and `parallel`, the major difference points to
`doParallel`. If I get my tests running by avoiding `doParallel`, we would
have some kind of evidence.
Any source on how to quickly substitute my `doParallel` code using
`parallel`? I know its just a few lines, but thats how it always starts..;)

2017-02-07 21:16 GMT+01:00 Henrik Bengtsson <henrik.bengtsson at gmail.com>:

> On Mon, Feb 6, 2017 at 12:41 AM, Gábor Csárdi <csardi.gabor at gmail.com>
> wrote:
> > I don't think it is specific for testthat. R CMD check sets R_TESTS when
> it
> > runs the tests, so the separate R process it starts can have some special
> > startup options.
> >
> > The problem happens if you start another R process from your R test, and
> > then R_TESTS confuses this process. (I am not sure how exactly.)
>
> I wasn't aware of R_TESTS.  Looking at the R source code, if set, will
> cause the base package to source that file _before_ and .Rprofile
> scripts; it basically calls source(Sys.getenv("R_TESTS")).
>
> Next, R CMD check runs each of the test script using
> R_TESTS=startup.Rs, where startup.Rs:
>
>     file.copy(file.path(R.home("share"), "R", "tests-startup.R"),
> "startup.Rs")
>     if (use_gct) cat("gctorture(TRUE)" , file = "startup.Rs", append =
> TRUE)
>
> Looking at the tests-startup.R file that comes with my R 3.3.2 on
> Ubuntu 16.04, it seems pretty harmless:
>
> ## A custom startup file for tests
> ## Run as if a system Rprofile, so no packages, no assignments
> options(useFancyQuotes = FALSE)
>
> So, to me it's not clear how this could make a difference in Patrick
> case.   By disabling this, i.e. Sys.setenv(R_TESTS=""), I don't see
> how it affects running parallel+foreach+doParallel.  Maybe because one
> of those packages are relying on fancy quotes in some protocol,
> passing command-line arguments or ... something.  I would be curious
> to see what file.path(R.home("share"), "R", "tests-startup.R")
> contains on the system where the problem occurs.
>
> >
> > A lot of packages have to work around this:
> > https://github.com/search?q=user%3Acran+R_TESTS&type=Code
>
> I wonder if those are mostly there because of cut'n'paste behavior.
>
> >
> > Most of these use testthat, but not all of them.
>
> My interest in this issue is because I haven't yet experienced this
> myself and I run lots and lots of package tests in future that
> utilizes the parallel package.  In doFuture I do similar tests, which
> is on top of the foreach package.   I don't use testthat and I also
> don't use doParallel in my testing.
>
> /Henrik
>
> >
> > Gabor
> >
> > On Mon, Feb 6, 2017 at 2:17 AM, Henrik Bengtsson
> > <henrik.bengtsson at gmail.com> wrote:
> >>
> >> In case someone else bumps into this later and finds this thread; can
> >> you confirm that this was a problem specific to using the testthat
> >> package for running the tests?
> >>
> >> /Henrik
> >>
> >> On Sun, Feb 5, 2017 at 11:28 AM, Patrick Schratz
> >> <patrick.schratz at gmail.com> wrote:
> >> > @gaborcsardi solved it :) See here:
> >> > https://github.com/hadley/testthat/issues/567#issuecomment-277536577
> >> >
> >> >
> >> > 2017-02-05 16:07 GMT+01:00 Patrick Schratz <patrick.schratz at gmail.com
> >:
> >> >>
> >> >> Thanks for the hint, Hendrik!
> >> >> However, this change did not make a difference :/
> >> >>
> >> >> I tried to use all cluster closing functions I came across but tests
> >> >> are
> >> >> still running infinite..
> >> >>
> >> >> cl <- makeCluster(par.args$par.units, outfile = out.progress)
> >> >> registerDoParallel(cl)
> >> >>
> >> >> foreach()
> >> >>
> >> >> parallel::stopCluster(cl)
> >> >> doParallel::registerDoSEQ()
> >> >> doParallel::stopImplicitCluster()
> >> >>
> >> >> 2017-02-05 15:04 GMT+01:00 Henrik Bengtsson
> >> >> <henrik.bengtsson at gmail.com>:
> >> >>>
> >> >>> Use
> >> >>>
> >> >>>   registerDoParallel(cl)
> >> >>>
> >> >>> The number of parallel workers is already contained in the 'cl'
> >> >>> object,
> >> >>> so don't specify 'cores'!  (If you do that, I suspect you create yet
> >> >>> another
> >> >>> cluster (a multicore one) which is used but never closed)
> >> >>>
> >> >>> registerDoParallel() should ideally give an error in your case.
> Author
> >> >>> BCC:ed.
> >> >>>
> >> >>> Henrik
> >> >>>
> >> >>> On Feb 5, 2017 03:56, "Patrick Schratz" <patrick.schratz at gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> Dear Uwe,
> >> >>>>
> >> >>>> thanks for the hint. My cluster is closed after the `foreach`call
> >> >>>> using
> >> >>>> `stopCluster()`.
> >> >>>>
> >> >>>> Before, I´ll do the following to init the cluster:
> >> >>>>
> >> >>>> *cl <- makeCluster(par.args$par.units, outfile = out.progress)*
> >> >>>> *registerDoParallel(cl, cores = par.args$par.units)*
> >> >>>>
> >> >>>> *foreach()*
> >> >>>>
> >> >>>> *stopCluster(cl)*
> >> >>>>
> >> >>>>
> >> >>>> Do you know of any other package which is using foreach in
> >> >>>> combination
> >> >>>> with
> >> >>>> tests and is hosted on Github? So I could compare settings.
> >> >>>>
> >> >>>> Best, Patrick
> >> >>>>
> >> >>>> 2017-02-02 0:01 GMT+01:00 Uwe Ligges
> >> >>>> <ligges at statistik.tu-dortmund.de>:
> >> >>>>
> >> >>>> > Check whether the parallel cluster is closed. Can it be that the
> >> >>>> > cluster
> >> >>>> > is still open and the check process waits for them to complete?
> >> >>>> >
> >> >>>> > Best,
> >> >>>> > Uwe Ligges
> >> >>>> >
> >> >>>> >
> >> >>>> >
> >> >>>> > On 31.01.2017 13:45, Patrick Schratz wrote:
> >> >>>> >
> >> >>>> >> Hello,
> >> >>>> >>
> >> >>>> >> when running R CMD check / devtools::check, section "running
> >> >>>> >> tests..." is
> >> >>>> >> not finishing (40 min+).
> >> >>>> >>
> >> >>>> >> *Checking tests only works:*
> >> >>>> >>
> >> >>>> >> *==> Sourcing R files in 'tests' directory*
> >> >>>> >>
> >> >>>> >> *testthat results
> >> >>>> >> ============================================================
> ====*
> >> >>>> >> *OK: 7 SKIPPED: 0 FAILED: 0*
> >> >>>> >>
> >> >>>> >> *Tests complete*
> >> >>>> >>
> >> >>>> >>
> >> >>>> >> As well as running tests line-by-line.
> >> >>>> >>
> >> >>>> >> How can I debug my tests to discover the problem during R CMD
> >> >>>> >> check?
> >> >>>> >>
> >> >>>> >> *Tests are using parallelization (foreach + doParallel)*
> >> >>>> >>
> >> >>>> >> Best, Patrick
> >> >>>> >>
> >> >>>> >>         [[alternative HTML version deleted]]
> >> >>>> >>
> >> >>>> >> ______________________________________________
> >> >>>> >> R-package-devel at r-project.org mailing list
> >> >>>> >> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >> >>>> >>
> >> >>>> >>
> >> >>>>
> >> >>>>         [[alternative HTML version deleted]]
> >> >>>>
> >> >>>> ______________________________________________
> >> >>>> R-package-devel at r-project.org mailing list
> >> >>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >> >>
> >> >>
> >> >
> >>
> >> ______________________________________________
> >> R-package-devel at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> >
> >
>

	[[alternative HTML version deleted]]

More information about the R-package-devel mailing list