[R-pkg-devel] how to skip tests on CRAN but NOT on travis-ci?

[Jennifer Bryan]

Thanks for all of the responses!

What Gabor says below is true and is the path of least resistance in my particular case. As Dirk suggested, environment variables are the way to go.

If I put this into .travis.yml:

env:
  - NOT_CRAN=true

I get what I want

  * The Travis build still runs with the option --as-cran.
  * The tests harbouring testthat::skip_on_cran() are run locally and on Travis
  * The tests harbouring testthat::skip_on_cran() are NOT run on CRAN (at least, the little experiment I just ran on winbuilder suggests this … haven't tried with actual CRAN submission)

TIL: the NOT_CRAN environment variable and the --as-cran option for R CMD check are completely independent of each other.

I also confirmed what Zhian said: if you're using skip_on_cran() and *don't* set NOT_CRAN=true in .travis.yml, you can still get some info about the affected tests in the coveralls result. But this doesn't seem a great solution, since you don't get much detail in the case of failure.

-- Jenny

On 2015-06-21, at 8:51 AM, Gábor Csárdi <csardi.gabor at gmail.com> wrote:

> I think you can just use skip_on_cran() for the tests that you want to
> skip on CRAN, and set the NOT_CRAN environment variable in your Travis
> config.
> 
> Gabor
> 
> On Sun, Jun 21, 2015 at 10:46 AM, Jennifer Bryan <jenny at stat.ubc.ca> wrote:
>> I hope this is an appropriate to place to ask this. My question involves add-on tools and services, but I think they are in common use and others might have same question.
>> 
>> I am using testthat for testing and Travis (https://travis-ci.org) for continuous integration.
>> 
>> I do not want the vast majority of my tests to run on CRAN. They take a long time, require an internet connection, etc. I believe the official testthat solution for this is to use testthat::skip_on_cran() at the beginning of the body of the test.
>> 
>> However, I DO want these tests to run on Travis. In fact, I'd like to be as crazy strict on Travis as possible. More strict than on CRAN. So running R CMD check on Travis "as CRAN" is no good for me.
>> 
>> I think this coincides with this unanswered question on stackoverflow:
>> http://stackoverflow.com/questions/27557150/check-as-cran-but-do-not-skip-any-tests?lq=1
>> 
>> Thanks for any suggestions,
>> Jenny
>> 
>> Jennifer Bryan
>> Associate Professor
>> Department of Statistics and
>>   the Michael Smith Laboratories
>> University of British Columbia
>> Vancouver, BC Canada
>> 
>> ______________________________________________
>> R-package-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel

Jennifer Bryan
Associate Professor
Department of Statistics and
   the Michael Smith Laboratories
University of British Columbia
Vancouver, BC Canada