package checks and tests; was: Re: [R] A "rude" question

[Uwe Ligges]

Paulo Justiniano Ribeiro Jr wrote:

> One problem with distributing packages with the test sub-directory
> is that this can overload the daily tests in the CRAN machines
> 
> My workaround for that  is:
> For the geoR package I run the tests in my machine but remove the tests
> directory when submitting to CRAN.
> Also I mantain a package web-page where I make the version
> with tests available for downloading

[moved to R-devel, chosen a more sensible subject]

If tests are running in a reasonable amount of time, it does not make 
sense to remove them. People want to install the package on a huge 
number of platforms you probably have not tested the package on 
yourself. The only solution is to distribute the tests as well.

Overloading the daily tests on CRAN is another issue (strong related, 
though). Installing and checking (in maintainer's mode, i.e. without 
double-installing) all CRAN packages under R-release for Windows takes 8 
hours on a (dual, but using only one) Xeon 3.06GHz machine these days.
The checks (without isntallation) of the 15 packages with most intensive 
  tests take ~1 hour.

Uwe Ligges





> P.J.
> 
> On Tue, 1 Feb 2005, Spencer Graves wrote:
> 
> 
>>Hi, Paul:
>>
>>      How can I access "the package tests directory" you mentioned?
>>Only one of the 52 subdirectories of "library" in my current
>>installation of R 2.0.1 has a "test" folder.
>>
>>      Thanks,
>>      Spencer Graves
>>
>>Paul Gilbert wrote:
>>
>>
>>>One point that did not get mentioned in this discussion, and I believe
>>>deserves
>>>much more publicity, is the impact of packages tests. The design of
>>>the package
>>>system allows package developers to put tests in packages, and these
>>>are checked
>>>regularly (see <http://cran.at.r-project.org/contrib/checkSummary.html>).
>>>
>>>These are intended to test the package functionality, but also give R
>>>what is
>>>perhaps the largest test suite of any statistical software (certainly
>>>the most
>>>quickly growing). While any single package's test will never guarantee
>>>that
>>>the package works perfectly, the ensemble goes a long way toward ensuring
>>>that core R functionality behaves as intended. It seems unlikely to me
>>>that any
>>>commercial effort will ever be able to catch up.
>>>
>>>There are several ways that tests can add to our confidence that
>>>calculations can be trusted. They can
>>>  - check against theoretical results
>>>  - check against published results
>>>  - check against results from other software
>>>  - check that calculations done in different ways give the same result
>>>  - check that monte carlo experiments give distributions that are
>>>consistent
>>>     with expected results
>>>
>>>Some of these are relatively time consuming to set up and check the
>>>first time,
>>>but after that they can be automatic.
>>>
>>>If you have particular calculations with specific packages that you
>>>are especially
>>>concerned about, I encourage you to participate by devising good tests
>>>and sending
>>>them to the package developers. (But first check the tests they are
>>>already doing
>>>in the package tests directory.)
>>>
>>>Paul Gilbert
>>>
>>>msck9 at mizzou.edu wrote:
>>>
>>>
>>>>Dear all, I am beginner using R. I have a question about it. When you
>>>>use it,
>>>>since it is written by so many authors, how do you know that the
>>>>results are trustable?(I don't want to affend anyone, also I trust
>>>>people). But I think this should be a question.
>>>>
>>>>Thanks,
>>>>Ming
>>>>
>>>>______________________________________________
>>>>R-help at stat.math.ethz.ch mailing list
>>>>https://stat.ethz.ch/mailman/listinfo/r-help
>>>>PLEASE do read the posting guide!
>>>>http://www.R-project.org/posting-guide.html
>>>>
>>>>
>>>>
>>>
>>>______________________________________________
>>>R-help at stat.math.ethz.ch mailing list
>>>https://stat.ethz.ch/mailman/listinfo/r-help
>>>PLEASE do read the posting guide!
>>>http://www.R-project.org/posting-guide.html
>>
>>______________________________________________
>>R-help at stat.math.ethz.ch mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>>
>>
> 
> 
> Paulo Justiniano Ribeiro Jr
> LEG (Laboratório de Estatística e Geoinformação)
> Departamento de Estatística
> Universidade Federal do Paraná
> Caixa Postal 19.081
> CEP 81.531-990
> Curitiba, PR  -  Brasil
> Tel: (+55) 41 361 3573
> Fax: (+55) 41 361 3141
> e-mail: paulojus at est.ufpr.br
> http://www.est.ufpr.br/~paulojus
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html