[Bioc-devel] Nightly builds

[Leonardo Collado Torres]

On Wed, Feb 7, 2018 at 10:36 AM, Leonardo Collado Torres
<lcollado at jhu.edu> wrote:
> Hi,
>
> I try to follow all the best practices but sometimes I end up relying
> on the Bioc nightly build.
>
> Anyhow, I personally use r-travis** for quick checks to detect broken
> commits instead of running R CMD check/build on my laptop. This way I
> can submit a new commit that fixes a broken commit before the Bioc
> nightly build in about the same time as using my laptop. r-travis
> doesn't always work as Bioc might have newer versions of dependencies
> or other issues (cache, timing, installation of deps), but it helps
> most of the time. A scenario where I definitely rely on the Bioc
> nightly build is when I can't reproduce the error/warning on my
> machine (with the latest setup). For example, Windows issues***.
>
> Note that I do run a quick local test of whatever bug I'm solving, but
> I might miss how my changes affected another unit test.
>
> Best,
> Leonardo
>
> ** Example setup
> https://github.com/leekgroup/derfinderPlot/blob/master/.travis.yml
> *** You can also use r-hub for testing in other OS including Windows
> but maybe not with the R version that bioc-devel needs and/or without
> being able to set useDevel(). Example
> https://github.com/LieberInstitute/jaffelab/blob/master/.travis.yml#L17-L18
> I've only used it for that non-bioc package.

More on r-hub and Bioc https://github.com/r-hub/rhub/issues/38 which I
just found

>
> On Wed, Feb 7, 2018 at 9:51 AM, Martin Morgan
> <martin.morgan at roswellpark.org> wrote:
>>
>>
>> Bioc developers!
>>
>> I've been exploring the Bioconductor nightly builds
>>
>>   http://bioconductor.org/checkResults/
>>
>> a bit using this in-development package.
>>
>>   https://github.com/mtmorgan/BiocBuildReports
>>
>> This
>>
>>   rpt <- report()
>>   filter_recent(rpt) %>% print(n = n())
>>
>> summarizes the packages changed in the 24 hours before the current build snapshot, and their fate on the nightly builds
>>
>> > filter_recent(rpt) %>% print(n=nrow(.))
>> # A tibble: 18 x 5
>>    package           buildsrc checksrc push     txt
>>    <chr>             <fct>    <fct>    <fct>    <chr>
>>  1 AllelicImbalance  ERROR    skipped  skipped  skipped
>>  2 ChIPpeakAnno      ERROR    skipped  skipped  skipped
>>  3 GenomicScores     ERROR    skipped  skipped  skipped
>>  4 genphen           ERROR    skipped  skipped  skipped
>>  5 metavizr          ERROR    skipped  skipped  skipped
>>  6 QUBIC             ERROR    skipped  skipped  skipped
>>  7 Repitools         OK       ERROR    skipped  skipped
>>  8 rtracklayer       OK       WARNINGS UNNEEDED same version exists in interna…
>>  9 genoset           OK       WARNINGS YES      new version is higher than in …
>> 10 IRanges           OK       WARNINGS YES      new version is higher than in …
>> 11 LOLA              OK       WARNINGS YES      new version is higher than in …
>> 12 DAPAR             OK       OK       UNNEEDED same version exists in interna…
>> 13 AnnotationHubData OK       OK       YES      new version is higher than in …
>> 14 BEARscc           OK       OK       YES      new version is higher than in …
>> 15 ChemmineR         OK       OK       YES      new version is higher than in …
>> 16 DESeq2            OK       OK       YES      new version is higher than in …
>> 17 Onassis           OK       OK       YES      new version is higher than in …
>> 18 synergyfinder     OK       OK       YES      new version is higher than in …
>>
>> A couple of things strike me about this.
>>
>>   - All the packages with 'ERROR' in the buildsrc or checksrc column failed to propagate because the commit, or a previous version, does not build or check. It is not a very productive development model to push broken commits to git.bioconductor.org, especially if the developer doesn't realize that the commits are broken. The best practice is to build and check packages locally, and then commit.
>>
>>   R CMD build <YourPackage>
>>   R CMD check <YourPackage_<version>.tar.gz
>>
>> An even more robust approach is to clone the repository locally, so that the build and check are against only the committed changes and not miscellaneous files or versions hanging out in your source directory
>>
>>   cd /tmp
>>   git clone /path/to/<YourPackage git>
>>   R CMD build <YourPackage>
>>   ...
>>
>> I would not suggest using devtools during this final stage; the build system doesn't use devtools, as devtools has its own ideas about options etc with which to build packages. The very careful would also use the build and check options found on the build report of each package, e.g., from
>>
>>
>> http://bioconductor.org/checkResults/3.7/bioc-LATEST/rtracklayer/malbec2-buildsrc.html
>>
>> one might
>>
>>   R CMD build --keep-empty-dirs --no-resave-data rtracklayer
>>
>> A common problem is that packages are built and checked locally using the wrong version of R and / or Bioconductor -- commits to the devel branch should be against the devel version of Bioconductor and (for the current release cycle) the devel version of R.
>>
>>   http://bioconductor.org/developers/how-to/useDevel/
>>
>>   - two packages (rtracklayer, DAPAR) built and checked successfully, but did not propagate to the public repository. This is because they did not bump their version number according to policy
>>
>>   http://bioconductor.org/developers/how-to/version-numbering/
>>
>> While there might be reasons for pushing commits to Biocoductor without version bumps, this usually seems to be an oversight on the developer's part. Basically, every public version of a package should have a different version -- there is no shortage of version numbers.
>>
>> Martin
>>
>>
>> This email message may contain legally privileged and/or...
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>