[Bioc-devel] PPA with built bioconductor packages (for continuous integration)

[Dan Tenenbaum]

----- Original Message -----
> From: "Martin Morgan" <mtmorgan at fredhutch.org>
> To: "Laurent Gautier" <lgautier at gmail.com>, bioc-devel at r-project.org
> Sent: Sunday, November 9, 2014 8:26:48 AM
> Subject: Re: [Bioc-devel] PPA with built bioconductor packages (for continuous integration)
> 
> On 11/09/2014 07:23 AM, Laurent Gautier wrote:
> > Hi,
> >
> > Continuous integration is a convenient way to automate some of the
> > steps
> > necessary to ensure quality software.
> >
> > Popular ways to do it create a vanilla virtual machine 9VM) with a
> > Linux
> > distribution, and scripts prepares the VM with 3rd-party
> > dependencies
> > required by the software. For example, the popular CI system Travis
> > for
> > github creates by default a VM running ubuntu, and dependencies can
> > be
> > installed with `apt-get install`.
> >
> > When developing software that requires CRAN/bioconductor, the
> > latest R is
> > available precompiled but the R packages must be downloaded
> > installed from
> > source.
> >
> > This can take a relatively long time. On a recent project over 80%
> > of the
> > time is spent downloading/installing the R/BioC packages. The
> > remaining is
> > building the code and running the unit tests.
> >
> > Having a Personal Package Archive (PPA) with bioconductor packages
> > already
> > compiled would both speed up the process and make the use of
> > continuous
> > integration by projects relying on bioconductor packages easier.
> >
> > Is this something others would like to have, and is this something
> > that
> > bioconductor would see to its mission to provide / help provide
> > quality
> > software and be able to host ?
> 
> It would be interesting to catalogue objectives (e.g., development
> vs.
> reproducibility) and available alternatives (e.g., PPA, docker /
> Rocker, AMI,
> existing or possible cloud services [such as the Bioc 'single package
> builder'
> used to build and check new package submissions, or travis itself],
> the Becker
> repository management scheme Michael and Gabe mention, ...);


Just to add to the mix of options, it's possible to run
R CMD INSTALL --build on a source tarball on Linux and it will create a 'binary' version that is already compiled.
The problem with this is (AFAIK) there is no corresponding package type that can be used with install.packages();
otherwise the simplest solution would be to add a CRAN-style repos containing these "binaries". Maybe R could be patched to allow this?
But it's possible that the requirements for Linux "binaries" could vary depending on many things: cpu type (intel or solaris, or...),
architecture (i386, x64), presence/absence of BLAS/LAPACK, etc etc etc. This suggests that a vm or container-based approach might be better.

Dan




> if there
> is a clear
> path forward satisfying some plurality of users without too many
> technical
> obstacles then it might fall within the Bioc purview; my initial
> sense is that
> there is not a consensus on use cases or viable implementations, but
> I can be
> convinced otherwise...
> 
> In terms of Tim's post, getting your colleague to use a PPA /
> existing
> alternative (e.g., the Bioc AMI,
> http://bioconductor.org/help/bioconductor-cloud-ami/ which comes with
> Rstudio
> server installed...) is not likely to be easier / faster than getting
> them to
> download / install relevant R / Bioc packages. One interesting
> possibility is a
> 'hosted' bioconductor with sufficient computational resources on the
> back-end
> and Rstudio server on the front end; this is not impossible to
> imaging seeking
> funding for.
> 



> Martin
> 
> >
> > Best,
> >
> > Laurent
> >
> > 	[[alternative HTML version deleted]]
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> 
> 
> --
> Computational Biology / Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N.
> PO Box 19024 Seattle, WA 98109
> 
> Location: Arnold Building M1 B861
> Phone: (206) 667-2793
> 
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>