[Bioc-devel] PPA with built bioconductor packages (for continuous integration)

[Laurent Gautier]

On Nov 9, 2014 8:06 PM, "Dan Tenenbaum" <dtenenba at fredhutch.org> wrote:
>
>
>
> ----- Original Message -----
> > From: "Martin Morgan" <mtmorgan at fredhutch.org>
> > To: "Laurent Gautier" <lgautier at gmail.com>, bioc-devel at r-project.org
> > Sent: Sunday, November 9, 2014 8:26:48 AM
> > Subject: Re: [Bioc-devel] PPA with built bioconductor packages (for
continuous integration)
> >
> > On 11/09/2014 07:23 AM, Laurent Gautier wrote:
> > > Hi,
> > >
> > > Continuous integration is a convenient way to automate some of the
> > > steps
> > > necessary to ensure quality software.
> > >
> > > Popular ways to do it create a vanilla virtual machine 9VM) with a
> > > Linux
> > > distribution, and scripts prepares the VM with 3rd-party
> > > dependencies
> > > required by the software. For example, the popular CI system Travis
> > > for
> > > github creates by default a VM running ubuntu, and dependencies can
> > > be
> > > installed with `apt-get install`.
> > >
> > > When developing software that requires CRAN/bioconductor, the
> > > latest R is
> > > available precompiled but the R packages must be downloaded
> > > installed from
> > > source.
> > >
> > > This can take a relatively long time. On a recent project over 80%
> > > of the
> > > time is spent downloading/installing the R/BioC packages. The
> > > remaining is
> > > building the code and running the unit tests.
> > >
> > > Having a Personal Package Archive (PPA) with bioconductor packages
> > > already
> > > compiled would both speed up the process and make the use of
> > > continuous
> > > integration by projects relying on bioconductor packages easier.
> > >
> > > Is this something others would like to have, and is this something
> > > that
> > > bioconductor would see to its mission to provide / help provide
> > > quality
> > > software and be able to host ?
> >
> > It would be interesting to catalogue objectives (e.g., development
> > vs.
> > reproducibility) and available alternatives (e.g., PPA, docker /
> > Rocker, AMI,
> > existing or possible cloud services [such as the Bioc 'single package
> > builder'
> > used to build and check new package submissions, or travis itself],
> > the Becker
> > repository management scheme Michael and Gabe mention, ...);
>
>
> Just to add to the mix of options, it's possible to run
> R CMD INSTALL --build on a source tarball on Linux and it will create a
'binary' version that is already compiled.
> The problem with this is (AFAIK) there is no corresponding package type
that can be used with install.packages();
> otherwise the simplest solution would be to add a CRAN-style repos
containing these "binaries". Maybe R could be patched to allow this?

This would be a solution to the problem, and more general one ( a PPA would
be possibly restricted to Ubuntu or Debian.

For some similar situations with python packages I am using Python "
wheels", which are the same as the proposed binary R packages.

Laurent

> But it's possible that the requirements for Linux "binaries" could vary
depending on many things: cpu type (intel or solaris, or...),
> architecture (i386, x64), presence/absence of BLAS/LAPACK, etc etc etc.
This suggests that a vm or container-based approach might be better.
>
> Dan
>
>
>
>
> > if there
> > is a clear
> > path forward satisfying some plurality of users without too many
> > technical
> > obstacles then it might fall within the Bioc purview; my initial
> > sense is that
> > there is not a consensus on use cases or viable implementations, but
> > I can be
> > convinced otherwise...
> >
> > In terms of Tim's post, getting your colleague to use a PPA /
> > existing
> > alternative (e.g., the Bioc AMI,
> > http://bioconductor.org/help/bioconductor-cloud-ami/ which comes with
> > Rstudio
> > server installed...) is not likely to be easier / faster than getting
> > them to
> > download / install relevant R / Bioc packages. One interesting
> > possibility is a
> > 'hosted' bioconductor with sufficient computational resources on the
> > back-end
> > and Rstudio server on the front end; this is not impossible to
> > imaging seeking
> > funding for.
> >
>
>
>
> > Martin
> >
> > >
> > > Best,
> > >
> > > Laurent
> > >
> > >     [[alternative HTML version deleted]]
> > >
> > > _______________________________________________
> > > Bioc-devel at r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >
> >
> >
> > --
> > Computational Biology / Fred Hutchinson Cancer Research Center
> > 1100 Fairview Ave. N.
> > PO Box 19024 Seattle, WA 98109
> >
> > Location: Arnold Building M1 B861
> > Phone: (206) 667-2793
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >

	[[alternative HTML version deleted]]