[Bioc-devel] Federation of sub-packages acceptable for Bioconductor?

[Martin Morgan]

On 04/16/2013 05:59 AM, Ulrich Bodenhofer wrote:
> Hi,
>
> Some students of mine are currently working on a unified R interface to multiple
> C/C++ algorithms of a kind (too early to mention details). The special challenges
> are that some of these algorithm have special requirements, that not
> necessarily all of them will run on all target platforms, and that all make quite
> special settings in their Makefiles. In order to deal with these challenges,
> in particular, to be able to provide a maximum of algorithms on each platform, we
> have thought of putting all algorithms in separate packages and writing a wrapper
> package that provides a unified interface to all algorithms available on the
> respective platform.
>
> To illustrate our idea, consider the following example: suppose that the algorithms
> are implemented in packages fooAlg1, fooAlg2 and fooAlg3, which are mutually
> independent of each other. Basic functionality that is used by all three packages
> is contained in a package fooCommon on which the packages fooAlg1, fooAlg2 an
> fooAlg3 all depend. Then the wrapper package foo provides a unified
> interface to those packages which are available on the given platform:
>
>           +-- suggests --> fooAlg1 >-- depends --+
>           |                                      |
>    foo >--+-- suggests --> fooAlg2 >-- depends --+--> fooCommon
>           |                                      |
>           +-- suggests --> fooAlg3 >-- depends --+
>
> If the user tries to call an algorithm via the unified interface, package foo
> checks
> if the required package is available. If so, it loads the package and runs the
> algorithm. If not, an error message is shown.
>
> So far, so good. Now my questions:
>
> - Do you think this is a good idea?
>    If not, do you have a better suggestion?

Hi Ulrich -- sounds like an interesting project, and of course it's great to 
make highly performant and relevant code available to a wide audience!

The Bioconductor policy is that packages are available across platforms. Really 
this benefits the end user (no need to discover why it worked on my Mac but not 
on the computer cluster) and the developer (only one branch of code to 
maintain). There are exceptions in Bioconductor, but invariably these cause 
problems for users, developers, and Bioconductor, and weaken the appeal of a 
high-level language touting reproducible research -- it is a _mistake_ to plan 
to implement something that will be so unsatisfactory!

I would instead recommend the difficult path of identifying and implementing the 
cross-platform core of your ideas and ambitions; this in itself can be a 
rewarding software development activity.

> - Would it be acceptable to submit a whole bunch of packages to Bioconductor
>    instead of one single package that contains all algorithms?
>    If yes, would it be ok if the documentation were concentrated in the
>    wrapper package foo? (In particular, would it be ok if fooAlg1, fooAlg2 and
>    fooAlg3 had no vignettes?

Bioconductor policy is that packages have vignettes.

Having complicated dependencies requires considerable discipline on the part of 
developers (maybe you have control over this...) and users (but not this!). It 
also makes use of the package difficult, as your consideration of documentation 
implies -- the user will get easily lost in the already confusing R help system. 
I would instead identify strategies to organize your code within a single 
package. I would also encourage an evolutionary design, where if the full 
ambition of your project is realized and adopted by a broad or deep community of 
users, perhaps in the future the single package evolves to several packages; 
this approach is simplified when the software works uniformly across platforms.

I think also that R and Bioconductor users are, speaking broadly, different from 
general purpose programming language users; they have well-defined use cases 
(differential expression in RNA-seq; copy number variation in DNA-seq; 
annotation of regions of interest, machine learning for exploratory analysis, 
...) and are looking for a 'package' that fulfils their use case, rather than 
for algorithms that can in principle be stitched together to form a solution. 
This does not encourage the packaging of algorithms, but of solutions. This can 
sometimes be less than optimal, e.g., Import'ing a single function from a much 
larger package. Perhaps as a corollary, my opinion is that code should be 
organized into packages in which vignettes make sense.

Hope that helps, and look forward to your contributions!

Martin

>
> Thanks in advance for your kind assistance!
>
> Kind regards,
> Ulrich
>
> --
> Dr. Ulrich Bodenhofer
> Associate Professor
> Institute of Bioinformatics
>
> Johannes Kepler University
> Altenberger Str. 69
> 4040 Linz, Austria
>
> Tel. +43 732 2468 4526
> Fax +43 732 2468 4539
> bodenhofer at bioinf.jku.at
> http://www.bioinf.jku.at/
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel


-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793