[Bioc-devel] External software dependencies, bioc builders, and docs maintenance

[Vincent Carey]

Thanks for this note Spencer.  I started looking at the meme
infrastructure to start to learn how to proceed.

An initial concern is the licensing of meme, with which I was
unfamiliar.  The license includes

    Those  desiring to  incorporate this  software into commercial
    products  or use for  commercial  purposes  should contact the
    Technology  Transfer  Office,  University of California,   San
    Diego,  9500 Gilman Drive,  La Jolla,  California, 92093-0910,
    Phone: (858) 534-5815.

Let's put that to one side for now.  Then there's the issue of
installability of meme-5.4.1.  After
updating my xml/xslt infrastructure, make succeeded, but make check gave

make[5]: Entering directory '/home/stvjc/meme-5.4.1/tests/scripts'
Can't locate Sys/Info.pm in @INC (you may need to install the
Sys::Info module) (@INC contains: /home/stvjc/meme

This was not remedied with a simple cpanm operation, so some dev ops
expertise is needed.

What about the dockerized meme?  Could we use that?

---

I looked at the specifics of meme to get somewhat concrete about the
tradeoffs between  expansion of build system infrastructure and its
maintenance, and full
testability of all contributed packages in the build system.  Frequent
integrated testing is an
important component of the Bioconductor ecosystem,
and blocking package code from being checked in the build system should be
minimized.  I would like to have an arrangement in which images of the
"build systems"
(for linux, mac and windows) are accessible for
experimentation and enhancement by contributor-developers.  This would
allow contributor-developers to become very precise about the "System
requirements" of their
packages, to define precisely the configurations needed for successful
integrated testing, and to obtain
data on resource consumption useful in determining the cadence of
testing (choosing between
long-running and daily-running tests). "Containerization" of the linux
build system could form a step in this
direction.

Further discussion is most welcome.

On Tue, Aug 2, 2022 at 10:34 AM Spencer Nystrom <nystromdev using gmail.com> wrote:
>
> Hi all,
>
> Related to the recent discussion of packages building "fake" docs, I wanted
> to call myself out here as being guilty of it and discuss how to mediate
> the issue.
>
> In my case, the {memes} package (
> https://bioconductor.org/packages/release/bioc/html/memes.html) has core
> functionality that depends on an external install of the meme suite family
> of tools. This is currently missing on the builders, so there are a few
> vignettes and examples that use a `NOT_CRAN` style check if the software is
> missing at build time to skip those executions. In order to build more
> useful docs, I render them with a github action using a container with the
> software installed and post them to my own site, which I also point users
> towards. I completely agree this is suboptimal, and it has haunted me for
> some time now.
>
> I see 3 potential solutions to the problem:
>
> 1. Install the latest Meme Suite version on bioc builders. Without
> enumerating all the ways this could cause issues, I foresee mismatches
> between the version I support vs installed on the builders could make this
> a recurring maintenance burden for the core team.
>
> 2. Use {basilisk} to provide a conda env with the current supported MEME
> Suite version within the {memes} package. This has its own issues because
> the current conda envs for the MEME Suite don't really work right, and I'm
> not sure I can take on the maintenance burden of something that extensive.
> There have historically been a few cryptic bugs in their configs.
>
> 3. Some kind of {memes}-driven installer. I've seen other bioc packages
> provide helper functions that install software for the user with their
> permission & in a user-provided location. Provided the builders have the
> system dependencies installed, I guess the package could install the
> software to a tmpdir and clean up after itself on bioc builders. This also
> seems pretty hacky and error prone from  a builder perspective.
>
> Anyway, I'd appreciate any feedback here. In the course of writing this
> I've talked myself into doing another round of due diligence with the conda
> versions but curious to hear others thoughts. In particular this pattern of
> relying on an external dependency is one I've seen a few times in other
> packages and have seen it solved in a variety of creative ways.
>
> Best,
>   -Spencer
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel

-- 
The information in this e-mail is intended only for the ...{{dropped:16}}