[R-pkg-devel] suggestion: conda for third-party software

[Kevin Ushey]

On Tue, Jan 7, 2020 at 10:42 PM Sokol Serguei <serguei.sokol using gmail.com> wrote:
>
> Thanks for this hint.
>
> Le 07/01/2020 à 20:47, Kevin Ushey a écrit :
> > The newest version of reticulate does something very similar: R
> > packages can declare their Python package dependencies in the
> > Config/reticulate field of a DESCRIPTION file, and reticulate can read
> > and use those dependencies to provision a Python environment for the
> > user when requested (currently using Miniconda).
>
> If miniconda is used, does it mean that not only Python but any conda
> package can be indicated in dependency ?

In theory yes, but reticulate only accepts Python package dependencies
since its primary goal is interoperation with Python.

> And another question, do you know if miniconda is installed on testing
> CRAN machines? (Without this I cannot see how your packages with conda
> dependencies could be tested during their submission.)

I don't think so. I can't speak for CRAN, but their time is precious
and it seems unlikely to me that they would be willing to expend the
time needed to maintain Conda installations across their fleet of CRAN
machines.

Packages using Miniconda in this way could still run their tests on
different types of infrastructure, though (e.g. Travis CI).

> Best,
>
> Serguei.
>
> >
> > Similarly, rather than having this part of SystemRequirements, package
> > authors could declare these in a separate field called e.g.
> > Config/conda. Then, you could have an R package that knows how to read
> > and parse these configuration requests, and install those packages for
> > the user.
> >
> > That said, maintaining a Conda installation and its environments is
> > non-trivial, and things do not always work as expected when mixing
> > Conda applications with non-Conda applications. Most notably, Conda
> > installations bundle their own copies of libraries; e.g. the C++
> > standard library, Qt, OpenSSL, and so on. If an application tries to
> > mix and match both system-provided and Conda-provided libraries in the
> > same process, bad things often happen. This was still the
> > lowest-friction way forward for us with reticulate, but it's worth
> > being aware that Conda is not a total panacea.
> >
> > Best,
> > Kevin
> >
> > On Tue, Jan 7, 2020 at 6:50 AM Serguei Sokol <serguei.sokol using gmail.com> wrote:
> >> Best wishes for 2020!
> >>
> >> I would like to suggest a new feature for R package management. Its aim
> >> is to enable package developers and end-users to rely on conda (
> >> https://docs.conda.io/en/latest/ ) for managing third-party software
> >> (TPS) on major platforms: linux64, win64 and osx64. Currently, many R
> >> packages include TPS as part of them thus bloating their sizes and often
> >> duplicating files on a given system.  And even when TPS is not included
> >> in an R package but is just installed on a system, it is not so obvious
> >> to get the right path to it. Sometimes pkg-config helps but it is not
> >> always present.
> >>
> >> So, the new feature would be to let R package developers to write in
> >> DESCRIPTION/SystemRequirements field something like
> >> 'conda:boost-cpp>=1.71' where 'boost-cpp' is an example of a conda
> >> package and '>=1.71' is an optional version requirement. Having this
> >> could allow install.packages() to install TPS on a testing CRAN machine
> >> or on an end-user's one. (There is just one line to execute in a shell:
> >> conda install <pkg-name>. It will install the package itself as well as
> >> all its dependencies).
> >>
> >> To my mind, this feature would have the following advantages:
> >>    - on-disk size economy as the same TPS does not have to be included in
> >> R package itself and can be shared with other language wrappers, e.g.
> >> Python;
> >>    - an easy flag configuring in Makevars as paths to TPS will be well
> >> known in advance;
> >>    - CRAN machines could test packages relying on a wide panel of TPS
> >> without bothering with their manual installation;
> >>    - TPS installation can become transparent for the end-user on major
> >> platforms;
> >>
> >> Note that even R is part of conda (
> >> https://anaconda.org/conda-forge/r-base ), it is not mandatory to use
> >> the conda's R version for this feature. Here, conda is just meant to
> >> facilitate access to TPS. However, a minimal requirement is obviously to
> >> have conda itself.
> >>
> >> Does it look reasonable? appealing?
> >> Best,
> >> Serguei.
> >>
> >> ______________________________________________
> >> R-package-devel using r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
>