[Rd] binary R packages for GNU/Linux

[Tobias Verbeke]

Hi Simon,

Thank you for your prompt reply.

----- Original Message -----
> From: "Simon Urbanek" <simon.urbanek using R-project.org>
> To: "Tobias Verbeke" <tobias.verbeke using openanalytics.eu>
> Cc: "r-devel using r-project.org" <r-devel using r-project.org>
> Sent: Monday, February 10, 2025 10:33:40 AM
> Subject: Re: [Rd] binary R packages for GNU/Linux

> Tobias,
> 
> although we did discuss the possibility of extending the
> os/toolchain/architecture notation for binary packages beyond macOS, Linux was
> not necessarily on the list as Linux distributions have already established
> ways of providing binaries, so it does not seem productive to duplicate the
> effort. Can you elaborate a bit more on what you had in mind? Binaries are by
> design specific to toolchain, distribution and architecture, so there is no
> such thing as a "GNU/Linux binary". 

We agree. It is just used as a generic term here to denote a package built for a specific version of a distribution for a specific architecture and its toolchain.

> The only reliable way to distribute
> packages in Linux is from sources or by the Linux distribution repositories.
> Binaries are inherently tied to system dependencies and their versions, so such
> concept doesn't make any sense outside of the distribution. There is no such
> thing as a "jammy binary" to take up your example - it would have to depend on
> the distribution, toolchain and all library versions as well.

jammy is a specific version of the distribution (Ubuntu 22.04 LTS) and the architecture is included in my proposal (x86_64)

http://mydomain.com/bin/linux/jammy/x86_64/contrib/4.4

In my personal experience (~25y of GNU/Linux, mostly Debian, Ubuntu and CentOS) versions of the toolchain will not differ in a practically relevant way within a particular version of a distribution, so it is possible to build and distribute packages for a specific version of a distribution on a specific architecture for a specific series of R (4.4). I guess that if it was not possible, the effort would also not have been undertaken by PPM mentioned earlier. They offer (I skip the non-publicly available Linux distros) CentOS 7 (centos7), Rocky Linux 9 (rhel9), Ubuntu 20.04 (focal), Ubuntu 22.04 (jammy), Ubuntu 24.04 (noble), Debian 11 (bullseye), Debian 12 (bookworm). Another argument to demonstrate the feasibility is the r2u project (https://github.com/eddelbuettel/r2u). It offers CRAN as Ubuntu Binaries, but in order to build these Ubuntu Binaries it actually makes use of the binary R packages built by PPM. Quoting from https://eddelbuettel.github.io/r2u/: "For the CRAN binaries we either repackage P3M/RSPM/PPM builds (where available) or build natively." They cover all CRAN packages. The usage of PPM as a source is, of course, a weakness (in the grand scheme of things), but the point here is about the feasibility of building the packages in a portable way per version of a particular distribution, architecture etc.

I do understand arguments pro 'distribution binaries' (e.g. dependency resolution of system dependencies), but there are also arguments pro 'CRAN binaries' (binary builds of the R packages), since it can be convenient to allow for fast installation of arbitrary R packages without the need of more general sudo rights (required for installation of distribution binaries).

If R-core/CRAN maintainers agree that the answer to my first question is 'no', I am fine with that, but I can only know when asking.

It still leaves the answer to my second question ('official' repository layout conventions) open. It could be: 'we think it is a bad idea, so don't propose a structure' or 'we think it is a bad idea, but propose the following structure for people with bad ideas' :-).

Kind regards,
Tobias

>> On Feb 10, 2025, at 10:08 PM, Tobias Verbeke <tobias.verbeke using openanalytics.eu>
>> wrote:
>> 
>> L.S.
>> 
>> AFAICS the Writing R Extensions and R Installation and Administration manuals do
>> not explicitly discuss binary R packages on GNU/Linux. Only installation from
>> source is mentioned
>> (https://cran.r-project.org/doc/manuals/R-admin.html#Installing-packages-1)
>> and when discussing repository layouts
>> (https://cran.r-project.org/doc/manuals/R-admin.html#Setting-up-a-package-repository)
>> no mention is made of conventions for GNU/Linux distributions.
>> 
>> The proprietary Package Manager (PPM) from Posit
>> (https://packagemanager.posit.co/client/#/) does offer binary packages for
>> GNU/Linux, but the usage of this service is restricted in ways that go against
>> the principles of open source
>> (https://posit.co/about/posit-service-terms-of-use/). By way of example,
>> mirroring is not allowed and certain categories of users are excluded (age
>> categories, competitors, ...). This is maybe expected to some, but this clearly
>> does not offer a proper foundation for the distribution of open source R
>> packages.
>> 
>> For this reason I am wondering whether the R project / CRAN would not be better
>> placed and/or open to support distribution of binary R packages on GNU/Linux.
>> 
>> A second, orthogonal question is whether the R project can advance an official
>> convention for the repository layout related to the distribution of binary
>> GNU/Linux packages. Our proposal would be to use something along
>> 
>> http://mydomain.com/bin/linux/jammy/x86_64/contrib/4.4
>> 
>> which IMHO is more elegant than
>> 
>> http://mydomain.com/bin/linux/jammy-x86_64/contrib/4.4
>> 
>> (and which mimicks the documented MacOS convention
>> 
>> http://mydomain.com/bin/macosx/big-sur-x86_64/contrib/4.4).
>> 
>> Anyone?
>> 
>> Obviously willing to work out details and collaborate on the topic.
>> 
>> Kind regards,
>> Tobias
>> 
>> ______________________________________________
>> R-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel