[Rd] Packages manually setting NeedsCompilation

[Hervé Pagès]

FWIW NeedsCompilation is a misnomer. IIRC when I was discussing this 
feature with R core members many years ago, it doesn't only flag 
packages that require compilation (and thus contain arch-specific binary 
files), but it is more generally intended to flag packages that the user 
wouldn't be able to install **from source** on a pristine Windows or Mac 
system because the installation process depends on tools that are not 
necessarily present on such systems.

For example I think I remember seeing at the time some packages that 
were using 'make' for non-compilation related business. These packages 
still needed to be flagged with 'NeedsCompilation: yes' in order to 
recognize them as packages for which "binaries" needed to be made for 
the purpose distributing them.

Packages without 'NeedsCompilation' field (or with 'NeedsCompilation: 
no') don't need binaries because the source tarballs is guaranteed to be 
installable on **any** system, in particular on systems that don't have 
any development tools.

Best,

H.

On 15/05/2025 03:38, Iñaki Ucar wrote:
> On Thu, 15 May 2025 at 11:42, Jeroen Ooms <jeroenooms using gmail.com> wrote:
>> On Wed, May 14, 2025 at 11:05 PM Simon Urbanek
>> <simon.urbanek using r-project.org> wrote:
>>> Can you give an example, please? I wonder if there is a real use-case or just bad package design.
>> The case that bit me yesterday was a Bioconductor package Rigraphlib,
>> see https://bioconductor.org/packages/release/bioc/src/contrib/Rigraphlib_1.0.0.tar.gz
>>
>> This package builds a static library entirely from a configure script.
>> Because there is no 'src' dir, we assumed the package to be all-arch,
>> and get linking errors when trying to use it on another arch then it
>> was built for. Arguably this is indeed bad package design, but that is
>> hard to protect against.
> The opposite case is also true: there is a src directory but no
> compilation happens. This is a problem for deb/rpm shipment for two
> reasons. In rpm packaging at least, archful packages are inspected to
> extract debuginfo; since there is no compiled output there, this fails
> and the whole build fails. If the build succeeds (e.g. debuginfo is
> disabled), then we are building the same thing for all architectures,
> when there should be just one, noarch, for all.
>
> I can list at least 3 examples of this:
>
> - BANOVA: requires compilation, there is a src directory, nothing
> happens there, no source files, just a Makefile sitting there.
> - mixl: same, and the Makefile is just used to print if OpenMP is supported (?).
> - mathjaxr: same, and the Makefile is used to install a javascript library (!).
>
>>> I wouldn't think that should happen as configure is supposed to only guide the compilation in src - if there is no src no binaries are expected as the package did not provide any native sources hence there should be no binary content. This looks like something that could be added to R CMD check?
>> In that case maybe R should warn if it runs a package configure script
>> but there is no "src"? And check if NeedsCompilation: yes is set?
>>
>> One way or another, what matters is that when R runs configure or make
>> during install, the value in the "Built" field in the binary package
>> should contain the platform value. This is also important when distros
>> ship R packages in deb or rpm format, to prevent packages that have
>> compiled code from getting "Architecture: all" and be installed on the
>> wrong platforms.
> IMHO, the flag NeedsCompilation in the DESCRIPTION file should be the
> one and only source of truth here, regardless of the presence or not
> of a src directory. Then I see two paths here:
>
> - Either the developer is responsible for setting this flag, and the R
> CMD check should fail if it's yes but no compilation happens or the
> other way around.
> - Or R CMD build is responsible for setting this flag after checking
> if compilation happens or not.
>
-- 
Hervé Pagès

Bioconductor Core Team
hpages.on.github using gmail.com