[Rd] Should package version requirements assume installation from sources?

[Ben Bolker]

   Interesting.

   How is the version number bumping communicated back to package 
maintainers? Or is there a rule like "third digit-blocks of package 
versions are reserved for Bioconductor modifications" ?

On 2022-09-13 7:11 p.m., Hervé Pagès wrote:
> FWIW this happens all the time in Bioconductor, where S4 is used very 
> intensively, when we make that kind of change to a core infrastructure 
> package. Then a bunch of Windows and Mac binary packages need to be 
> rebuilt because they contain cached stuff that is now out-of-sync with 
> the new state of affairs. The way we deal with this is by bumping the 
> version numbers of all the affected packages. This triggers automatic 
> rebuilt and propagation of the new binaries.
> 
> Note that these problems also affect packages that were already 
> installed on the user machines _prior_ to the changes to the upstream 
> package. Because the caching we are talking about is actually in the 
> package installation folder (and the fact that it ends up in the 
> binaries is just a consequence of that). So even on a system where 
> packages are installed from source, the affected packages need to be 
> reinstalled. Bumping the version numbers of all the affected packages 
> also solves that.
> 
> I don't see how this could be handled or mitigated via package 
> requirements.
> 
> Best,
> 
> H.
> 
> 
> On 13/09/2022 14:45, Mikael Jagan wrote:
>> [Arguably also appropriate for R-package-devel, but posted to R-devel
>>     as the discussion is aimed primarily at "experts" ... ]
>>
>> We, the authors of Matrix, have encountered a somewhat subtle issue
>> induced by caching of S4 classes and methods in package namespaces.
>>
>> The namespaces of three reverse dependent packages (SeuratObject, 
>> conText,
>> mcmcsae) cache the formal definition of our virtual class Matrix (and 
>> some
>> subclasses).  For example:
>>
>> > ns <- asNamespace("SeuratObject")
>> > grep("^[.]__C__.*Matrix$", names(ns), value = TRUE)
>> [1] ".__C__dMatrix"       ".__C__compMatrix"    ".__C__AnyMatrix"
>> [4] ".__C__generalMatrix" ".__C__CsparseMatrix" ".__C__sparseMatrix"
>> [7] ".__C__dsparseMatrix" ".__C__Matrix"
>>
>> The cached definition (which includes a _validity method_) is obtained 
>> from
>> the version of Matrix available when the reverse dependent package was 
>> built
>> from sources.  For example, if SeuratObject was built under Matrix 1.4-1,
>> then we get:
>>
>> > getValidity(ns$.__C__Matrix)
>> function (object)
>> {
>>     if (!isTRUE(r <- .Call(Dim_validate, object, "Matrix")))
>>         r
>>     else .Call(dimNames_validate, object)
>> }
>> <bytecode: 0x11e7ca508>
>> <environment: namespace:Matrix>
>>
>> whereas if SeuratObject was built under Matrix >= 1.5-0, then we get:
>>
>> > getValidity(ns$.__C__Matrix)
>> function (object)
>> .Call(Matrix_validate, object)
>> <bytecode: 0x107dc1698>
>> <environment: namespace:Matrix>
>>
>> There are two "questions" here:
>>
>> 1.  The symbol 'Matrix_validate' is not defined until Matrix 1.5-0.
>>     Is it necessary, for this reason alone, for SeuratObject to have
>>     'Imports: Matrix (>= 1.5-0)'?  Or can SeuratObject continue using
>>     'Imports: Matrix (>= 1.3-3)', at the risk of errors like
>>
>>     > Error: object 'Matrix_validate' not found
>>
>>     (as already seen here: https://stackoverflow.com/questions/73700130)?
>>
>>     Note that this error would not occur for anyone installing 
>> SeuratObject
>>     from sources, unless they decide to _downgrade_ Matrix after doing 
>> so.
>>     Hence this primarily concerns Windows and macOS where R users would
>>     typically install a binary built by CRAN (i.e., not on their system).
>>
>>     We are aware that package TMB tests in .onLoad() that the current 
>> Matrix
>>     version is equal to or greater than the version available at build 
>> time,
>>     thus avoiding a "strict" version requirement, but do not want this 
>> practice
>>     to spread ...
>>
>> 2.  For how long should Matrix retain the superceded 'Dim_validate' and
>>     'dimNames_validate', in order to ensure that "stale" cached validity
>>     methods continue to work?
>>
>> We hope that this discussion will highlight the potential ramifications
>> of importing classes and methods from other packages, and having one's
>> classes and methods imported _by_ other packages, especially for version
>> requirements.
>>
>> Mikael, Martin, and Doug
>>
>> ______________________________________________
>> R-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> 

-- 
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics
 > E-mail is sent at my convenience; I don't expect replies outside of 
working hours.