[R-pkg-devel] Best practices for built version checking in packages LinkingTo others?

[Pavel Krivitsky]

Dear All,

I think I have something that works and may even work as a more general
API for checking for LinkingTo ABI compatibility.

It's in 'ergm' 4.8.1 on CRAN, but would it be useful to anyone else if
I posted an explanation of how it works somewhere? Would a package that
provides this API for saving and checking ABI versions be useful to
anyone?

				Best,
				Pavel

On Sun, 2025-01-19 at 05:31 +0000, Pavel Krivitsky via R-package-devel
wrote:
> Thanks, Ivan! Replies below.
> 
> On Fri, 2025-01-17 at 16:50 +0300, Ivan Krylov wrote:
> > В Fri, 17 Jan 2025 10:21:27 +0000
> > Pavel Krivitsky via R-package-devel <r-package-devel using r-project.org>
> > пишет:
> > 
> > >    1. What is the best way to save this information? I have a
> > >       rudimentary implementation [1]
> > 
> > For now, this is the best way we have. Export the ABI version as an
> > include-time constant and as a registered callable. The LinkingTo
> > reverse dependencies can give the include-time constant to the
> > registered callable to compare the two versions.
> 
> Ben Bolker pointed out that since the code in R/ is executed at
> build-
> time, it is also possible to store the build-time ABI information
> entirely in R; the downside of this approach is that it requires
> instrumenting every client package rather than just the library
> package.
> 
> > I agree that we could do better and introduce an ABIversion: field
> > to
> > the DESCRIPTION file. R CMD INSTALL would write down the current
> > ABIversion of every package that the current package is LinkingTo
> > (into the installed package's DESCRIPTION or Meta/features.rds).
> > Later, the namespace loader would check the ABI versions and
> > provide
> > an informative message in case a LinkingTo dependency is updated
> > with
> > an incremented ABI version. A machine-readable list of ABI-level
> > dependencies should simplify the lives of the binary package
> > maintainers as well.
> 
> Yes, that would be great to have. (One nitpick: not just incremented,
> but different in either direction.)
> 
> AFAIK, for those platforms that use binaries, there is already a
> mechanism for downloading a different ZIP for a different version of
> R,
> but I believe it's accomplished simply by having a directory for
> every
> version of R with a potentially different ABI.
> 
> I suppose this could be generalised along the lines of a directory
> hierarchy of the form
> 
> /bin/windows/contrib/RVERSION/PKG_ABIVERSION/.../PKG_ABIVERSION/PKG_P
> KGVERSION.zip
> 
> with a nesting level for every LinkingTo package, in lexicographic
> order. Then, given the installed R version and the installed versions
> of each package in the LinkingTo list, a unique URL can be
> constructed.
> However, I don't know how practical this is and how likely to be
> implemented in the foreseeable future.
> 
> > >    2. How to best test the C API without making updates
> > > impossible?
> > 
> > Would abi-compliance-checker [2] have helped there?
> 
> I meant specifically in the context of R CMD check; the only want to
> fully test the C API and associated facilities is to actually install
> the client package and see if it works.
> 
> That having been said, 'ergm' may be unusual in this, since the API
> is
> mostly used as a way for others to implement calculation of graph
> statistics that are then operated on by functions in 'ergm'. That is,
> rather than simply offering C functions for the client packages to
> use,
> 'ergm' calls C functions from those packages that it locates via
> R_FindSymbol(). Thus, it is helpful to test it as a part of 'ergm'
> itself rather than just reverse-dependency checks.
> 
> > >    3. What should be done in case of a mismatch?
> > 
> > It's hard to guarantee anything when the ABI is broken. Maybe it's
> > completely harmless because your code will know that the new
> > structure member is not initialised. Maybe the wrong stack layout
> > will crash the process due to a return address mismatch, dooming
> > the
> > process from the first function call.k
> > 
> > >          1. Is there a way for 'ergm' to detect when a package
> > > LinkingTo it is being loaded?
> > 
> > With inline functions, you could make your every API call perform
> > ABI
> > version checking, but that obviously comes with potential
> > performance
> > problems:
> 
> The reason I ask is that the best time to inform the user of an ABI
> change is when the client package is loaded, and it would be helpful
> if
> this could be done without modifying the client package.
> 
> Is there a mechanism for the library package to monitor packages
> being
> loaded, and for each one check if it's LinkingTo it to do the version
> check?
> 
> > >          2. What should happen if a mismatch is detected? Should
> > > the
> > > loading be blocked (e.g., by having .onLoad() throw an error), or
> > > should it lead to a "use at your own risk" type warning?
> > 
> > The safest option would be to signal an error before any of the
> > potentially incompatible functions are called. It's probably
> > pragmatic to provide an "I know what I'm doing, void my warranty"
> > emergency override.
> 
> Probably an options() setting.
> 
> 				Best,
> 				Pavel
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel