[Rd] Sys.which() caching path to `which`

[Simon Urbanek]

Harmen,

thanks for the additional details, it wasn't exactly clear what this is about. Ivan's post didn't mention that the issue here is the caching, not the path replacement which you are apparently already doing, now it makes more sense.

I still think it is dangerous as you have no way of knowing who else is caching values at installation time since there is no reason to assume that the system will change after installation - technically, it breaks the contract with the application. We are trying to get packages to not hard-code or cache paths, but that typically only applies to the package library location, not to system tools.

Cheers,
Simon


> On Jan 11, 2024, at 10:36 AM, Harmen Stoppels <me using harmenstoppels.nl> wrote:
> 
> For context: I don't think Nix and Guix have to relocate anything, cause I think they require absolute paths like /nix/store where all binaries go. Spack on the other hand can install packages w/o sudo to a location of choice, e.g. ~/spack/opt/spack. That's why we have to patch binaries.
> 
> However, Spack's relocation stuff is not about creating truly relocatable binaries where everything is referenced by relative paths. We still use absolute paths almost everywhere, it's just that they have to be rewired when the location things are built is different from where they are installed.
> 
> I'm sure there are people who would like to have an actually fully relocatable R, but that's not my goal.
> 
>> I would claim it is not an unreasonable expectation that the user doesn't delete tools after R was built. Obviously it is tedious, but Spack may need to patch all absolute paths if it wants to relocate things (it's not easy as it includes all linker paths as well FWIW) - they must be doing something like that already as even the R start-up script uses absolute paths
> 
> Basically Spack does (a) special handling of the dynamic section of ELF files for Linux / FreeBSD, and load commands of mach-o files for macOS, (b) find & replace of prefixes in text files / scripts and (c) somewhat fancy but still basic replacement of C-strings containing prefixes in binaries.
> 
> This works reliably because the package prefixes contain hashes that make false positives unlikely.
> 
> It's just that it does not work when the absolute path to be relocated is captured inside serialized bytecode in a zlib-compressed database base.rdb :)
> 
> I believe `which` is the only occurrence of this.
> 
>> That said, WHICH is a mess - it may make sense to switch to the command -v built-in which is part of POSIX (where available - which is almost everywhere today) which would not require an external tool
> 
> That sounds like a decent solution to me, probably `command -v` is more commonly available than `which`.
> 
>> I don't think this is the only problem Spack has... (and that's just core R - even a bigger can of worms with R packages :P).
> 
> We can deal with most issues, just not with compressed byte code.
> 
>