[R-SIG-Mac] CRAN installer for macOS - directory permissions

Wed Jun 8 11:05:18 CEST 2022

>>>>> Henrik Bengtsson 
>>>>>     on Fri, 3 Jun 2022 10:45:26 -0700 writes:

    > Hi Duncan, thank you for the follow-up.
    >> You didn't address the issue of having different versions of a package
    >> installed in the system library versus the personal library, an issue
    >> that causes lots of confusion for unsophisticated users.

    > I think this is a problem that applies to all operating systems and is
    > not specific to macOS. Whatever the solution, or best practices, is
    > for that, I think it should be addressed uniformly across platforms.
    > I have, some experience and thoughts around this too, but wouldn't it
    > be better if we move that discussion to R-devel?

    > /Henrik

I (as lurker) agree.
The problem and solutions definitely also apply (even more)
to other "Unix derivatives" than just macOS, notably to Linux
where it is probably much more common to have multi-user systems
than on the Mac ((but what do I know about Macs?)).

Martin

    > On Fri, Jun 3, 2022 at 10:38 AM Duncan Murdoch <murdoch.duncan using gmail.com> wrote:
    >> 
    >> On 03/06/2022 1:21 p.m., Peter Dalgaard wrote:
    >> > Eek! No, just no...
    >> >
    >> > That way users can't upgrade a package for himself on a multi-user system if the sysop won't do it for everyone.
    >> 
    >> "by default"
    >> 
    >> Duncan Murdoch
    >> 
    >> >
    >> > - pd
    >> >
    >> >> On 3 Jun 2022, at 19:01 , Duncan Murdoch <murdoch.duncan using gmail.com> wrote:
    >> >>
    >> >> Hi Henrik.
    >> >>
    >> >> You didn't address the issue of having different versions of a package installed in the system library versus the personal library, an issue that causes lots of confusion for unsophisticated users.
    >> >>
    >> >> A possible solution for this is for R by default to refuse to install a package in the personal library if it is already installed in the system library.
    >> >>
    >> >> Duncan Murdoch
    >> >>
    >> >> On 03/06/2022 12:45 p.m., Henrik Bengtsson wrote:
    >> >>> I see two fairly big problems with users installing R packages to
    >> >>> .Library by default.  One is related to package checking and CRAN, and
    >> >>> one is related to translation of expectations when moving between
    >> >>> operating systems (as Patrick already pointed out).  At the end, I'll
    >> >>> also argue that R_LIBS_SITE exists for those who wish to maintain
    >> >>> site-wide R package libraries to be shared among users, which is
    >> >>> better than using .Library for this.
    >> >>> # R CMD check
    >> >>> When you check a package with 'R CMD check --as-cran', or, with
    >> >>> environment variable `_R_CHECK_DEPENDS_ONLY_` set to true, the checks
    >> >>> are run in a sandbox where only declared package dependencies and any
    >> >>> packages in the system package library (= .Library) are on the library
    >> >>> path (= .libPaths()), e.g.
    >> >>> print(.libPaths())
    >> >>> [1] "/tmp/alice/RtmpYDq3KF/RLIBS_2410b74eb16752"
    >> >>> [2] "/path/to/R-4.2.0/lib/R/library"
    >> >>> What's in the user's library (= R_LIBS_USER) or in the site library (=
    >> >>> .Library.site/R_LIBS_SITE) is not part of the testing.  This mechanism
    >> >>> is very valuable since it helps to identify undeclared package
    >> >>> dependencies.
    >> >>> **The default behavior on macOS discussed here, where R packages are
    >> >>> installed to .Library, breaks this.**  Developers with non-base R
    >> >>> packages in .Library will not benefit from the 'R CMD check --as-cran'
    >> >>> checks for undeclared packages. This increases the risk of them not
    >> >>> being aware of the problem of undeclared packages, which is a
    >> >>> discussion we see from time to time on R-devel and R-pkg-devel, e.g.
    >> >>> when it comes to what should be listed under Suggests: or not.
    >> >>> BTW, this makes me wonder how many macOS developers notice this
    >> >>> problem only as they submit to CRAN, and have to resubmit. Also, this
    >> >>> issue might add extra work to the CRAN Team, e.g. spending time
    >> >>> locking at and responding to possible false positives, handling more
    >> >>> emails, and handling more re-submissions.
    >> >>> # Social expectations
    >> >>> The second problem with the current default macOS behavior is when
    >> >>> people hop between systems and operating systems.  Particularly, a
    >> >>> macOS user coming to Unix or Windows does not immediately understand
    >> >>> how and where R packages are installed.  They get a prompt about a
    >> >>> "personal library" and might choose to decline because it's not what
    >> >>> they're used to seeing.  Then they might end up in the Stack Overflow
    >> >>> cut'n'paste rabbit hole, where they find some instructions on setting
    >> >>> 'R_LIBS_USER=$HOME/R-lib' without version specifiers.  Works fine
    >> >>> until they upgrade R next year, when they start getting weird warnings
    >> >>> or errors of some packages not working that they slowly start to
    >> >>> accept as the normal behavior of R. I see this problem on large HPC
    >> >>> environments where I help out thousands of HPC users. Also, reading
    >> >>> various support forums out there, I think this is a real problem. It's
    >> >>> only recently, thanks to Patrick, I learned about this rather odd
    >> >>> macOS behavior, and I do think it is a cause for confusion and
    >> >>> miscommunication.  Another problem with different OS behaviors is that
    >> >>> it complicated documentation and instructions.  I strongly believe, it
    >> >>> would be beneficial to the R community if we all have the same
    >> >>> experience and expectations regardless of OS.
    >> >>> I believe the above problems are best addressed by changing the
    >> >>> *default* settings on macOS so that it is *not* possible to install to
    >> >>> .Library, and instead require a user to install to their personal
    >> >>> package library.  Advanced users who prefer to install to .Library,
    >> >>> can still configure R, or .Library, to do so.
    >> >>> As Patrick suggests, defaulting .Library to 755, instead of 775, or
    >> >>> avoid setting the "admin" group, seems like a simple solution that
    >> >>> would help harmonize the user experience of R.
    >> >>> # Maintaining a site-wide package library
    >> >>>> I would argue that the current setup tends to be a lot safer than the alternatives, because it allows commonly used packages to be installed at the system level and private packages to be installed at user level. This is also the design typically used on shared machines, where you separate local packages from user packages where local ones are installed by administrators - so exactly the same setup. Moreover R upgrades are a lot cleaner, since you can easily upgrade all system packages at once so you don't have to worry about individual users having stale packages - the biggest problem for admins.
    >> >>> I believe .Library.site/R_LIBS_SITE exists to address this purpose,
    >> >>> and I argue it is better suited for this than using .Library.  It also
    >> >>> has the benefit of *not* conflicting with the
    >> >>> _R_CHECK_DEPENDS_ONLY_=true checks (Problem #1 above).  It also allows
    >> >>> users to opt out from centrally installed packages (by setting
    >> >>> R_LIBS_SITE=""), which they cannot do if they're installed in the
    >> >>> system library.
    >> >>> All the best,
    >> >>> Henrik
    >> >>> PS. FWIW, for all of the above reasons, I always set 755 on .Library,
    >> >>> even if I install R to my personal home folder; it simply lowers the
    >> >>> risk for side effects when developing and troubleshooting.
    >> >>> On Thu, Jun 2, 2022 at 11:50 PM Patrick Schratz
    >> >>> <patrick.schratz using gmail.com> wrote:
    >> >>>>
    >> >>>> I’d like to bump this thread again, especially since I am interested in a reply from Simon (or any other core member) on the arguments raised in the most recent messages of this thread.
    >> >>>>
    >> >>>> What would be the best way for this idea/proposal going forward?
    >> >>>>
    >> >>>> I’ve found https://contributor.r-project.org/, which links to a public Slack channel. Would this be a better place to discuss this matter?
    >> >>>> Or https://github.com/r-devel/rcontribution?
    >> >>>>
    >> >>>> Best,
    >> >>>> Patrick_______________________________________________
    >> >>>> R-SIG-Mac mailing list
    >> >>>> R-SIG-Mac using r-project.org
    >> >>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
    >> >>> _______________________________________________
    >> >>> R-SIG-Mac mailing list
    >> >>> R-SIG-Mac using r-project.org
    >> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
    >> >>
    >> >> _______________________________________________
    >> >> R-SIG-Mac mailing list
    >> >> R-SIG-Mac using r-project.org
    >> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
    >> >
    >>