[R-SIG-Mac] CRAN installer for macOS - directory permissions

[Patrick Schratz]

> They don't go there "silently" as in unnoticed - they go there if you instruct R to do so. That's why there is an explicit choice in the Installer. Otherwise regular R rules apply.

Where is this choice in the installer? I don’t see a menu/setting which users could change to install packages into a user lib instead of the system lib (if they are part of the `admin` group).
To me, they go there if the lib is writable - and “the common” R user does not know that the system lib is writeable by default.

> It only does so for admin users. Unlike "managed" unix systems, on macOS you have essentially two situations:
>
> On a "personal" machine (like laptop) the user is the main user and admin. Therefore it makes a lot more sense for the user to use a single location for managing packages which is at the system level. This also allows easy R upgrades. In addition, locations in user home raise a lot of issues (see the various discussion where this bites people on Windows) due to interactions with software that does mirroring, backups etc.. Hence this approach "just works" as one would expect on a Mac.

To be clear: I don’t question the system-wide installation and I think this is good as is (this also happens on Linux).
I am questioning the group write permissions for the system lib.

> If the user wishes to use his/her private library, it is trivial - just click on "At User Level" and from then on all packages are managed user's local library just like on any other platform.

I might be missing something obvious but where is this option “at the user level”? I assumed you’re talking about the official `.dmg` installer - which does not have such an option AFAICS?

> On a "managed" Mac the user is not an admin and therefore the behavior makes no difference. The status quo just makes it easier for admins to manage the shared library, but in reality this doesn't matter as one would assume the admins know what they are doing.

I disagree on this, especially the point “makes it easier for admins to manage the shared library”.
Admins should (and will) always be able to manage the system lib *after* authenticating (as on Linux). The authentication step does not really make a difference in practice for admins and is required in almost all places where system-wide changes are desired.

This is also the *core* of the whole discussion: 775 vs 755 (WRT to directory permissions).
I don’t see any (strong) reason that would result in 775 > 755.
(If so, then the default should probably also be changed on Linux.)
In a previous message, Kevin Ushey also agreed on the point that explicit authorization should be required to write into the system lib.
I assume there might be many more people who would actually agree on this 755 being preferable in this situation.
How can a proposal be phrased to reconsider this setting that is evaluated by a representative group of people?
I am not claiming to be right but I’d be interested in a multi-person evaluation of this setting rather than keeping this at a person-to-person discussion level.

> Well, having administered company-wide R installations in large companies for almost two decades I'd strongly disagree. As an admin you want as few user-installed packages as possible, because they are guaranteed to cause problems. You want to limit this for things like development of packages where you want the stable version globally and development version locally (and this is not just me - have a look how the top tech companies manage their software). You have a reliable, stable central location - if you don't do that then you'll have n libraries to manage for n users which is absolutely not sustainable as users will break their libraries and you cannot even upgrade R. Also having a central, shared library is crucial for collaboration. Unlike in Python in R it actually works since R and CRAN doesn't allow randomly breaking reverse-dependencies.

As a system engineer and admin myself (for several “large” companies/institutions), I kindly disagree on your view here.
User packages are not a problem but *a feature*, everyone can install the versions they need for their project.
They don’t interfere with packages from other users and are not forced go with the update interval of an admin.
With the additional use of renv (thanks Kevin!), redundancy is highly reduced as a shared cache can be used from with users can simply use symlinks rather than installing the x-th copy of the same R package version. But this is partly off-topic WRT to the actual discussion.

Overall this sub-discussion part might come down to the philosophy of having a “centrally managed, unflexible admin installation” or a “centrally managed, partly-flexible admin installation” where only the R versions and system libs are managed but users have the freedom to install any R packages they want.
Also in “my” philosophy, it’s not about “upgrading” R and removing the previous version but adding new versions as they come in and keeping previous ones - for the purpose of reproducibility.
I usually keep the latest patch version of a minor version and aim to provide a consistent R environment for various minor versions where users are guaranteed to be able to work with that minor version in a flexible way (i.e. by installing user packages as they want) for many years ahead.

> As mentioned before and above I disagree. The proposal doesn't matter for managed Macs but would negatively affect users that are single-user admins and since that is typically the case for the majority of Mac R users (as they typically are on their personal machines) I don't see any upshot. All it would do is to prevent typical R users to install packages directly.

How would it affect single-user admins in a negative way?
They can

- still install packages per R minor version into a dedicated user library
- install multiple R minor versions side by side
- actually enjoy the same behavior as on Linux

> All it would do is to prevent typical R users to install packages directly.

I don’t understand this point. It would behave similar as on Linux, where users are prompted to create a user library (on first use and if non exists yet).

As you can see, the overall discussion topic is quite important to me and I am still convinced that the current state on macOS is suboptimal.
Thanks for your time and sharing your thoughts.

Cheers
Patrick

On 25 Apr 2022, at 1:46, Simon Urbanek wrote:

> Patrick,
>
> sorry fo the delayed reply - this was not a quick e-mail so I had to find time after the release :)
>
>
>> On Apr 3, 2022, at 8:26 PM, Patrick Schratz <patrick.schratz using gmail.com> wrote:
>>
>> Hi Simon,
>>
>> thanks for your extensive reply.
>>
>> The choice is deliberate: the admin group on macOS corresponds to users that are allowed to install system-wide software so it allows all admins on the machine to install packages which is the expected way on macOS.
>>
>> I think this choice is unfortunate as it contrasts with existing behavior on other platforms where one needs to explicitly request admin privileges by either using sudo or starting R as an admin.
>> On macOS, packages just go into the system lib “silently” because of the write permissions granted via the admin group.
>> See also my comments further down for more details on this.
>>
>
> They don't go there "silently" as in unnoticed - they go there if you instruct R to do so. That's why there is an explicit choice in the Installer. Otherwise regular R rules apply.
>
>
>> Also the versioning of the R framework as x.y is also deliberate - upgrading R to a new patch version does *not* require re-installation of packages, they work by design so in fact the system location is the safest way to do that. Also note that packages are never removed by the installer.
>>
>> Thanks, I am aware that a patch update does not require a reinstallation as the packages are functional across minor versions.
>>
>> I checked again and was indeed wrong, patch updates from the CRAN installer do not remove additional custom packages in the system lib.
>> I was confused by some custom approaches of mine which cause this behavior - sorry for this!
>>
>> So out of the items listed in "The problem" they are all not true with the exception of the comparison with the other platforms, but even that difference is very subtle as it only affects the default on the first installation and not regular use (and I'm, not even sure it that is true since admin users can still install in the system location on other platforms).
>>
>> On Linux you would need to do explicitly invoke sudo R to allow writing into the system lib.
>> The issue on macOS is that a regular start of R allows system lib write permissions.
>> In my current view I think a similar behavior as on Linux would be great, i.e. to explicitly having to request admin privileges for this task.
>>
>
> It only does so for admin users. Unlike "managed" unix systems, on macOS you have essentially two situations:
>
> On a "personal" machine (like laptop) the user is the main user and admin. Therefore it makes a lot more sense for the user to use a single location for managing packages which is at the system level. This also allows easy R upgrades. In addition, locations in user home raise a lot of issues (see the various discussion where this bites people on Windows) due to interactions with software that does mirroring, backups etc.. Hence this approach "just works" as one would expect on a Mac. If the user wishes to use his/her private library, it is trivial - just click on "At User Level" and from then on all packages are managed user's local library just like on any other platform.
>
> On a "managed" Mac the user is not an admin and therefore the behavior makes no difference. The status quo just makes it easier for admins to manage the shared library, but in reality this doesn't matter as one would assume the admins know what they are doing.
>
>
>> I don’t understand the part “as it only affects the default on the first installation and not regular use” of your reply - could you clarify this?
>> Unless a user creates a user lib manually, packages will always go into the system lib - not only on first use - but I might be misunderstanding your comment here.
>>
>> I would argue that the current setup tends to be a lot safer than the alternatives, because it allows commonly used packages to be installed at the system level and private packages to be installed at user level. This is also the design typically used on shared machines, where you separate local packages from user packages where local ones are installed by administrators - so exactly the same setup. Moreover R upgrades are a lot cleaner, since you can easily upgrade all system packages at once so you don't have to worry about individual users having stale packages - the biggest problem for admins.
>>
>> Yes and no.
>> Sometimes system admins want to install certain packages globally - however, I never do so for the following reason:
>> Often this will lead to multiple package installations, i.e. one in the syslib and one in the user lib (if the user installs the package again for some reason which quite often happens).
>> Often these duplicated packages will have different versions and users are confused which one is actually loaded (the user lib one is as it is first in the path).
>>
>> Aside from this practical point, Macs are rarely used in a shared way.
>> And even if, I’d highly favor having to request write permissions into the syslib rather it happening by default.
>>
>> Imagine a scenario where the admin of a shared Mac constantly writes into the syslib (because this is the default). This syslib is then also used by other non-admin users on the system.
>> I don’t think this is a desired scenario and might cause lot’s of confusion (not even mentioning the fact if all people in this scenario are aware what’s going on given that this is a niche topic).
>> Here I think a one-time central installation of R and then only working with user libs (as on Linux) would be preferable.
>>
>
> Well, having administered company-wide R installations in large companies for almost two decades I'd strongly disagree. As an admin you want as few user-installed packages as possible, because they are guaranteed to cause problems. You want to limit this for things like development of packages where you want the stable version globally and development version locally (and this is not just me - have a look how the top tech companies manage their software). You have a reliable, stable central location - if you don't do that then you'll have n libraries to manage for n users which is absolutely not sustainable as users will break their libraries and you cannot even upgrade R. Also having a central, shared library is crucial for collaboration. Unlike in Python in R it actually works since R and CRAN doesn't allow randomly breaking reverse-dependencies.
>
>
>> From a technical perspective, I know that setting root:root on macOS is not possible. My proposed change to 755 (and leaving root:admin) would however exactly mimic this (and the one of Linux installs) behavior:
>>
>> 	• admins would need to do sudo R to install into the system library
>> 	• otherwise they are prompted to create a user library
>> Which downsides would this approach have? Currently I don’t see any. It would even harmonize CRAN installer behavior across platforms.
>>
>> I'd be happy to hear from more Mac user if there are reasons to change the default, but as I outlined the choices were deliberate after weighting the pros and cons. In my view the major issue with the proposal it that is would prevent sharing of packages, make R upgrades a lot harder and prevent admin users from using the current tools for package management - and that includes the ability to separate system and user packages on single-user machines.
>>
>> I’ll try to vision the practical changes of this:
>>
>> 	• Patch update experience would not change as custom packages will be in the user lib for the respective minor version (by default)
>> 	• Admins are still able to install into the system lib when using sudo R
>> 	• AFAICS admins will still be able to separate system and user packages as they can use sudo R for syslib installs. To me, the proposed change would even make the behaviour more clear than before (which requires to create a hidden folder (user lib) in the right place to actually use a user lib).
>> Let me know if I overlook something - but currently I don’t see any downside but various positive impacts.
>>
>
> As mentioned before and above I disagree. The proposal doesn't matter for managed Macs but would negatively affect users that are single-user admins and since that is typically the case for the majority of Mac R users (as they typically are on their personal machines) I don't see any upshot. All it would do is to prevent typical R users to install packages directly.
>
>
>> Last, I wanted to ask if the source code for the CRAN installer is publicly available? I could not find it and would be interested to take a look into it. If this is not possible for some reason, I would also be interested in getting to know the reason for this decision.
>>
>
>
> Everything is in the R SVN, the R build and release system is in
> https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4
> and Apple Installer packaging is in
> https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging
> and the relevant postflight script is in
> https://svn.r-project.org/R-dev-web/trunk/QA/Simon/R4/packaging/scripts-R-fw/postflight
>
>
>> On Apr 13, 2022, at 8:43 PM, Patrick Schratz <patrick.schratz using gmail.com> wrote:
>>
>> Related to this Q: Are the macOS CRAN policies actively discussed by a team of people (who might eventually also be willing to share their thoughts or could be addressed with such questions) or are you solely responsible for it?
>>
>
>
> CRAN is an entire team, so yes, but as for anything Mac-related it includes R-core and other stake holders that have expressed interest before (e.g. Bioconductor). Obviously, this (R-SIG-Mac) is also a good place as that includes anyone who cares about R on macOS.
>
> Cheers,
> Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20220430/25e81cb4/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 870 bytes
Desc: OpenPGP digital signature
URL: <https://stat.ethz.ch/pipermail/r-sig-mac/attachments/20220430/25e81cb4/attachment-0001.sig>