[R-SIG-Mac] Libre SSL bug on MacOS Monterey => error in download.file()

[Petr Bouchal]

Dear all,

In brief: on Monterey, R cannot reach certain web domains due to a bug in Libre SSL - and perhaps not relying on system curl/openssl in R would be a systematic solution to this and símilar issues.

Specifically: on MacOS Monterey 12.1 using R 4.1.2, download.file() and other functions that rely on system-provided curl/openssl/Libre SSL (including in the curl package) have been failing on specific domains. 

So running 

download.file(“https://www.czso.cz/”, tempfile()) 

returns: 

status was ‘SSL connect error’

the underlying error being
 
error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length.

This is caused by the Libre SSL bundled in MacOS Monterey and also affects several other domains, most notably https://libzip.org.

It is clearly an OS bug but infortunately also a situation where it affects R users because of how R relates to system libraries and is very difficult to work around.

It has manifested on CRAN (causing a package archival) and Github outside of R, so is not caused by a specific machine. It can be replicated on both M1 and Intel and also occurs when using curl in the system command line. 

The czso.cz domain is the Czech Statistical Office, which makes it quite important for a number of users, also of a package I maintain (czso) which relies on accessing this domain. I have reported this to the server admin but since the problem is in the OS, I do not expect them to be able to help. I am not an expert in web security so cannot tell if there is anything in the certificates which could be causing this. In browsers, no such issue occurs and the server is configured correctly as per ssllabs.com testing. I have also reported to Apple but it is unclear whether they will fix this given the rare nature of the issue.

It is difficult to work around even on individual machines as replacing the system curl/openssl requires steps beyond what a most users are comfortable with (or should be doing to begin with). Using HTTP instead of HTTPS does not work, nor does using curl —insecure and equivalents.

This brings back the question of whether R on MacOS should include its own openssl instead of relying on the system-provided library. This has been discussed on the r-devel list: https://stat.ethz.ch/pipermail/r-devel/2020-June/079657.html.

Apple also recommends against relying on shared openssl, if I understand this correctly: https://developer.apple.com/forums/thread/89051. Given Apple’s approach to openssl/Libre SSL in MacOS (the bundled Libre SSL version is 3 years old), such hard-to-handle issues are likely to reappear over time. (I don’t have in-depth knowledge of how R is compiled, so apologies for any inaccuracies; hopefully it is clear what I mean.)

I’d be grateful for any thoughts on how this might be handled in the specific case and perhaps generally.

Kind regards
Petr Bouchal