[R-SIG-Mac] Libre SSL bug on MacOS Monterey => error in download.file()

[Petr Bouchal]

Just an update: this particular issue (MacOS Libre SSL on Monterey vs. a particular cert/domain) is resolved in MacOS 12.3 beta (21E5206e) on M1 using the standard R 4.1.2 Arm64 binary. 

Secure connections to the previously problematic server are now opened correctly in {curl}, curl in shell as well as download.file() without needing to switch SSL backends. 

This is probably because the MacOS 12.3 beta comes with Libre SSL 3.3.5. Presumably this change resolves other similar issues with SSL on MacOS as well.

Kind regards
Petr


> On 13. 1. 2022, at 2:56, Kasper Daniel Hansen <kasperdanielhansen using gmail.com> wrote:
> 
> I am not an expert, but it seems to me that switching the backend is a
> runtime setting. Couldn't we detect which version of OS X we're running and
> then select the backend conditionally on that test?
> 
> Best,
> Kasper
> 
> On Wed, Jan 12, 2022 at 5:12 PM Jeroen Ooms <jeroenooms using gmail.com> wrote:
> 
>> On Wed, Jan 12, 2022 at 10:05 PM Simon Urbanek
>> <simon.urbanek using r-project.org> wrote:
>>> Yes, but if you are using an old version of R on a new system, you have
>> a lot of other worries - you can't expect new technologies to work with old
>> software. CURL itself has fewer evolution issues than SSL libraries. As I
>> said, I am a big proponent of re-using system libraries as much as
>> possible, but, for example, High Sierra doesn't ship with ST back-end
>> support, so using a static version that does is better there as Apple
>> doesn't not maintain the curl CAs but it does the system ones so it's
>> arguably better. The current issue is quite curious since breaking on the
>> latest system is quite unusual, just preferring ST works only because it is
>> the latest system that breaks and it has the ST option.
>>> 
>>> As Brian pointed out static curl has its own issues since its pkg-config
>> flags are broken - that's why I have not activated the add-on recipes by
>> default, I have seen those issues before.
>>> 
>>> For R itself there are thee options:
>>> 
>>> a) add CURL_SSL_BACKEND=${CURL_SSL_BACKEND-'SecureTransport'} to
>> $R_HOME/etc/Renviron of the distribution
>>> 
>>> b) add something like your
>> https://github.com/r-devel/r-svn/pull/75/commits/79b22b461e527e8a46de84c145e8e5fb59e75d14
>> to R
>>> 
>>> c) build against static libcurl
>>> 
>>> 
>>> The big advantage of the first one is that it applies to all processes,
>> so even command line curl will then work and so will all packages.
>>> 
>>> The drawback of the second one is that it only applies the R itself. The
>> third one could be done both for R and packages, but causes headaches resp.
>> requires slight patching of libcurl.pc. The advantage is that it can bring
>> more recent curl to all older systems.
>>> 
>>> I don't have a strong opinion. I am not thrilled with option b) because
>> that is a hack just to react to something which is never a good idea from
>> maintenance point of view (we would require all curl-based code to use it).
>> So I think a) and c) are more palatable with a) having the benefit of
>> handling non-R cases. A slight benefit of c) is that some dependencies
>> require more recent curl version than provided by older systems, so that
>> would cover it at the cost of maintaining the curl binaries. Finally, the
>> real benefit of c) is that if Apple screws things up even more we don't
>> care - we may not be at that point yet, though.
>> 
>> I don't think apple screwed up per se; they probably tested several
>> configurations and picked this one to be the safest default. TLS is a
>> complex protocol with many versions and implementations; if some weird
>> server uses some non-standard cipher or unusual response, it may just
>> depend on the TLS library if it can handle that. I'm sure you'll be
>> able to find counter examples where libre/openssl works and
>> SecureTransport does not. For example, a case that we often encounter
>> on Windows are corporate networks which require connecting via
>> authenticated proxy servers or using a TLS client cert, which only
>> works on certain back-ends, see the table in:
>> https://cran.r-project.org/web/packages/curl/vignettes/windows.html
>> 
>> So I much favor of option A. This introduces the least complexity, and
>> keeps the ability for users to undo our change and switch back to
>> CURL_SSL_BACKEND=openssl in their .Renviron. Also it is a big benefit
>> in practice that curl in R behaves the same as command line curl on
>> that same machine, in order to narrow down if a connection problem is
>> a bug in our R code, or if it also exists outside of R.
>> 
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>> 
> 
> 
> -- 
> Best,
> Kasper
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac