[R-SIG-Mac] Change to BLAS (from default to Apple vecLib) doesn't report in sessionInfo()

Tomas Kalibera tom@@@k@||ber@ @end|ng |rom gm@||@com
Tue Jan 10 18:05:49 CET 2023 

On 1/10/23 04:21, Kevin Ushey wrote:
> My suspicion is the code here:
>
> https://github.com/wch/r-source/blob/068a6a0d9d60b1ce42abc879900038649f4a929c/src/main/platform.c#L3358-L3363
>
> It seems that the call to 'realpath()' fails on the path reported via
> .dli_fname here, which (for me) is:
>
> /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
>
> And indeed, the library does not appear to exist at that location. I'm
> not sure if this is a real macOS bug or some property of how vecLib /
> macOS frameworks work, though.
>
> For posterity, I'm on macOS Ventura 13.1, running on an M1 macOS machine.

This is a feature of macOS since Big Sur 11.0.1, it has a dynamic-linker 
cache of system libraries. The dynamic linker knows how to get the files 
from the cache, but one cannot see them using regular file-based 
utilities such as realpath. Yes, I think we can fall back to the 
argument of realpath() when it returns ENOENT (but we know the library 
should exist as we got it from the dynamic linker).

More here: 
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes

I can reproduce the problem on my system. Now, what is interesting is 
that when R is rebuilt after the libraries have already been installed 
(e.g. when I use an uninstalled version of R linking to 
libRblas.vecLib.dylib from an installed R-devel snapshot build), the 
linker in the detection code reports:

/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.vecLib.dylib

which works fine with realpath(). But, when R is not re-built after the 
library is re-installed, I see what has been reported. It is not clear 
to me why the difference and possibly it means that, after the fix, 
users would see a bit unpredictably one from these:

BLAS: 
/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.vecLib.dylib
BLAS: 
/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib

but that is still an improvement of what we get now and my intention was 
that the detection code output would be interpreted by a human, anyway.

Best
Tomas

>
>> sessionInfo()
> R version 4.2.1 (2022-06-23)
> Platform: aarch64-apple-darwin20 (64-bit)
> Running under: macOS Ventura 13.1
>
> Matrix products: default
> LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
>
> tl;dr: perhaps R should allow for failures in 'realpath()' above, at
> least on macOS, and report the library path as gleaned directly from
> dladdr()?
>
> Thanks,
> Kevin
>
>
> On Tue, Jan 10, 2023 at 8:54 AM Clement Kent <clementfkent using gmail.com> wrote:
>> I experienced the same thing. I switched to libRblas.vecLib.dylib and got
>> the remarkable speedup
>> in matrix multiplication, but sessionInfo doersn't show that I'm using it.
>> MacBook Pro (14-inch, 2021)
>>
>> sessionInfo()
>> R version 4.2.2 (2022-10-31)
>> Platform: aarch64-apple-darwin20 (64-bit)
>> Running under: macOS Monterey 12.6.2
>>
>> Matrix products: default
>> LAPACK:
>> /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
>>
>> Random number generation:
>>   RNG:     Mersenne-Twister
>>   Normal:  Inversion
>>   Sample:  Rounding
>>
>> locale:
>> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>>
>> attached base packages:
>> [1] datasets  parallel  utils     stats     graphics  grDevices methods
>> base
>>
>> other attached packages:
>> [1] showtext_0.9-5    showtextdb_3.0    sysfonts_0.8.8    data.table_1.14.6
>> nlme_3.1-160      MASS_7.3-58.1
>>
>> loaded via a namespace (and not attached):
>> [1] compiler_4.2.2  tools_4.2.2     grid_4.2.2      lattice_0.20-45
>> R 4.2.2 GUI 1.79 Big Sur ARM build (8160)
>>
>> On Mon, Jan 9, 2023 at 5:25 PM David Novgorodsky <
>> david.novgorodsky using gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> First post, and my apologies if this was addressed already elsewhere, but
>>> did not seem so to me. This post is (I think) purely about the
>>> behavior of sessionInfo()
>>> after I switch the BLAS to Apple's Accelerate vecLib.
>>>
>>> I installed R 4.2.2 on macOS Monterey 12.6.2 (with an M1 chip)
>>> successfully. I run sessionInfo() with the following result:
>>>
>>> R version 4.2.2 (2022-10-31)
>>>
>>> Platform: aarch64-apple-darwin20 (64-bit)
>>>
>>> Running under: macOS Monterey 12.6.2
>>>
>>>
>>>
>>> Matrix products: default
>>>
>>> BLAS:
>>>
>>> /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib
>>>
>>> LAPACK:
>>>
>>> /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
>>>
>>>
>>>
>>> locale:
>>>
>>> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>>>
>>>
>>>
>>> attached base packages:
>>>
>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>>
>>>
>>> loaded via a namespace (and not attached):
>>>
>>> [1] compiler_4.2.2
>>>
>>> And then I run a small piece of test code (taken from somewhere, sorry,
>>> can't recall where):
>>>
>>> N <- 20000
>>> M <- 2000
>>> X <- matrix(rnorm(N*M),N)
>>> system.time(crossprod(X))
>>>
>>> with result
>>>
>>>> system.time(crossprod(X))
>>> user  system  elapsed
>>> 49.471  0.082  49.553
>>>
>>> Then I change my BLAS (using the vecLib supplied as part of the R
>>> installation package), e.g., in the Terminal app:
>>>
>>> cd /Library/Frameworks/R.framework/Resources/lib
>>> ln -sf libRblas.vecLib.dylib libRblas.dylib
>>>
>>> Afterwards, I restart R. Re-running the small piece of test code above
>>> yields
>>>
>>>> system.time(crossprod(X))
>>> user  system  elapsed
>>> 0.339  0.015  0.197
>>>
>>> which suggests to me that the switch to vecLib did, indeed, succeed.
>>>
>>> However, sessionInfo() does something unexpected -- it no longer reports
>>> anything for the BLAS row:
>>>
>>> R version 4.2.2 (2022-10-31)
>>>
>>> Platform: aarch64-apple-darwin20 (64-bit)
>>>
>>> Running under: macOS Monterey 12.6.2
>>>
>>>
>>>
>>> Matrix products: default
>>>
>>> LAPACK:
>>>
>>> /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
>>>
>>>
>>>
>>> locale:
>>>
>>> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>>>
>>>
>>>
>>> attached base packages:
>>>
>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>>
>>>
>>> loaded via a namespace (and not attached):
>>>
>>> [1] compiler_4.2.2
>>>
>>> Anyone have any leads on whether a) this is a problem, and b) how to fix it
>>> (if so)? Seems this has been the case at least for a couple of years, at
>>> least based on an only tangentially related past thread (link here
>>> <https://stat.ethz.ch/pipermail/r-sig-mac/2020-November/013794.html>)
>>> where
>>> the responder implicitly shows the same thing -- faster matrix
>>> multiplication, but not reporting of BLAS in sessionInfo().
>>>
>>> Thanks,
>>> David
>>>
>>>          [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>
>>          [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

More information about the R-SIG-Mac mailing list