[R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip

Gilberto Camara g||berto@c@m@r@ @end|ng |rom |npe@br
Sat Nov 23 05:36:41 CET 2024 

Dear Simon 

I understand your responses and your point. I hope you'll be able to understand my problem. It is hard to provide a simple working example, since we need to find a case where a package loads an OpenMP library that conflicts with the OpenMP library loaded by R. 

Replicating the problems I'm facing can be done by following the same actions I did. Please run the following commands on an R terminal to get the same error as we have using version R-4.4.2

== r code
% install.packages(“sits”)
% devtools::test(“sits”)
=== 

You should get the same error as I do. 

As I stated earlier, there is a conflict between “libomp.dylib" loaded in the R core libraries and the "libiomp5.dylib" loaded by the “torch” package.  Since our package (“sits”) uses the torch package,  I did the following experiment: (a) removed “libomp.dylib” from the R core libraries;  and (b) ran the same two lines above. Everything works. 

Meanwhile, I will try to produce an MWE that does not need to use the “sits” package. 

Best regards
Gilberto

============================
Prof Dr Gilberto Camara
Senior Researcher
Getulio Vargas Foundation (FGV)
National Institute for Space Research (INPE), Brazil
https://gilbertocamara.org/
=============================





> On 22 Nov 2024, at 20:54, Simon Urbanek <simon.urbanek using R-project.org> wrote:
> 
> Gilberto,
> 
> 
>> On Nov 23, 2024, at 12:40 PM, Gilberto Camara <gilberto.camara using inpe.br> wrote:
>> 
>> Dear Simon 
>> 
>> You state that “R  itself doesn't load OpenMP”. So why is libomp.dylib loaded in core libraries of R-4.4.2 for Mac? 
>> 
> 
> It is not loaded - you can check yourself:
> 
> echo '' | R CMD sh -c 'DYLD_PRINT_LIBRARIES=1 $R_HOME/bin/exec/R' 2>&1 | grep libomp
> 
> 
>> Please see the contents of the core R libaries for R-4.2 and R-4.4.2 in my MacOS. In both cases, these are downloads straight from CRAN. 
>> 
>> Why has libomp.dylib been included in the R core libraries? 
>> 
> 
> 
> We simply distribute it as a convenience, because some packages (like data.table) benefit hugely from OpenMP so by providing a fixed location it allows them to avoid shipping it separately and they all can use one shared version. See also https://mac.r-project.org/openmp/ for details
> 
> But as noted, I don't think this has anything to do with your problem precisely because it is not loaded by R itself.
> 
> As I said before I would appreciate is you avoided random speculations (there are many changes between R 4.2 and 4.4 especially in the packages between the two versions) as that is not productive, but instead share a reproducible example that we can look at. Any discussion is entirely pointless until you do that.
> 
> Cheers,
> Simon
> 
> 
> 
>> Best regards
>> Gilberto
>> 
>> 
>> 
>> R 4.2 
>> 
>> 
>> ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.2/Resources/lib
>> total 19976
>> drwxrwxr-x  15 root  admin   480B 22 Nov 15:28 .
>> drwxrwxr-x  18 root  admin   576B 22 Nov 15:28 ..
>> -rwxrwxr-x   1 root  admin   4,0M 17 Mar  2023 libR.dylib
>> drwxrwxr-x   3 root  admin    96B 17 Mar  2023 libR.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   221K 17 Mar  2023 libRblas.0.dylib
>> drwxrwxr-x   3 root  admin    96B 17 Mar  2023 libRblas.0.dylib.dSYM
>> lrwxrwxr-x   1 root  admin    16B 22 Nov 15:28 libRblas.dylib -> libRblas.0.dylib
>> drwxrwxr-x   3 root  admin    96B 17 Mar  2023 libRblas.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   151K 17 Mar  2023 libRblas.vecLib.dylib
>> drwxrwxr-x   3 root  admin    96B 17 Mar  2023 libRblas.vecLib.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   2,2M 17 Mar  2023 libRlapack.dylib
>> drwxrwxr-x   3 root  admin    96B 17 Mar  2023 libRlapack.dylib.dSYM
>> -rw-rw-r--   1 root  admin   157K 17 Mar  2023 libgcc_s.1.dylib
>> -rwxrwxr-x   1 root  admin   2,7M 17 Mar  2023 libgfortran.5.dylib
>> -rwxrwxr-x   1 root  admin   302K 17 Mar  2023 libquadmath.0.dylib
>> 
>> 
>> R 4.4.2
>> 
>> 
>> ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/lib
>> total 27320
>> drwxrwxr-x  17 root  admin   544B 22 Nov 15:03 .
>> drwxrwxr-x  17 root  admin   544B 22 Nov 15:03 ..
>> -rwxrwxr-x   1 root  admin   5,3M 31 Out 22:15 libR.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:14 libR.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   238K 31 Out 22:15 libRblas.0.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:14 libRblas.0.dylib.dSYM
>> lrwxrwxr-x   1 root  admin    16B 22 Nov 15:03 libRblas.dylib -> libRblas.0.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:14 libRblas.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   151K 31 Out 22:15 libRblas.vecLib.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:14 libRblas.vecLib.dylib.dSYM
>> -rwxrwxr-x   1 root  admin   2,3M 31 Out 22:15 libRlapack.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:14 libRlapack.dylib.dSYM
>> -rw-rw-r--   1 root  admin   183K 31 Out 22:15 libgcc_s.1.1.dylib
>> -rwxrwxr-x   1 root  admin   3,3M 31 Out 22:15 libgfortran.5.dylib
>> -rwxrwxr-x   1 root  admin   1,5M 31 Out 22:15 libomp.dylib
>> -rwxrwxr-x   1 root  admin   376K 31 Out 22:15 libquadmath.0.dylib
>> drwxrwxr-x   3 root  admin    96B 31 Out 22:11 pkgconfig
>> 
>> 
>> ============================
>> Prof Dr Gilberto Camara
>> Senior Researcher
>> Getulio Vargas Foundation (FGV)
>> National Institute for Space Research (INPE), Brazil
>> https://gilbertocamara.org/
>> =============================
>> 
>> 
>> 
>> 
>> 
>>> On 22 Nov 2024, at 19:48, Simon Urbanek <simon.urbanek using R-project.org> wrote:
>>> 
>>> Gilberto,
>>> 
>>> you still didn't provide any useful examples, and all you have are baseless speculations. R itself doesn't load OpenMP, so the problem could very well be in the other software or libraries you use - it may have nothing to do with R, but without an example we can neither tell nor help. Based on this there is nothing to solve "at the highest level of R core developers for Mac" - it has to start with you providing a working example as no one else has that problem.
>>> 
>>> Cheers,
>>> Simon
>>> 
>>> 
>>>> On Nov 23, 2024, at 7:30 AM, Gilberto Camara <gilberto.camara using inpe.br> wrote:
>>>> 
>>>> Dear Peter and Simon 
>>>> 
>>>> I may have found the cause of the crashes in R-4.4.2 in the Intel MacOs environment. In R-4.4.2 for MacOS, the “libomp.dylib” is loaded from the CRAN version. Comparing with earlier version, this is a new addition to the core R libraries. The OpenMP library is not provided in R-4.2. 
>>>> 
>>>> It so happens that the “torch” package is also including the an OpenMP library (“libiomp5.dylib”) which is causing the crash. In previous R versions, this behavoir did not cause problems, since R did not provide the OpenMP library by default. 
>>>> 
>>>> Thus, the problem needs to be solved at the highest level of R core developers for Mac. Please bear in mind that the “torch” package is essential for running deep learning algorithms in R. I hope you can find a solution that preserves the use of “torch” in R. I am copying the message to Daniel Falbel, who is the maintainer of the “torch” package.
>>>> 
>>>> Your support in solving this problem is highly appreciated.
>>>> 
>>>> Best regards
>>>> Gilberto
>>>> ============================
>>>> Prof Dr Gilberto Camara
>>>> Senior Researcher
>>>> Getulio Vargas Foundation (FGV)
>>>> National Institute for Space Research (INPE), Brazil
>>>> https://gilbertocamara.org/
>>>> =============================
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> On 22 Nov 2024, at 14:18, Gilberto Camara <gilberto.camara using inpe.br> wrote:
>>>>> 
>>>>> Dear Peter
>>>>> 
>>>>> Many thanks for your help. I ran R in a terminal, something I admit I had not done before.
>>>>> 
>>>>> It crashed and produced the following message:
>>>>> 
>>>>> ===
>>>>> OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
>>>>> OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
>>>>> ===
>>>>> 
>>>>> I am using Rcpp Armadillo, and I am following its guidelines for C++ compilation. The “Makevars” file is produced by running 
>>>>> 
>>>>>> usethis::use_rcpp_armadillo()
>>>>> 
>>>>> which produces the following Makevars file:
>>>>> 
>>>>> PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS)
>>>>> PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
>>>>> 
>>>>> I removed the Makevars file, compiled my package from scratch but the error continues. Any help on how to solve this problem would be most appreciated. 
>>>>> 
>>>>> Best regards
>>>>> Gilberto
>>>>> 
>>>>> ============================
>>>>> Prof Dr Gilberto Camara
>>>>> Senior Researcher
>>>>> Getulio Vargas Foundation (FGV)
>>>>> National Institute for Space Research (INPE), Brazil
>>>>> https://gilbertocamara.org/
>>>>> =============================
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 22 Nov 2024, at 12:08, peter dalgaard <pdalgd using gmail.com> wrote:
>>>>>> 
>>>>>> A couple of questions:
>>>>>> 
>>>>>> - Is it the GUI or R itself that crashes? (i.e, can you run  R in Terminal an still crash it? or even RStudio?)
>>>>>> - Is it a Sequoia issue as such? I'm not seeing issues on Monterey/4.4.1 (I'm a little superstitious about upgrading production machines mid-semester and some of my machines are too old for Sequoia.) The source tarballs 4.4.2 were built and tested on Intel/Monterey.
>>>>>> 
>>>>>> - pd
>>>>>> 
>>>>>>> On 21 Nov 2024, at 16:07 , Gilberto Camara <gilberto.camara using inpe.br> wrote:
>>>>>>> 
>>>>>>> Dear Simon 
>>>>>>> 
>>>>>>> Since your message last week, I have been trying to reproduce the errors I am finding with R-4.4.2 in an Intel MacMini to build a minimum testable example. I am not succeeding in doing so.
>>>>>>> 
>>>>>>> The error is a total collapse of R and occurs in a random fashion. Sometimes calling another package (e.g. “xgboost”) produces the error. Sometimes making a simple operation in a data table leads to error. 
>>>>>>> 
>>>>>>> The problem affects only Intel-based MacMinis with R-4.4.2. Running MacMinis with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with ARM, in Windows and in Lunix/Ubuntu and Linux/Fedora. 
>>>>>>> 
>>>>>>> One issue I noticed is that the default C++ compiler in MacOS Sequoia is compatible with c++-17, while the one used by CRAN is version c++-14.00.
>>>>>>> 
>>>>>>> Do you have any ideas on how to proceed? Any test data set I could use? 
>>>>>>> 
>>>>>>> Many thanks 
>>>>>>> Gilberto
>>>>>>> ============================
>>>>>>> Prof Dr Gilberto Camara
>>>>>>> Senior Researcher
>>>>>>> Getulio Vargas Foundation (FGV)
>>>>>>> National Institute for Space Research (INPE), Brazil
>>>>>>> https://gilbertocamara.org/
>>>>>>> =============================
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On 12 Nov 2024, at 18:53, Simon Urbanek <simon.urbanek using R-project.org> wrote:
>>>>>>>> 
>>>>>>>> Gilberto,
>>>>>>>> 
>>>>>>>> please read https://www.r-project.org/bugs.html first, in particular about how to report a problem. Just saying "cannot execute my scripts" is not helpful at all - please provide exact output, how it differs between the versions etc. For example, the problem may be in your code or packages used - we have no way of knowing without the necessary additional information.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Simon
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Nov 13, 2024, at 9:44 AM, Gilberto Camara <gilberto.camara using inpe.br> wrote:
>>>>>>>>> 
>>>>>>>>> Dear R-SIG-MAC
>>>>>>>>> 
>>>>>>>>> I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in a Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such problems occur with R-4.4.2 in MacBook with ARM chip. 
>>>>>>>>> 
>>>>>>>>> I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, MacOS X 15.1). 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> I will try to provide an MWE to help those of you who know a lot about Mac OS X.
>>>>>>>>> 
>>>>>>>>> Best
>>>>>>>>> Gilberto
>>>>>>>>> 
>>>>>>>>> ============================
>>>>>>>>> Prof Dr Gilberto Camara
>>>>>>>>> Senior Researcher
>>>>>>>>> Getulio Vargas Foundation (FGV)
>>>>>>>>> National Institute for Space Research (INPE), Brazil
>>>>>>>>> https://gilbertocamara.org/
>>>>>>>>> =============================
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> R-SIG-Mac mailing list
>>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> R-SIG-Mac mailing list
>>>>>>>> R-SIG-Mac using r-project.org
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>> 
>>>>>>> 
>>>>>>> [[alternative HTML version deleted]]
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> R-SIG-Mac mailing list
>>>>>>> R-SIG-Mac using r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>> 
>>>>>> -- 
>>>>>> Peter Dalgaard, Professor,
>>>>>> Center for Statistics, Copenhagen Business School
>>>>>> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>>>>>> Phone: (+45)38153501
>>>>>> Office: A 4.23
>>>>>> Email: pd.mes using cbs.dk  Priv: PDalgd using gmail.com
>>>>>> 
>>>>>> _______________________________________________
>>>>>> R-SIG-Mac mailing list
>>>>>> R-SIG-Mac using r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>> 
>>>>> _______________________________________________
>>>>> R-SIG-Mac mailing list
>>>>> R-SIG-Mac using r-project.org
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>> 
>>> 
>> 
> 
> _______________________________________________
> R-SIG-Mac mailing list
> R-SIG-Mac using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

More information about the R-SIG-Mac mailing list