[R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip

[Simon Urbanek]

Gilberto,

thanks! TLDR; its a bug in the torch package - fix below.

It took a while to sort out your example as it was very incomplete, here are the full required steps:

---
## sits doesn't install tests, so have the get them from sources:
download.packages("sits", type='source')
untar(Sys.glob("sits_*.tar.gz"))
## there are many other packages required for the tests
install.packages(c("sits","torch","lwgeom","randomForest","gdalcubes","xgboost","devtools"))

## also torch installation is incomplete so it requires additional downloads on first load
# (NB: this looks like a serious problem - does it mean torch bypasses CRAN checks?)
#> library(torch)
# - Additional software needs to be downloaded and installed for torch to work correctly.
#Do you want to continue? (Yes/no/cancel) yes

devtools::test(“sits”)
---

But all of it is not necessary since the key is that the tests load both data.table and torch so this is a more minimal example (based on one of the sits tests):

library(data.table)
library(sits)
sits_tuning(samples_modis_ndvi,
        ml_method = sits_tempcnn(),
        params = sits_tuning_hparams(epochs = 10, optimizer = torch::optim_adam,
            opt_hparams = list(lr = choice(0.01, 0.05, 0.001, 0.0005))),
        trials = 2,multicores = 1, progress = FALSE
)

The use of data.table is just an example, any OpenMP-enabled package will trigger the bug. As the error you sent indicated the problem is the incompatibility of the run-time used by torch. If you use any R package with OpenMP support (which loads the official OpenMP run-time - in above example it is data.table) and then run torch afterwards, torch will fail, because it loads its private OpenMP run-time which is incompatible. This is precisely why we provide a common run-time and torch fails to use it so I would recommend filing a bug report with the torch package. It is easily fixed with something like

system(
  paste("ln","-sfn",shQuote(file.path(R.home(),"lib","libomp.dylib")),
  shQuote(system.file("lib","libiomp5.dylib",package="torch"))))

to point torch to the common run-time which guarantees compatibility between packages. After that fix torch and the above example works again.

The current torch setup is really bad by design, because it essentially means that no one may use OpenMP as it will always clash with its private library (since no one else can use it). Even though it may not bite many people now given the sequence of packages involved, if we decided to enable OpenMP in CRAN build of R then torch won't work at all until this is fixed.

Cheers,
Simon


> On Nov 23, 2024, at 5:36 PM, Gilberto Camara <gilberto.camara using inpe.br> wrote:
> 
> Dear Simon 
> 
> I understand your responses and your point. I hope you'll be able to understand my problem. It is hard to provide a simple working example, since we need to find a case where a package loads an OpenMP library that conflicts with the OpenMP library loaded by R. 
> 
> Replicating the problems I'm facing can be done by following the same actions I did. Please run the following commands on an R terminal to get the same error as we have using version R-4.4.2
> 
> == r code
> % install.packages(“sits”)
> % devtools::test(“sits”)
> === 
> 
> You should get the same error as I do. 
> 
> As I stated earlier, there is a conflict between “libomp.dylib" loaded in the R core libraries and the "libiomp5.dylib" loaded by the “torch” package.  Since our package (“sits”) uses the torch package,  I did the following experiment: (a) removed “libomp.dylib” from the R core libraries;  and (b) ran the same two lines above. Everything works. 
> 
> Meanwhile, I will try to produce an MWE that does not need to use the “sits” package. 
> 
> Best regards
> Gilberto
> 
> ============================
> Prof Dr Gilberto Camara
> Senior Researcher
> Getulio Vargas Foundation (FGV)
> National Institute for Space Research (INPE), Brazil
> https://gilbertocamara.org/
> =============================
> 
>