[R-SIG-Mac] openmp and xcode

Thu Apr 24 14:40:52 CEST 2025

For version diversity, this returns false for me on R-4.4.3 (it’s not a 
4.5 thing).

(If this were linux, I’d strace it to see what’s going on. Since this is 
macos, I cannot get dtrace or execsnoop to get anywhere since SIP is 
enabled.)

Bill

On 4/23/25 11:52, bill+rsigmac using 8pawexpress.com wrote:

> Thanks Taras, I understand that, perhaps that portion (semver and big 
> changes on minor-looking version bumps) was more of a vent, I should 
> not have wasted the email-time on that :-)
>
> As I learn more about the dev required for macos, I’m impressed things 
> are a stable as they are, due to strong efforts by Simon and whomever 
> else helps on this side of things. It’s also because of this effort 
> and fragility that I recommend things like updating docs (for future 
> new-to-macos devs) and asking to grease the skids for doing things for 
> one’s self (by adding a package of same-version-headers). I think 
> having an |R-macos-dev-*.tar.gz| package of headers is consistent with 
> many/most OS and application packaging; the biggest question is if 
> there are other gotchas I’m not aware of that would make that a moving 
> target. (Perhaps it would have to be dependent on more than just the R 
> version … but I thought not since there is one compiled version of R 
> that covers most/all macos versions.)
>
> Cheers,
> Bill
>
> On 4/23/25 11:42, Taras Zakharko wrote:
>
>> Bill,
>>
>> Apple does not support OpenMP in their toolchain, and OpenMP is disabled in XCode. Simon and others rely on the fact that the toolchain is built on top of Clang and thus the OpenMP support can be patched in. However, as far as Apple goes all this is undocumented behavior and subject to change at any time. So I wouldn�t expect things improving any time soon.
>>
>> Best,
>>
>> Taras
>>
>> ________________________________
>> From: R-SIG-Mac<r-sig-mac-bounces using r-project.org> on behalf ofbill+rsigmac using 8pawexpress.com <bill+rsigmac using 8pawexpress.com>
>> Sent: 23 April 2025 15:33
>> To: Simon Urbanek<simon.urbanek using R-project.org>
>> Cc: r-mac<r-sig-mac using r-project.org>
>> Subject: Re: [R-SIG-Mac] openmp and xcode
>>
>> Simon,
>>
>> Long time fan :-)
>>
>> Downgrading to 16.2 worked. I�m baffled why a /minor/ release change was
>> sufficient to make it work, I had assumed that the major different (14
>> to 16) would have been more responsible. I guess they aren�t using
>> semver? Thank you for that advice, I�m on 16.2 now.
>>
>> I found 16.2 athttps://xcodereleases.com. While the release notes for
>> 16.3 are detailed (much of it beyond my familiarity), I don�t see
>> anything clear that would indicate to me that 16.2 would work and 16.3
>> would not. (For instance, no mention of OMP or OpenMP.) Besides you
>> having a running familiarity with the topic of R compilation on macos in
>> general, is there something that should have clued me in to suggest
>> 16.2? (Quick background: I�ve been on Linux and FOSS since 1993, when
>> floppies were the rage. I�ve used R on Windows and Linux since 2011. I
>> started on MacOS and therefore R on MacOS in Dec 2024. I�m definitely a
>> freshman on campus here, constantly befuddled by some of the nuanced
>> differences in macos.)
>>
>>      /will have to do a bit of legwork to make sure it works as it is not
>>      entirely trivial to do that correctly/
>>
>> My legwork has been educating. Can we ease the burden slightly? The docs
>> are clear about XCode 14.2/14.3, but since that version isn�t supported
>> on Sequoia there is no hint at a path forward other than to find this
>> mailing list and ask (you). The |/openmp/| portion of the website
>> discusses downloading one of 26 versions of |openmp-*.tar.gz|; it seems
>> risky to me that it would be wise to use a version of OpenMP that is
>> different than what R is linked against, ergo /�chaos when more OMP
>> run-times get loaded into one process�/.
>>
>> Perhaps I�m making a hasty assumption here � the R packaging includes
>> libraries for OMP, BLAS, and LAPACK, to name a few, and none of their
>> headers are included. I had assumed that including headers for all of
>> them would be useful, but I think you�re saying that you�d rather make
>> devs learn some of the nuances of building R against those libraries
>> before writing packages that link against them, is that right?
>>
>> How about two alternatives:
>>
>>   1.
>>
>>      Update documentation onhttps://mac.r-project.com to include a note
>>      about 16.3, perhaps as simple as �16.3 is known to break some
>>      packages�. This may be hampered if the �16.2� downgrade works for
>>      |data.table| but other packages would require perhaps �15.*� or
>>      �14.*� to compile.
>>
>>   2.
>>
>>      Provide a tarball that contains the header files for libraries
>>      against which a particular version of R is compiled. For instance,
>>      �R-headers-4.4.tar.gz� would include the OMP headers and possibly
>>      (I�m stretching) headers for BLAS and LAPACK. It would be an
>>      optional download that is updated when the linked libraries are
>>      updated, probably not at every R release.
>>
>>      Having a tarball of /just/ headers helps keep the headers the same
>>      version as the linked |libomp|, and still requires some legwork for
>>      the devs. I think this encourages safer version compatibility, and
>>      requires the dev to work harder if they need something further.
>>
>> Thoughts?
>>
>> Last oh-btw: you said you are using CLT 16.2, is that available from
>> Apple, are you using homebrew to install it, or something else? I�m not
>> yet tight on disk space, but I don�t use the rest of XCode so it seems
>> I�m wasting some space.
>>
>> Thanks,
>> Bill
>>
>> On 4/22/25 22:31, Simon Urbanek wrote:
>>
>>> Bill,
>>>
>>> the libomp.dylib inside R is there solely for the CRAN binaries to make sure that package binaries on CRAN can use the correct OpenMP runtime. Is it not intended to be used outside of that setup.
>>>
>>> That is also why we don�t ship the headers as anyone compiling from sources will have to do a bit of legwork to make sure it works as it is not entirely trivial to do that correctly. The CRAN build machines are setup such that it "just works" to make it easy on package maintainers. However, there are lot of issues/moving parts when building OpenMP-enabled code yourself (as you found out), so it is not encouraged unless you are very familiar with the issues (correct run-time and complier match-up etc.).
>>>
>>> That said, Xcode 16.3 is a big jump in the clang versions (from 17 to 19) so I would not recommend it just yet - have you tried Xcode 16.2? I tried to reproduce your problem and it worked just fine with Xcode 16.2 (CLT 16.2 to be precise and my corresponding OpenMP run-time from LLVM 17.0.6). Xcode 16.2 is supported on all macOS versions (and what the arm64 R builds are moving to) so I would suggest trying that first. (I didn't have CLT 16.3 installed yet, so I�ll have to wait for a quiet minute to see if that�s really what breaks.)
>>>
>>> Cheers,
>>> Simon
>>>
>>>
>>>> On 23 Apr 2025, at 02:21,bill+rsigmac using 8pawexpress.com wrote:
>>>>
>>>> If the CRAN R installation for macos includes |libomp.dylib|, is there a reason it does not include the paired |./include/omp*.h| files as well? It�s just three, they�re small, and it seems a reasonable step to making
>>>> building packages that use OpenMP more direct.
>>>>
>>>> Background: I�ve been having some difficulty compiling |data.table| (dev) on my MBP (M4, 15.4.1, R-4.4.3), due to an error (I do not assume it�s this package, I�ve had trouble with other packages too).
>>>>
>>>> |symbol not found in flat namespace &#x27;___kmpc_dispatch_deinit&#x27; |
>>>>
>>>> When troubleshooting this, I�ve verified that I have Xcode, but it is version 16.3, well past the 14.3 used for R-4.4.3. Sleuthing some more, I realized that it�s an easy mistake to update the |/usr/local/lib/libomp.dylib| file to be a different version than what is distributed in CRAN�s R. I don�t see an easy way to change version within XCode, and when I tried installing XCode-14.3 it says my OS does not support that (old) version. (Even with gfortran 12.2, I can�t fix that error, but that�s a different subject.)
>>>>
>>>> I recognize the challenge in distributing binaries for multiple versions of macos and xcode. I believe R-4.5 is still being built on XCode-14.3, which begs another question of if/when XCode updates are planned for R binaries and CRAN. (Big topic, I understand, I did not find a recent discussion. My apologies if I missed something.)
>>>>
>>>> Thanks,
>>>> Bill
>>>>
>>>> 
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-SIG-Mac mailing list
>>>> R-SIG-Mac using r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>
>> 
>>          [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>
>> 	[[alternative HTML version deleted]]
>>
>>
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>
> 

	[[alternative HTML version deleted]]