[R-SIG-Mac] Linker warnings and error when building R on Apple silicon

[Mikael Jagan]

On 2022-02-10 1:42 pm, Mikael Jagan wrote:
> It seems my first message to mailing list, which had all of the attachments,
> was caught in the spam filter. In case it helps, I've dumped the config.site,
> config.log, and build.out on GitHub:
> 
> https://github.com/jaganmn/attachments/tree/6a7257014792f5e8cf7bb0bcd3bb27c7fabf7577/r-sig-mac/2022-02-10 
> 
> 
> On 2022-02-10 1:11 pm, Prof Brian Ripley wrote:
>> On 10/02/2022 17:28, Mikael Jagan wrote:
>>> Oops - forgot to mention: I am trying to build R-devel, currently at r81706.
>>>
>>> Mikael
>>>
>>> On 2022-02-10 12:13 pm, Mikael Jagan wrote:
>>>> Hello,
>>>>
>>>> I am running Big Sur on an M1 Mac and trying to build R from sources.
>>>> Some system details:
>>>>
>>>> R> sessionInfo()
>>>>  > Platform: aarch64-apple-darwin20 (64-bit)
>>>>  > Running under: macOS Big Sur 11.6.3
>>>>
>>>> $ clang --version
>>>>  > Apple clang version 13.0.0 (clang-1300.0.29.30)
>>>>  > Target: arm64-apple-darwin20.6.0
>>>>
>>>> $ /opt/R/arm64/gfortran/bin/gfortran --version
>>>>  > GNU Fortran (GCC) 11.0.0 20201219 (experimental)
>>>>
>>>> It has been a few months since I last built R on this machine.
>>>> At the time, I was building with LLVM clang rather than Apple clang
>>>> to obtain OpenMP support. I was recently nudged about the OpenMP
>>>> runtime library made available here:
>>>>
>>>> https://mac.r-project.org/openmp/
>>>>
>>>> which would allow me to build "normally" with Apple clang.
>>>> I am trying now to make the switch.
>>>>
>>>> I have installed all of the binaries hosted here
>>>>
>>>> https://mac.r-project.org/bin/darwin20/arm64/
>>>>
>>>> rather than those hosted here
>>>>
>>>> https://mac.r-project.org/libs-arm64/
>>>>
>>>> which I had previously installed. (Though: the first URL is missing
>>>> binaries for gfortran, pandoc, tcl, tk, and tkTable. I obtained
>>>> these from the second URL. _Both_ URLs are missing a zlib binary,
>>>> so I reluctantly obtained zlib from Homebrew.)
>>
>> zlib is part of the OS.  For pkg-config you need the zlib stub available from 
>> https://mac.r-project.org/libs-4/.  That *is* in the R-admin manual.  Also, 
>> that tcl, tk, and tkTable can be installed as part of an R binary installation.
> 
> Thanks - I wasn't sure whether that zlib stub was suitable for my system.
> 
>>
>> For pandoc I use the official Intel builds under emulation without any 
>> problems (including checking all of CRAN).  But you don't need pandoc to build R.
>>
>> I understand that the transition to https://mac.r-project.org/bin is work in 
>> progress, but its newer versions of libs are preferred.
>>
>>>>
>>>> Anyway, I'm running into trouble now... I have attached my
>>>> config.site, config.log, and build.out (from 'make >&') ...
>>>> config.log seems completely normal, but in build.out I see
>>>> many linker warnings of the form
>>>>
>>>>  > ld: warning: could not create compact unwind for _dchdc_: registers 72 
>>>> and 73 not saved contiguously in frame
>>>>  > ld: warning: could not create compact unwind for _dpoco_: registers 72 
>>>> and 73 not saved contiguously in frame
>>>>  > ld: warning: object file (/opt/R/arm64/lib/libreadline.a(history.o)) was 
>>>> built for newer macOS version (20.0) than being linked (11.0)
>>>>  > ld: warning: object file 
>>>> (/opt/R/arm64/lib/liblzma.a(liblzma_la-alone_decoder.o)) was built for newer 
>>>> macOS version (20.0) than being linked (11.0)
>>
>> Hmm, that's about a binary liblzma.  I'd download again (maybe try both 
>> sites), and if it is still wrong, build from the sources.
> If you search build.out, you'll find that this warning is issued
> for _every_ binary obtained from
> 
> https://mac.r-project.org/bin/darwin20/arm64/
> 
> I'll try the old site again.

Reverting to the old binaries resolves all of the version mismatch warnings,
so perhaps there is just a glitch in the way the new binaries were built.

> 
>>>> And the build eventually fails for a seemingly unrelated
>>>> reason:
>>>>
>>>>  > clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined 
>>>> dynamic_lookup -single_module -multiply_defined suppress 
>>>> -falign-functions=64 -g -O2 -Wall -pedantic 
>>>> -Wno-implicit-function-declaration -flto=thin -fPIC -Wl,-mllvm,-threads=4 
>>>> -L/opt/homebrew/opt/zlib/lib -L/opt/R/arm64/lib -L/usr/local/lib -lomp -o 
>>>> tools.so text.o init.o Rmd5.o md5.o signals.o install.o getfmts.o http.o 
>>>> gramLatex.o gramRd.o pdscan.o -Wl,-framework -Wl,CoreFoundation
>>>> mkdir ../../../../library/tools/libs
>>>>  > installing 'sysdata.rda'
>>>>  > Error in dyn.load(file, DLLpath = DLLpath, ...) :
>>>>  >   unable to load shared object 
>>>> '/Users/mikael/Desktop/R/R-devel-build/library/tools/libs/tools.so':
>>>>  > dlopen(/Users/mikael/Desktop/R/R-devel-build/library/tools/libs/tools.so, 
>>>> 6): Symbol not found: _INTEGER
>>>>  >   Referenced from: 
>>>> /Users/mikael/Desktop/R/R-devel-build/library/tools/libs/tools.so
>>>>  >   Expected in: flat namespace
>>>>  >  in /Users/mikael/Desktop/R/R-devel-build/library/tools/libs/tools.so
>>>>  > Error: unable to load R code in package 'tools'
>>>>  > Execution halted
>>>>
>>>> I've seen (and ignored) the "compact unwind" warnings before.
>>>> I've come across suggestions to append '-Wl,-no_compact_unwind'
>>>> to LDFLAGS, but it is not obvious to me that that is safe.
>>
>> They can be ignored: they are from the use of gfortran which does not fully 
>> use Apple conventions.
>>
>>>> The warnings about macOS version mismatch are new, and strange,
>>>> given that 20 is my Darwin version and 11 is my macOS version.
>>>> I have tried all of
>>>>
>>>> CC=clang
>>>> CC="clang -target arm64-apple-macos11"   [copied from an example in R-admin]
>>>> CC="clang -arch arm64"                   [copied from CRAN's Makeconf]
>>>>
>>>> (and similarly for CXX) to no avail. I can trick the linker
>>>> with '-target arm64-apple-macos20' but that doesn't seem safe
>>>> either.
>>
>> But that is not the issue, which is
>>
>>  >> dlopen(/Users/mikael/Desktop/R/R-devel-build/library/tools/libs/tools.so,
>>  >> 6): Symbol not found: _INTEGER
>>
>> Now tools.so should require that symbol, and it is linked to libR.dylib
>> which supplies it:
>>
>> auk2% nm -g library/tools/libs/tools.so | grep _INTEGER
>>                   U _INTEGER
>>
>> auk2% otool -L library/tools/libs/tools.so
>> library/tools/libs/tools.so:
>>      tools.so (compatibility version 0.0.0, current version 0.0.0)
>>      libR.dylib (compatibility version 4.2.0, current version 4.2.0)
>>      /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 
>> (compatibility version 150.0.0, current version 1856.105.0)
>>      /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 
>> 1311.0.0)
>>
>> auk2% nm -g lib/libR | grep _INTEGER
>> libR.dylib*       libRblas.dylib*   libRlapack.dylib*
>> auk2% nm -g lib/libR.dylib  | grep _INTEGER
>> 0000000000103490 T _INTEGER
>>
>> which comes from
>>
>> auk2% nm -g src/main/memory.o | grep _INTEGER
>> 00000000000066a8 T _INTEGER
>>
>> So my next step would be to investigate the building of libR.dylib.

Indeed, my proximal problem seems to be that libR.dylib wasn't built:

$ otool -L library/tools/libs/tools.so
library/tools/libs/tools.so:
	tools.so (compatibility version 0.0.0, current version 0.0.0)
	/usr/local/lib/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 
(compatibility version 150.0.0, current version 1856.105.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

$ ls lib
libRlapack.dylib

I don't see any indication of problems in the build output prior to
the loading of tools.so, so I'm not quite sure what to investigate...
Updated files are here:

https://github.com/jaganmn/attachments/tree/793f2e312d217a8c3898dce46b9c5dadad9fb615/r-sig-mac/2022-02-10

Mikael

>>
>> BTW, if one symbol is missing likely many are -- linkers usually only report 
>> the first one they encounter.
>>
>>>>
>>>> I'd be grateful for any hints on how to get things working,
>>>> again.
>>>>
>>>> Mikael
>>>
>>
>>