[Rd] R-4.2.3 build from source on Windows (w Rtools42) - lto1.exe error
Tomas Kalibera
tom@@@k@||ber@ @end|ng |rom gm@||@com
Fri May 12 09:50:05 CEST 2023
On 5/11/23 18:26, W. D. wrote:
> Thanks to everybody for the reples.
>
> tl;dr: it was a config problem with my environment being polluted by
> other apps (also via $USERPROFILE/.bashrc).
> R-4-2-branch builds with LTO "march=native" (Rtools42, also managed to
> build with "mtune=native", which afaik is inherently included when
> doing march, but still).
>
> Details: I will try to aggregate answers to comments/hints here and
> document a sequentially processed train of action here including
> results as some form of documentation for current status and hickups
> from my setup described in this email chain.
>
> @Avraham, et al
> Since I wasn't sure if and how maybe lib updates I had done before
> starting to build had played a role I downloaded a fresh copy of the
> Rtools42 installer from here
> [https://cran.r-project.org/bin/windows/Rtools/rtools42/files/]
> current version via `cat /x86_64-w6cat
> /x86_64-w64-mingw32.static.posix/.version` gave "5355"
> Did some more "tests":
> `cd R-4-2-branch/src/gnuwin32`
> Ran build with"mtune=native" gave same error
> Ran `make clean`
> Then using no EOPTS options in MkRules.local with version "5355" -> same error
> Ran `make clean`
> ---
> Then ran an update via `pacman -Syuu`
> Which updates the following:
> Packages (55) brotli-1.0.9-8 bsdtar-3.6.2-3
> ca-certificates-20230311-1 coreutils-8.32-5 curl-8.0.1-1
> dash-0.5.12-1 db-5.3.28-4 diffutils-3.9-1 file-5.44-5
> gawk-5.2.1-2 gcc-libs-11.3.0-3
> gnupg-2.2.41-1 grep-1~3.0-6 heimdal-libs-7.8.0-3
> info-7.0.3-1 less-633-1 libcrypt-2.1-4
> libcurl-8.0.1-1 libdb-5.3.28-4 libedit-20221030_3.1-1
> libexpat-2.5.0-1 libffi-3.4.4-1
> libgcrypt-1.10.2-1 libgnutls-3.8.0-1
> libgpg-error-1.47-1 libidn2-2.3.4-2 libksba-1.6.3-1
> liblzma-5.4.3-1 libnghttp2-1.52.0-1 libopenssl-3.1.0-2
> libpcre-8.45-3 libpsl-0.21.2-1
> libreadline-8.2.001-3 libsqlite-3.41.2-3
> libssh2-1.10.0-3 libunistring-1.1-2 libxml2-2.10.4-1
> libzstd-1.5.5-1 make-4.4.1-1 mpfr-4.2.0.p4-1
> msys2-keyring-1~20230316-1 ncurses-6.4-1
> openssl-3.1.0-2 patch-2.7.6-2 perl-5.36.0-1
> pinentry-1.2.1-1 rebase-4.5.0-4 rsync-3.2.7-2
> sed-4.9-1 tcl-8.6.12-3 texinfo-7.0.3-1
> texinfo-tex-7.0.3-1 xz-5.4.3-1 zlib-1.2.13-1 zstd-1.5.5-1
>
> But what confuses me a bit is that after that ` cat /x86_64-w6cat
> /x86_64-w64-mingw32.static.posix/.version` still only gives me "5355"?
> As will be shown later (below) this was not a breaking issue.
Rtools contains multiple separate things.
It contains Msys2, which you can update using pacman and which has its
own versioning. Msys2 is used for build tools and their dependencies
(like make) - so e.g. pcre from above is used by some build tool, not
linked to R packages.
It also contains a pre-built compiler toolchain and libraries, which
live in x86_64-w64-mingw32.static.posix and the .version file refers to
that. You can update this part also, but differently. These libraries
are linked to R packages, and you will find e.g. pcre there as well.
Then Rtools also contains several additional tools used by packages when
building/checking.
These things are described in the documentation in detail, for R 4.2
(and Rtools42) in
https://cran.r-project.org/bin/windows/base/howto-R-4.2.html
> Important change to my environment for the next attempt!
> I decided to check my ENV Variables (one more time) after thoroughly
> reading Tomas comment "maybe there is some config problem on the
> system" as well as Prof. Ripley's "first build without LTO to
> isolate the issue" and noticed that the old Rtools40/.../bin folder
> was also in my WINdows %PATH% ENV Variable.
> So I started a cleanup initiative from there.
> Also noticed that the .bashrc file in my %USERPROFILE% folder
> cluttered up my PATH especially there were `/mingw-64/bin` and
> `.../Library/bin` and similar entries from a miniconda3 installation
> amongst others ghosting around in there!
> Despite gcc --version or make commands not returning anything other
> than `command not found` I cleaned that .bashrc file up quite a bit.
> After cleanup I restarted MSYS2 Bash and ran `make distcelan` in the
> /src/gnuwin32 folder one more time then added miktex,
> /x86_64-w64-mingw32.static.posix/bin, tar, etc to $PATH or environment
> again.
> `echo $PATH` now (ater the extension of PATH var as dscibed in
> https://cran.r-project.org/bin/windows/base/howto-R-4.2.html)
> finally looked like this
> ```
> /c/Users/gwd/AppData/Local/Programs/MiKTeX/miktex/bin/x64:/x86_64-w64-mingw32.static.posix/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/c/progra~1/git/cmd:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
> ```
> with all other Library, bin or similar (mainly miniconda3) folders gone now.
> and
> `which make gcc pdflatex tar` looks like this now (not much different
> to before, but ...)
> ```
> make is /usr/bin/make
> make is /bin/make
> gcc is /x86_64-w64-mingw32.static.posix/bin/gcc
> pdflatex is /c/Users/gwd/AppData/Local/Programs/MiKTeX/miktex/bin/x64/pdflatex
> tar is /usr/bin/tar
> tar is /bin/tar
> tar is /c/Windows/System32/tar
> ```
> seems to look "better"!
>
> Then I started to build again ... which took me some time - that's why
> there's this delay in my answer here. ..
> First w/o any EOPTS -> succeeded!
> Then `make distclean`
> And then with march=native` [as well as mtune] -> succeeded!
>
> I will give the R-devel version with Rtools43 a shot as well (next)
> week - if you don't hear back from me -> you can assume that (also)
> worked correctly with LTO.
>
> So ... as is so often the case: "the problem was sitting in front of the PC".
> Thanks for the assistance and informative hints and sorry for
> bothering you all basically b/c of my (.bashrc) setup conundrum!
Good you figured it out. You can use your Msys2 installation with Rtools
if needed (also covered in the documentation). You surely might be able
to make it work even with other distribution of tools, but you'd be on
your own, likely have to deal with technical issues.
Best
Tomas
> Greetings,
> Walter
>
>
> On Thu, 11 May 2023 at 09:06, Tomas Kalibera <tomas.kalibera using gmail.com> wrote:
>>
>> On 5/11/23 03:07, Gmail wrote:
>>> Windows 11 PRO Version 10.0.22621 Build 22621
>>> Processor: Intel(R) Core(TM) i7-1065G7 "Icelake-client"
>>> ```
>>> svn info
>>> Path: .
>>> Working Copy Root Path: /d/R_DEV/R-4/R42/R-4-2-branch
>>> URL: https://svn.r-project.org/R/branches/R-4-2-branch
>>> Relative URL: ^/branches/R-4-2-branch
>>> Repository Root: https://svn.r-project.org/R
>>> Repository UUID: 00db46b3-68df-0310-9c12-caf00c1e9a41
>>> Revision: 84417
>>> Node Kind: directory
>>> Schedule: normal
>>> Last Changed Author: kalibera
>>> Last Changed Rev: 84249
>>> Last Changed Date: 2023-04-13 07:12:24 +0000 (Thu, 13 Apr 2023)
>>> ```
>>>
>>> Only adaptation done in MkRules.local was adding: `EOPTS = -march=native` - that's why I included the cpu-type info above;
>>> running make all recommended fails at/with:
>>> ```
>>> gcc -shared -s -static-libgcc -o utils.dll tmp.def init.o io.o size.o sock.o stubs.o utils.o hashtab.o windows/dataentry.o windows/dialogs.o windows/registry.o windows/util.o windows/widgets.o ../../../gnuwin32/dllversion.o -lRgraphapp -lversion -L/x86_64-w64-mingw32.static.posix/lib/x64 -llzma -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib -L../../../../bin/x64 -lR
>>>
>>> lto1.exe: fatal error: bytecode stream in file 'windows/dataentry.o' generated with LTO version 9.3 instead of the expected 9.4
>>> compilation terminated.
>>>
>>> lto-wrapper.exe: fatal error: C:\rtools42\x86_64-w64-mingw32.static.posix\bin\gcc.exe returned 1 exit status
>>> compilation terminated.
>>> C:\rtools42\x86_64-w64-mingw32.static.posix\bin/ld.exe: error: lto-wrapper failed
>>> collect2.exe: error: ld returned 1 exit status
>>> cp: cannot stat 'utils.dll': No such file or directory
>>> make[4]: *** [Makefile.win:36: shlib] Error 1
>>> make[3]: *** [../../../share/make/basepkg.mk:145: mksrc-win2] Error 1
>>> make[2]: *** [Makefile.win:24: all] Error 2
>>> make[1]: *** [Makefile.win:34: R] Error 1
>>> make: *** [Makefile:18: all] Error 2
>>> ```
>>> Any hints/ideas on how to fix this? I guess I could
>>> gcc -c -flto ... windows/dataentry.c -o windows/dataentry.o
>>> with the exact path of that fiile ...
>> All of the object files are generated on your system during the build.
>> If dataentry.o is generated using an older version than other object
>> files, maybe there is some configuration problem on the system. It might
>> be worth checking the compilers and linkers from Rtools42 are used, and
>> then running "make distclean", and then trying the build again.
>>
>> I never tried building R with LTO on Windows myself, I don't know if
>> that works even on a system set up according to the documentation for R
>> 4.2 (https://cran.r-project.org/bin/windows/base/howto-R-4.2.html).
>>
>>> and it hopefully will fix that but I guess it would make sense to add a Revision to update that LTO version mismatch there, and I don't know yet if this is the only one?
>> Perhaps it is better to use Rtools43 and R-devel if you can, so that if
>> you find some problem in either, it can still be fixed.
>>
>>> Greetings,
>>> W
>> Best,
>> Tomas
>>
>>> ______________________________________________
>>> R-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> ______________________________________________
>> R-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
More information about the R-devel
mailing list