[Rd] New version of Rtools for Windows

Hin-Tak Leung htl10 at users.sourceforge.net
Thu Jan 8 20:27:20 CET 2015 

Oh, I forgot to mention that besides setting AR, RANLIB and the stack probing fix, you also need a very up to date binutils. 2.25 was out in december. Even with that , if you linker's default is not what you are compiling for (i.e. a multiarch toolchain), you need to set GNUTARGET also, i.e. -m32/-m64 is not enough. Some fix to autodetect non-default targets went in after christmas before the new year, but I am not brave enough to try that on a daily basis yet (only tested it and reported it, then reverting the change - how gcc invokes the linker is rather complicated and it is not easy to have two binutils installed...)- setting GNUTARGET seems safer :-).
Whether you need that depends on whether you are compiling for your toolchain's default target architecture.

AR, RANLIB, GNUTARGET are all environment variables - you set them the usual way. The stack probing fix is for passing "make check", when you finish make.

------------------------------
On Thu, Jan 8, 2015 6:14 PM GMT Avraham Adler wrote:

>On Thu, Jan 8, 2015 at 10:48 AM, Hin-Tak Leung
><htl10 at users.sourceforge.net> wrote:
>>
>> The r.dll crash is easy - you need to be using gcc-ar for ar, and gcc-ranlib for ranlib. I also posted a patch to fix the check failure for stack probing, as lto optimizes away the stack probing code, as it should.
>>
>> yes, lto build's speed gain is very impressive.
>>
>
>
>I apologize for my ignorance, but how would I do that? I tried by
>changing the following in src/gnuwin32/MkRules.local:
>
># prefix for 64-bit: path or x86_64-w64-mingw32-
>BINPREF64 = x86_64-w64-mingw32-gcc-
>
>I added the gcc- as the suffix there, but I guess that is insufficient
>as I still get the following error using 4.9.2:
>
>windres -F pe-x86-64  -I../include -i dllversion.rc -o dllversion.o
>gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
>psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
>system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
>../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
>../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
>../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
>../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
>-lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
>-lcomctl32 -lversion
>collect2.exe: error: ld returned 5 exit status
>Makefile:150: recipe for target 'R.dll' failed
>make[3]: *** [R.dll] Error 1
>Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>make[2]: *** [../../bin/x64/R.dll] Error 2
>Makefile:104: recipe for target 'rbuild' failed
>make[1]: *** [rbuild] Error 2
>Makefile:14: recipe for target 'all' failed
>make: *** [all] Error 2
>
>I still had to delete those lines in compat.c, so this build, were it
>to have completed, is still subject to the non-conformance of
>scientfic notation printing that was discussed earlier.
>
>Hin-tak, any suggestions for this error (and the compat.c for that
>matter) that you, or any reader of this list, may have would be
>greatly appreciated.
>
>Thank you!
>
>Avi
>
>
>> ------------------------------
>> On Thu, Jan 8, 2015 2:01 PM GMT Henric Winell wrote:
>>
>>On 2015-01-08 14:18, Avraham Adler wrote:
>>
>>> Very timely, as this is how I got into the problem I posted about
>>> earlier; maybe some of the problems I ran into will mean more to the
>>> you and the experts on this thread, Dr. Murdoch.For reference, I run
>>> Windows 7 64bit, and I am trying to build a 64 bit version of R-3.1.2.
>>>
>>> As we discussed offline, Dr. Murdoch, I've been trying to build R
>>> using more recent tools than GCC4.6.3 prerelease. Ruben Von Boxen
>>> (rubenvb) told me he is no longer developing his own builds of GCC,
>>> but is focusing on MSYS2 and the mingw64 personal builds. So, similar
>>> to what Jeroen said, I first installed MSYS2, whose initial
>>> installation on windows is not so simple[1]. After the initial
>>> install, the following packages need to be manually installed: make,
>>> tar, zip, unzip, zlib, and rsync. I also installed base-devel, which
>>> is way more than necessary, but there may be packages in there which
>>> are necessary.
>>>
>>> I originally installed the most up-to-date version of GCC (4.9.2)[2],
>>> and I did pick the -seh version, as since I install (almost) all
>>> packages from source (the one exception being nloptr for now), the
>>> exception handling should be consistent and it is supposed to up to
>>> ~15% faster[3].
>>>
>>> The initial build crashed with the following error:
>>>
>>> gcc -std=gnu99 -m64 -I../../include -I. -DHAVE_CONFIG_H  -O3 -Wall
>>> -pedantic -mtune=core2   -c xmalloc.c -o xmalloc.o
>>> ar crs libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o
>>> tre-match -approx.o tre-match-backtrack.o tre-match-parallel.o
>>> tre-mem.o tre-parse.o tre-stack.o xmalloc.o
>>> gcc -std=gnu99 -m64   -O3 -Wall -pedantic -mtune=core2   -c compat.c -o compat.o
>>> compat.c:65:5: error: redefinition of 'snprintf'
>>>   int snprintf(char *buffer, size_t max, const char *format, ...)
>>>       ^
>>> In file included from compat.c:3:0:
>>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:553:5: note: previous
>>> definition of 'snprintf' was here
>>>   int snprintf (char * __restrict__ __stream, size_t __n, const char *
>>> __restrict__ __format, ...)
>>>       ^
>>> compat.c:75:5: error: redefinition of 'vsnprintf'
>>>   int vsnprintf(char *buffer, size_t bufferSize, const char *format,
>>> va_list args)
>>>       ^
>>> In file included from compat.c:3:0:
>>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:543:7: note: previous
>>> definition of 'vsnprintf' was here
>>>     int vsnprintf (char * __restrict__ __stream, size_t __n, const char
>>> * __restrict__ __format, va_list __local_argv)
>>>         ^
>>> ../../gnuwin32/MkRules:218: recipe for target 'compat.o' failed
>>> make[4]: *** [compat.o] Error 1
>>> Makefile:120: recipe for target 'rlibs' failed
>>> make[3]: *** [rlibs] Error 1
>>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>>> make[2]: *** [../../bin/x64/R.dll] Error 2
>>> Makefile:104: recipe for target 'rbuild' failed
>>> make[1]: *** [rbuild] Error 2
>>> Makefile:14: recipe for target 'all' failed
>>> make: *** [all] Error 2
>>>
>>> After doing some checking (for example see [4]), I asked Duncan about
>>> the problem, and he suggested moving the #ifndef _W64 in compat.c up
>>> above the offending lines (65-75). That did not work, so, I figured
>>> (it seems mistakenly from the other thread) that if those functions
>>> are included from stdio already, I can just delete them from compat.c.
>>> The specific lines are:
>>>
>>> int snprintf(char *buffer, size_t max, const char *format, ...)
>>> {
>>>      int res;
>>>      va_list(ap);
>>>      va_start(ap, format);
>>>      res = trio_vsnprintf(buffer, max, format, ap);
>>>      va_end(ap);
>>>      return res;
>>> }
>>>
>>> int vsnprintf(char *buffer, size_t bufferSize, const char *format, va_list args)
>>> {
>>>      return trio_vsnprintf(buffer, bufferSize, format, args);
>>> }
>>>
>>> Continuing the build using 4.9.2 crashed again at the following point:
>>>
>>> gcc -std=gnu99 -m64 -I../include -I. -I../extra -DHAVE_CONFIG_H
>>> -DR_DLL_BUILD  -O3 -Wall -pedantic -mtune=core2   -c malloc.c -o
>>> malloc.o
>>> windres -F pe-x86-64  -I../include -i dllversion.rc -o dllversion.o
>>> gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>>> dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
>>> psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
>>> system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
>>> ../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
>>> ../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
>>> ../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
>>> ../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
>>> -lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
>>> -lcomctl32 -lversion
>>> collect2.exe: error: ld returned 5 exit status
>>> Makefile:150: recipe for target 'R.dll' failed
>>> make[3]: *** [R.dll] Error 1
>>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>>> make[2]: *** [../../bin/x64/R.dll] Error 2
>>> Makefile:104: recipe for target 'rbuild' failed
>>> make[1]: *** [rbuild] Error 2
>>> Makefile:14: recipe for target 'all' failed
>>> make: *** [all] Error 2
>>>
>>> As all those files existed in their correct places, the only reason I
>>> could think of that this would fail here is that GCC version 4.9 did
>>> make some changes to enhance link-time optimization [5], and probably
>>> something isn't compatible.
>>
>>Right.  Just before Christmas, Hin-Tak Leung reported build failure with
>>LTO:
>>
>>https://stat.ethz.ch/pipermail/r-devel/2014-December/070286.html
>>https://stat.ethz.ch/pipermail/r-devel/2014-December/070319.html
>>
>>
>>Many thanks to you and others for looking into this,
>>Henric
>>
>>
>>
>>> I then downgraded to GCC 4.8.4 [6], and,
>>> with the deletion of those 10 or so lines from compat.c, I can
>>> complete the build straight through rinstaller. However, I get that
>>> failure issue due to the extra 0 in scientific notation [7].
>>>
>>> It does not matter if I do the entire process in the MSYS2
>>> environment, or if I do in in Windows with msys\usr\bin in my path.
>>>
>>> Naïvely, it seems that if there were some what for stdio to be
>>> included in compat.c, yet the versions of snprintf and vsprintf in
>>> that file to "override" the standard, perhaps this method would work.
>>> Of course, running make check-all may uncover more issues. I intend to
>>> run the equivalent checks (from the tools library) inside of R with
>>> kill on failure turned off to see if anything else is problematic.
>>>
>>> Hopefully, something in this description resonates with one of the
>>> readers here. If anyone has any ideas as to how to circumvent the
>>> issues with compat.c, I'd be very grateful.
>>>
>>> Thank you,
>>>
>>> Avi
>>>
>>>
>>> [1] http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/
>>> [2] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.9.2/threads-win32/seh/x86_64-4.9.2-release-win32-seh-rt_v3-rev1.7z/download
>>> [3] https://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>>> [4] http://www.tt-forums.net/viewtopic.php?p=1034657&sid=613fa47a379ffaa0b9a9fb182a4180e3#p1034657
>>> [5] https://gcc.gnu.org/gcc-4.9/changes.html
>>> [6] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.8.4/threads-win32/seh/x86_64-4.8.4-release-win32-seh-rt_v3-rev0.7z/download
>>> [7] https://stat.ethz.ch/pipermail/r-devel/2015-January/070354.html
>>>
>>> Date: Wed, 07 Jan 2015 20:31:07 -0500
>>>
>>> From: Duncan Murdoch <murdoch.duncan at gmail.com>
>>> To: Jeroen Ooms <jeroenooms at gmail.com>
>>> Cc: "R-devel at r-project.org" <r-devel at r-project.org>
>>> Subject: Re: [Rd] New version of Rtools for Windows
>>> Message-ID: <54ADDDDB.4020500 at gmail.com>
>>> Content-Type: text/plain; charset=utf-8
>>>
>>> On 07/01/2015 5:20 PM, Jeroen Ooms wrote:
>>> On Wed, Jan 7, 2015 at 8:00 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
>>>
>>> This version includes only minor updates to the tools.  I indicated last summer that I was hoping to update GCC from the current version 4.6.3 before the R 3.2.0 release, but this now looks unlikely, unless someone else with experience building it can help.
>>>
>>> I have been looking into this a bit over the past few months, also
>>> with mixed success. Nevertheless, below some experiences that might be
>>> worth sharing.
>>>
>>> The guys from mingw-w64 recommended (quite strongly) to move away from
>>> multilib. They explained that the standard approach is to create two
>>> separate toolchains; one that targets win32 and the other one that
>>> targets win64 (both tool chains can compiled for win32). Hence the
>>> only difference for R would be that instead of passing "-m64" and
>>> "-m32", it would need to set the path to the proper compiler.
>>>
>>> There are several initiatives that provide very complete suites of
>>> precompiled mingw-w64 tools. I think the ideal scenario would be if we
>>> could take advantage of an existing tool chain as we do on other
>>> platforms, although perhaps I do not fully understand the R-specific
>>> requirements on the windows compiler.
>>>
>>> I feel quite strongly that we need to be able to build the toolchain,
>>> rather than relying on binaries produced by others.  We may need to
>>> customize the toolchain, or we may need to rebuild it when a bug is
>>> identified.  Lots of binary builders abandon their builds and you can't
>>> count on them to solve problems at a later date.
>>>
>>>
>>> One project that looks very promising is msys2 [1,2]. It has a package
>>> manager (port of pacman from arch linux) and comes with a pretty
>>> complete set of msys [3] and other [4] packages that seems quite well
>>> maintained.
>>>
>>> Do they post complete instructions for building?  That's what I'm
>>> looking for.  I don't want to develop a build script (I don't know how),
>>> but I would like to have one.
>>>
>>> Duncan Murdoch
>>>
>>>
>>> The only issue I ran into with msys2 is that it uses a different c++
>>> exception model (seh/dwarf) than the current Rtools (which uses sjlj).
>>> See also [5]. Therefore, if a library uses exceptions, we cannot use
>>> the current Rtools to link a static library that was created with
>>> msys2  [6]. I am not sure if it also be a problem the other way
>>> around, and if this is still the case for recent versions of
>>> gcc/mingw.
>>>
>>> Finally, Ruby has build very similar to Rtools called DevKit-mingw64
>>> [7] that we might be able to borrow from.
>>>
>>>
>>> [1] https://msys2.github.io/
>>> [2] http://stackoverflow.com/questions/25019057/how-are-msys-msys2-and-msysgit-related-to-each-other
>>> [3] https://github.com/Alexpux/MSYS2-packages
>>> [4] https://github.com/Alexpux/MINGW-packages
>>> [5] http://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>>> [6] http://stackoverflow.com/questions/7751640/undefined-reference-to-gxx-personality-sj0
>>> [7] http://rubyinstaller.org/downloads/
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>>

More information about the R-devel mailing list