[Rd] New version of Rtools for Windows

Hin-Tak Leung htl10 at users.sourceforge.net
Thu Jan 8 16:48:48 CET 2015 

The r.dll crash is easy - you need to be using gcc-ar for ar, and gcc-ranlib for ranlib. I also posted a patch to fix the check failure for stack probing, as lto optimizes away the stack probing code, as it should.

yes, lto build's speed gain is very impressive.



------------------------------
On Thu, Jan 8, 2015 2:01 PM GMT Henric Winell wrote:

>On 2015-01-08 14:18, Avraham Adler wrote:
>
>> Very timely, as this is how I got into the problem I posted about
>> earlier; maybe some of the problems I ran into will mean more to the
>> you and the experts on this thread, Dr. Murdoch.For reference, I run
>> Windows 7 64bit, and I am trying to build a 64 bit version of R-3.1.2.
>>
>> As we discussed offline, Dr. Murdoch, I've been trying to build R
>> using more recent tools than GCC4.6.3 prerelease. Ruben Von Boxen
>> (rubenvb) told me he is no longer developing his own builds of GCC,
>> but is focusing on MSYS2 and the mingw64 personal builds. So, similar
>> to what Jeroen said, I first installed MSYS2, whose initial
>> installation on windows is not so simple[1]. After the initial
>> install, the following packages need to be manually installed: make,
>> tar, zip, unzip, zlib, and rsync. I also installed base-devel, which
>> is way more than necessary, but there may be packages in there which
>> are necessary.
>>
>> I originally installed the most up-to-date version of GCC (4.9.2)[2],
>> and I did pick the -seh version, as since I install (almost) all
>> packages from source (the one exception being nloptr for now), the
>> exception handling should be consistent and it is supposed to up to
>> ~15% faster[3].
>>
>> The initial build crashed with the following error:
>>
>> gcc -std=gnu99 -m64 -I../../include -I. -DHAVE_CONFIG_H  -O3 -Wall
>> -pedantic -mtune=core2   -c xmalloc.c -o xmalloc.o
>> ar crs libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o
>> tre-match -approx.o tre-match-backtrack.o tre-match-parallel.o
>> tre-mem.o tre-parse.o tre-stack.o xmalloc.o
>> gcc -std=gnu99 -m64   -O3 -Wall -pedantic -mtune=core2   -c compat.c -o compat.o
>> compat.c:65:5: error: redefinition of 'snprintf'
>>   int snprintf(char *buffer, size_t max, const char *format, ...)
>>       ^
>> In file included from compat.c:3:0:
>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:553:5: note: previous
>> definition of 'snprintf' was here
>>   int snprintf (char * __restrict__ __stream, size_t __n, const char *
>> __restrict__ __format, ...)
>>       ^
>> compat.c:75:5: error: redefinition of 'vsnprintf'
>>   int vsnprintf(char *buffer, size_t bufferSize, const char *format,
>> va_list args)
>>       ^
>> In file included from compat.c:3:0:
>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:543:7: note: previous
>> definition of 'vsnprintf' was here
>>     int vsnprintf (char * __restrict__ __stream, size_t __n, const char
>> * __restrict__ __format, va_list __local_argv)
>>         ^
>> ../../gnuwin32/MkRules:218: recipe for target 'compat.o' failed
>> make[4]: *** [compat.o] Error 1
>> Makefile:120: recipe for target 'rlibs' failed
>> make[3]: *** [rlibs] Error 1
>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>> make[2]: *** [../../bin/x64/R.dll] Error 2
>> Makefile:104: recipe for target 'rbuild' failed
>> make[1]: *** [rbuild] Error 2
>> Makefile:14: recipe for target 'all' failed
>> make: *** [all] Error 2
>>
>> After doing some checking (for example see [4]), I asked Duncan about
>> the problem, and he suggested moving the #ifndef _W64 in compat.c up
>> above the offending lines (65-75). That did not work, so, I figured
>> (it seems mistakenly from the other thread) that if those functions
>> are included from stdio already, I can just delete them from compat.c.
>> The specific lines are:
>>
>> int snprintf(char *buffer, size_t max, const char *format, ...)
>> {
>>      int res;
>>      va_list(ap);
>>      va_start(ap, format);
>>      res = trio_vsnprintf(buffer, max, format, ap);
>>      va_end(ap);
>>      return res;
>> }
>>
>> int vsnprintf(char *buffer, size_t bufferSize, const char *format, va_list args)
>> {
>>      return trio_vsnprintf(buffer, bufferSize, format, args);
>> }
>>
>> Continuing the build using 4.9.2 crashed again at the following point:
>>
>> gcc -std=gnu99 -m64 -I../include -I. -I../extra -DHAVE_CONFIG_H
>> -DR_DLL_BUILD  -O3 -Wall -pedantic -mtune=core2   -c malloc.c -o
>> malloc.o
>> windres -F pe-x86-64  -I../include -i dllversion.rc -o dllversion.o
>> gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>> dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
>> psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
>> system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
>> ../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
>> ../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
>> ../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
>> ../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
>> -lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
>> -lcomctl32 -lversion
>> collect2.exe: error: ld returned 5 exit status
>> Makefile:150: recipe for target 'R.dll' failed
>> make[3]: *** [R.dll] Error 1
>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>> make[2]: *** [../../bin/x64/R.dll] Error 2
>> Makefile:104: recipe for target 'rbuild' failed
>> make[1]: *** [rbuild] Error 2
>> Makefile:14: recipe for target 'all' failed
>> make: *** [all] Error 2
>>
>> As all those files existed in their correct places, the only reason I
>> could think of that this would fail here is that GCC version 4.9 did
>> make some changes to enhance link-time optimization [5], and probably
>> something isn't compatible.
>
>Right.  Just before Christmas, Hin-Tak Leung reported build failure with
>LTO:
>
>https://stat.ethz.ch/pipermail/r-devel/2014-December/070286.html
>https://stat.ethz.ch/pipermail/r-devel/2014-December/070319.html
>
>
>Many thanks to you and others for looking into this,
>Henric
>
>
>
>> I then downgraded to GCC 4.8.4 [6], and,
>> with the deletion of those 10 or so lines from compat.c, I can
>> complete the build straight through rinstaller. However, I get that
>> failure issue due to the extra 0 in scientific notation [7].
>>
>> It does not matter if I do the entire process in the MSYS2
>> environment, or if I do in in Windows with msys\usr\bin in my path.
>>
>> Naïvely, it seems that if there were some what for stdio to be
>> included in compat.c, yet the versions of snprintf and vsprintf in
>> that file to "override" the standard, perhaps this method would work.
>> Of course, running make check-all may uncover more issues. I intend to
>> run the equivalent checks (from the tools library) inside of R with
>> kill on failure turned off to see if anything else is problematic.
>>
>> Hopefully, something in this description resonates with one of the
>> readers here. If anyone has any ideas as to how to circumvent the
>> issues with compat.c, I'd be very grateful.
>>
>> Thank you,
>>
>> Avi
>>
>>
>> [1] http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/
>> [2] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.9.2/threads-win32/seh/x86_64-4.9.2-release-win32-seh-rt_v3-rev1.7z/download
>> [3] https://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>> [4] http://www.tt-forums.net/viewtopic.php?p=1034657&sid=613fa47a379ffaa0b9a9fb182a4180e3#p1034657
>> [5] https://gcc.gnu.org/gcc-4.9/changes.html
>> [6] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.8.4/threads-win32/seh/x86_64-4.8.4-release-win32-seh-rt_v3-rev0.7z/download
>> [7] https://stat.ethz.ch/pipermail/r-devel/2015-January/070354.html
>>
>> Date: Wed, 07 Jan 2015 20:31:07 -0500
>>
>> From: Duncan Murdoch <murdoch.duncan at gmail.com>
>> To: Jeroen Ooms <jeroenooms at gmail.com>
>> Cc: "R-devel at r-project.org" <r-devel at r-project.org>
>> Subject: Re: [Rd] New version of Rtools for Windows
>> Message-ID: <54ADDDDB.4020500 at gmail.com>
>> Content-Type: text/plain; charset=utf-8
>>
>> On 07/01/2015 5:20 PM, Jeroen Ooms wrote:
>>> On Wed, Jan 7, 2015 at 8:00 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
>>>
>>> This version includes only minor updates to the tools.  I indicated last summer that I was hoping to update GCC from the current version 4.6.3 before the R 3.2.0 release, but this now looks unlikely, unless someone else with experience building it can help.
>>>
>>> I have been looking into this a bit over the past few months, also
>>> with mixed success. Nevertheless, below some experiences that might be
>>> worth sharing.
>>>
>>> The guys from mingw-w64 recommended (quite strongly) to move away from
>>> multilib. They explained that the standard approach is to create two
>>> separate toolchains; one that targets win32 and the other one that
>>> targets win64 (both tool chains can compiled for win32). Hence the
>>> only difference for R would be that instead of passing "-m64" and
>>> "-m32", it would need to set the path to the proper compiler.
>>>
>>> There are several initiatives that provide very complete suites of
>>> precompiled mingw-w64 tools. I think the ideal scenario would be if we
>>> could take advantage of an existing tool chain as we do on other
>>> platforms, although perhaps I do not fully understand the R-specific
>>> requirements on the windows compiler.
>>
>> I feel quite strongly that we need to be able to build the toolchain,
>> rather than relying on binaries produced by others.  We may need to
>> customize the toolchain, or we may need to rebuild it when a bug is
>> identified.  Lots of binary builders abandon their builds and you can't
>> count on them to solve problems at a later date.
>>
>>>
>>> One project that looks very promising is msys2 [1,2]. It has a package
>>> manager (port of pacman from arch linux) and comes with a pretty
>>> complete set of msys [3] and other [4] packages that seems quite well
>>> maintained.
>>
>> Do they post complete instructions for building?  That's what I'm
>> looking for.  I don't want to develop a build script (I don't know how),
>> but I would like to have one.
>>
>> Duncan Murdoch
>>
>>>
>>> The only issue I ran into with msys2 is that it uses a different c++
>>> exception model (seh/dwarf) than the current Rtools (which uses sjlj).
>>> See also [5]. Therefore, if a library uses exceptions, we cannot use
>>> the current Rtools to link a static library that was created with
>>> msys2  [6]. I am not sure if it also be a problem the other way
>>> around, and if this is still the case for recent versions of
>>> gcc/mingw.
>>>
>>> Finally, Ruby has build very similar to Rtools called DevKit-mingw64
>>> [7] that we might be able to borrow from.
>>>
>>>
>>> [1] https://msys2.github.io/
>>> [2] http://stackoverflow.com/questions/25019057/how-are-msys-msys2-and-msysgit-related-to-each-other
>>> [3] https://github.com/Alexpux/MSYS2-packages
>>> [4] https://github.com/Alexpux/MINGW-packages
>>> [5] http://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>>> [6] http://stackoverflow.com/questions/7751640/undefined-reference-to-gxx-personality-sj0
>>> [7] http://rubyinstaller.org/downloads/
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>

More information about the R-devel mailing list