[Rd] New version of Rtools for Windows
Avraham Adler
avraham.adler at gmail.com
Fri Jan 9 08:54:51 CET 2015
Regarding the redefinition error, I've asked on StackOverflow for
advice [1], but I have noticed the following; perhaps someone here can
understand what changed between the stdio.h of 4.6.3 and the stdio.h
of 4.8.4. In GCC 4.8.4, the section of stdio.h which is referenced in
the errors is the following:
#if !defined (__USE_MINGW_ANSI_STDIO) || __USE_MINGW_ANSI_STDIO == 0
/* this is here to deal with software defining
* vsnprintf as _vsnprintf, eg. libxml2. */
#pragma push_macro("snprintf")
#pragma push_macro("vsnprintf")
# undef snprintf
# undef vsnprintf
int __cdecl __ms_vsnprintf(char * __restrict__ d,size_t n,const char
* __restrict__ format,va_list arg)
__MINGW_ATTRIB_DEPRECATED_MSVC2005 __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
__mingw_ovr
__MINGW_ATTRIB_NONNULL(3)
int vsnprintf (char * __restrict__ __stream, size_t __n, const char
* __restrict__ __format, va_list __local_argv)
{
return __ms_vsnprintf (__stream, __n, __format, __local_argv);
}
int __cdecl __ms_snprintf(char * __restrict__ s, size_t n, const
char * __restrict__ format, ...);
#ifndef __NO_ISOCEXT
__mingw_ovr
__MINGW_ATTRIB_NONNULL(3)
int snprintf (char * __restrict__ __stream, size_t __n, const char *
__restrict__ __format, ...)
{
register int __retval;
__builtin_va_list __local_argv; __builtin_va_start( __local_argv, __format );
__retval = __ms_vsnprintf (__stream, __n, __format, __local_argv);
__builtin_va_end( __local_argv );
return __retval;
}
#endif /* !__NO_ISOCEXT */
#pragma pop_macro ("vsnprintf")
#pragma pop_macro ("snprintf")
#endif
The corresponding section in 4.6.3 as found in the Rtools for Windows
installation is:
#if !defined (__USE_MINGW_ANSI_STDIO) || __USE_MINGW_ANSI_STDIO == 0
/* this is here to deal with software defining
* vsnprintf as _vsnprintf, eg. libxml2. */
#pragma push_macro("snprintf")
#pragma push_macro("vsnprintf")
# undef snprintf
# undef vsnprintf
int __cdecl vsnprintf(char * __restrict__ d,size_t n,const char *
__restrict__ format,va_list arg)
__MINGW_ATTRIB_DEPRECATED_MSVC2005 __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
#ifndef __NO_ISOCEXT
int __cdecl snprintf(char * __restrict__ s, size_t n, const char *
__restrict__ format, ...);
#ifndef __CRT__NO_INLINE
__CRT_INLINE int __cdecl vsnprintf(char * __restrict__ d,size_t
n,const char * __restrict__ format,va_list arg)
{
return _vsnprintf (d, n, format, arg);
}
#endif /* !__CRT__NO_INLINE */
#endif /* !__NO_ISOCEXT */
#pragma pop_macro ("vsnprintf")
#pragma pop_macro ("snprintf")
#endif
The latter does not have a direct redefinition of the two functions. I
still don't know why the #undef calls do not work [1].
Thank you,
Avi
[1] https://stackoverflow.com/questions/27853225/is-there-a-way-to-include-stdio-h-but-ignore-some-of-the-functions-therein
On Thu, Jan 8, 2015 at 2:27 PM, Hin-Tak Leung
<htl10 at users.sourceforge.net> wrote:
> Oh, I forgot to mention that besides setting AR, RANLIB and the stack probing fix, you also need a very up to date binutils. 2.25 was out in december. Even with that , if you linker's default is not what you are compiling for (i.e. a multiarch toolchain), you need to set GNUTARGET also, i.e. -m32/-m64 is not enough. Some fix to autodetect non-default targets went in after christmas before the new year, but I am not brave enough to try that on a daily basis yet (only tested it and reported it, then reverting the change - how gcc invokes the linker is rather complicated and it is not easy to have two binutils installed...)- setting GNUTARGET seems safer :-).
> Whether you need that depends on whether you are compiling for your toolchain's default target architecture.
>
> AR, RANLIB, GNUTARGET are all environment variables - you set them the usual way. The stack probing fix is for passing "make check", when you finish make.
>
> ------------------------------
> On Thu, Jan 8, 2015 6:14 PM GMT Avraham Adler wrote:
>
>>On Thu, Jan 8, 2015 at 10:48 AM, Hin-Tak Leung
>><htl10 at users.sourceforge.net> wrote:
>>>
>>> The r.dll crash is easy - you need to be using gcc-ar for ar, and gcc-ranlib for ranlib. I also posted a patch to fix the check failure for stack probing, as lto optimizes away the stack probing code, as it should.
>>>
>>> yes, lto build's speed gain is very impressive.
>>>
>>
>>
>>I apologize for my ignorance, but how would I do that? I tried by
>>changing the following in src/gnuwin32/MkRules.local:
>>
>># prefix for 64-bit: path or x86_64-w64-mingw32-
>>BINPREF64 = x86_64-w64-mingw32-gcc-
>>
>>I added the gcc- as the suffix there, but I guess that is insufficient
>>as I still get the following error using 4.9.2:
>>
>>windres -F pe-x86-64 -I../include -i dllversion.rc -o dllversion.o
>>gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>>dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
>>psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
>>system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
>>../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
>>../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
>>../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
>>../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
>>-lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
>>-lcomctl32 -lversion
>>collect2.exe: error: ld returned 5 exit status
>>Makefile:150: recipe for target 'R.dll' failed
>>make[3]: *** [R.dll] Error 1
>>Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>>make[2]: *** [../../bin/x64/R.dll] Error 2
>>Makefile:104: recipe for target 'rbuild' failed
>>make[1]: *** [rbuild] Error 2
>>Makefile:14: recipe for target 'all' failed
>>make: *** [all] Error 2
>>
>>I still had to delete those lines in compat.c, so this build, were it
>>to have completed, is still subject to the non-conformance of
>>scientfic notation printing that was discussed earlier.
>>
>>Hin-tak, any suggestions for this error (and the compat.c for that
>>matter) that you, or any reader of this list, may have would be
>>greatly appreciated.
>>
>>Thank you!
>>
>>Avi
>>
>>
>>> ------------------------------
>>> On Thu, Jan 8, 2015 2:01 PM GMT Henric Winell wrote:
>>>
>>>On 2015-01-08 14:18, Avraham Adler wrote:
>>>
>>>> Very timely, as this is how I got into the problem I posted about
>>>> earlier; maybe some of the problems I ran into will mean more to the
>>>> you and the experts on this thread, Dr. Murdoch.For reference, I run
>>>> Windows 7 64bit, and I am trying to build a 64 bit version of R-3.1.2.
>>>>
>>>> As we discussed offline, Dr. Murdoch, I've been trying to build R
>>>> using more recent tools than GCC4.6.3 prerelease. Ruben Von Boxen
>>>> (rubenvb) told me he is no longer developing his own builds of GCC,
>>>> but is focusing on MSYS2 and the mingw64 personal builds. So, similar
>>>> to what Jeroen said, I first installed MSYS2, whose initial
>>>> installation on windows is not so simple[1]. After the initial
>>>> install, the following packages need to be manually installed: make,
>>>> tar, zip, unzip, zlib, and rsync. I also installed base-devel, which
>>>> is way more than necessary, but there may be packages in there which
>>>> are necessary.
>>>>
>>>> I originally installed the most up-to-date version of GCC (4.9.2)[2],
>>>> and I did pick the -seh version, as since I install (almost) all
>>>> packages from source (the one exception being nloptr for now), the
>>>> exception handling should be consistent and it is supposed to up to
>>>> ~15% faster[3].
>>>>
>>>> The initial build crashed with the following error:
>>>>
>>>> gcc -std=gnu99 -m64 -I../../include -I. -DHAVE_CONFIG_H -O3 -Wall
>>>> -pedantic -mtune=core2 -c xmalloc.c -o xmalloc.o
>>>> ar crs libtre.a regcomp.o regerror.o regexec.o tre-ast.o tre-compile.o
>>>> tre-match -approx.o tre-match-backtrack.o tre-match-parallel.o
>>>> tre-mem.o tre-parse.o tre-stack.o xmalloc.o
>>>> gcc -std=gnu99 -m64 -O3 -Wall -pedantic -mtune=core2 -c compat.c -o compat.o
>>>> compat.c:65:5: error: redefinition of 'snprintf'
>>>> int snprintf(char *buffer, size_t max, const char *format, ...)
>>>> ^
>>>> In file included from compat.c:3:0:
>>>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:553:5: note: previous
>>>> definition of 'snprintf' was here
>>>> int snprintf (char * __restrict__ __stream, size_t __n, const char *
>>>> __restrict__ __format, ...)
>>>> ^
>>>> compat.c:75:5: error: redefinition of 'vsnprintf'
>>>> int vsnprintf(char *buffer, size_t bufferSize, const char *format,
>>>> va_list args)
>>>> ^
>>>> In file included from compat.c:3:0:
>>>> F:/MinGW64/x86_64-w64-mingw32/include/stdio.h:543:7: note: previous
>>>> definition of 'vsnprintf' was here
>>>> int vsnprintf (char * __restrict__ __stream, size_t __n, const char
>>>> * __restrict__ __format, va_list __local_argv)
>>>> ^
>>>> ../../gnuwin32/MkRules:218: recipe for target 'compat.o' failed
>>>> make[4]: *** [compat.o] Error 1
>>>> Makefile:120: recipe for target 'rlibs' failed
>>>> make[3]: *** [rlibs] Error 1
>>>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>>>> make[2]: *** [../../bin/x64/R.dll] Error 2
>>>> Makefile:104: recipe for target 'rbuild' failed
>>>> make[1]: *** [rbuild] Error 2
>>>> Makefile:14: recipe for target 'all' failed
>>>> make: *** [all] Error 2
>>>>
>>>> After doing some checking (for example see [4]), I asked Duncan about
>>>> the problem, and he suggested moving the #ifndef _W64 in compat.c up
>>>> above the offending lines (65-75). That did not work, so, I figured
>>>> (it seems mistakenly from the other thread) that if those functions
>>>> are included from stdio already, I can just delete them from compat.c.
>>>> The specific lines are:
>>>>
>>>> int snprintf(char *buffer, size_t max, const char *format, ...)
>>>> {
>>>> int res;
>>>> va_list(ap);
>>>> va_start(ap, format);
>>>> res = trio_vsnprintf(buffer, max, format, ap);
>>>> va_end(ap);
>>>> return res;
>>>> }
>>>>
>>>> int vsnprintf(char *buffer, size_t bufferSize, const char *format, va_list args)
>>>> {
>>>> return trio_vsnprintf(buffer, bufferSize, format, args);
>>>> }
>>>>
>>>> Continuing the build using 4.9.2 crashed again at the following point:
>>>>
>>>> gcc -std=gnu99 -m64 -I../include -I. -I../extra -DHAVE_CONFIG_H
>>>> -DR_DLL_BUILD -O3 -Wall -pedantic -mtune=core2 -c malloc.c -o
>>>> malloc.o
>>>> windres -F pe-x86-64 -I../include -i dllversion.rc -o dllversion.o
>>>> gcc -std=gnu99 -m64 -shared -s -mwindows -o R.dll R.def console.o
>>>> dynload.o editor.o embeddedR.o extra.o opt.o pager.o preferences.o
>>>> psignal.o rhome.o rt_complete.o rui.o run.o shext.o sys-win32.o
>>>> system.o dos_wglob.o malloc.o ../main/libmain.a ../appl/libappl.a
>>>> ../nmath/libnmath.a getline/gl.a ../extra/xdr/libxdr.a
>>>> ../extra/pcre/libpcre.a ../extra/bzip2/libbz2.a
>>>> ../extra/intl/libintl.a ../extra/trio/libtrio.a ../extra/tzone/libtz.a
>>>> ../extra/tre/libtre.a ../extra/xz/liblzma.a dllversion.o -fopenmp -L.
>>>> -lgfortran -lRblas -L../../bin/x64 -lRzlib -lRgraphapp -lRiconv
>>>> -lcomctl32 -lversion
>>>> collect2.exe: error: ld returned 5 exit status
>>>> Makefile:150: recipe for target 'R.dll' failed
>>>> make[3]: *** [R.dll] Error 1
>>>> Makefile:179: recipe for target '../../bin/x64/R.dll' failed
>>>> make[2]: *** [../../bin/x64/R.dll] Error 2
>>>> Makefile:104: recipe for target 'rbuild' failed
>>>> make[1]: *** [rbuild] Error 2
>>>> Makefile:14: recipe for target 'all' failed
>>>> make: *** [all] Error 2
>>>>
>>>> As all those files existed in their correct places, the only reason I
>>>> could think of that this would fail here is that GCC version 4.9 did
>>>> make some changes to enhance link-time optimization [5], and probably
>>>> something isn't compatible.
>>>
>>>Right. Just before Christmas, Hin-Tak Leung reported build failure with
>>>LTO:
>>>
>>>https://stat.ethz.ch/pipermail/r-devel/2014-December/070286.html
>>>https://stat.ethz.ch/pipermail/r-devel/2014-December/070319.html
>>>
>>>
>>>Many thanks to you and others for looking into this,
>>>Henric
>>>
>>>
>>>
>>>> I then downgraded to GCC 4.8.4 [6], and,
>>>> with the deletion of those 10 or so lines from compat.c, I can
>>>> complete the build straight through rinstaller. However, I get that
>>>> failure issue due to the extra 0 in scientific notation [7].
>>>>
>>>> It does not matter if I do the entire process in the MSYS2
>>>> environment, or if I do in in Windows with msys\usr\bin in my path.
>>>>
>>>> Naïvely, it seems that if there were some what for stdio to be
>>>> included in compat.c, yet the versions of snprintf and vsprintf in
>>>> that file to "override" the standard, perhaps this method would work.
>>>> Of course, running make check-all may uncover more issues. I intend to
>>>> run the equivalent checks (from the tools library) inside of R with
>>>> kill on failure turned off to see if anything else is problematic.
>>>>
>>>> Hopefully, something in this description resonates with one of the
>>>> readers here. If anyone has any ideas as to how to circumvent the
>>>> issues with compat.c, I'd be very grateful.
>>>>
>>>> Thank you,
>>>>
>>>> Avi
>>>>
>>>>
>>>> [1] http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/
>>>> [2] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.9.2/threads-win32/seh/x86_64-4.9.2-release-win32-seh-rt_v3-rev1.7z/download
>>>> [3] https://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>>>> [4] http://www.tt-forums.net/viewtopic.php?p=1034657&sid=613fa47a379ffaa0b9a9fb182a4180e3#p1034657
>>>> [5] https://gcc.gnu.org/gcc-4.9/changes.html
>>>> [6] http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/4.8.4/threads-win32/seh/x86_64-4.8.4-release-win32-seh-rt_v3-rev0.7z/download
>>>> [7] https://stat.ethz.ch/pipermail/r-devel/2015-January/070354.html
>>>>
>>>> Date: Wed, 07 Jan 2015 20:31:07 -0500
>>>>
>>>> From: Duncan Murdoch <murdoch.duncan at gmail.com>
>>>> To: Jeroen Ooms <jeroenooms at gmail.com>
>>>> Cc: "R-devel at r-project.org" <r-devel at r-project.org>
>>>> Subject: Re: [Rd] New version of Rtools for Windows
>>>> Message-ID: <54ADDDDB.4020500 at gmail.com>
>>>> Content-Type: text/plain; charset=utf-8
>>>>
>>>> On 07/01/2015 5:20 PM, Jeroen Ooms wrote:
>>>> On Wed, Jan 7, 2015 at 8:00 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:
>>>>
>>>> This version includes only minor updates to the tools. I indicated last summer that I was hoping to update GCC from the current version 4.6.3 before the R 3.2.0 release, but this now looks unlikely, unless someone else with experience building it can help.
>>>>
>>>> I have been looking into this a bit over the past few months, also
>>>> with mixed success. Nevertheless, below some experiences that might be
>>>> worth sharing.
>>>>
>>>> The guys from mingw-w64 recommended (quite strongly) to move away from
>>>> multilib. They explained that the standard approach is to create two
>>>> separate toolchains; one that targets win32 and the other one that
>>>> targets win64 (both tool chains can compiled for win32). Hence the
>>>> only difference for R would be that instead of passing "-m64" and
>>>> "-m32", it would need to set the path to the proper compiler.
>>>>
>>>> There are several initiatives that provide very complete suites of
>>>> precompiled mingw-w64 tools. I think the ideal scenario would be if we
>>>> could take advantage of an existing tool chain as we do on other
>>>> platforms, although perhaps I do not fully understand the R-specific
>>>> requirements on the windows compiler.
>>>>
>>>> I feel quite strongly that we need to be able to build the toolchain,
>>>> rather than relying on binaries produced by others. We may need to
>>>> customize the toolchain, or we may need to rebuild it when a bug is
>>>> identified. Lots of binary builders abandon their builds and you can't
>>>> count on them to solve problems at a later date.
>>>>
>>>>
>>>> One project that looks very promising is msys2 [1,2]. It has a package
>>>> manager (port of pacman from arch linux) and comes with a pretty
>>>> complete set of msys [3] and other [4] packages that seems quite well
>>>> maintained.
>>>>
>>>> Do they post complete instructions for building? That's what I'm
>>>> looking for. I don't want to develop a build script (I don't know how),
>>>> but I would like to have one.
>>>>
>>>> Duncan Murdoch
>>>>
>>>>
>>>> The only issue I ran into with msys2 is that it uses a different c++
>>>> exception model (seh/dwarf) than the current Rtools (which uses sjlj).
>>>> See also [5]. Therefore, if a library uses exceptions, we cannot use
>>>> the current Rtools to link a static library that was created with
>>>> msys2 [6]. I am not sure if it also be a problem the other way
>>>> around, and if this is still the case for recent versions of
>>>> gcc/mingw.
>>>>
>>>> Finally, Ruby has build very similar to Rtools called DevKit-mingw64
>>>> [7] that we might be able to borrow from.
>>>>
>>>>
>>>> [1] https://msys2.github.io/
>>>> [2] http://stackoverflow.com/questions/25019057/how-are-msys-msys2-and-msysgit-related-to-each-other
>>>> [3] https://github.com/Alexpux/MSYS2-packages
>>>> [4] https://github.com/Alexpux/MINGW-packages
>>>> [5] http://stackoverflow.com/questions/15670169/what-is-difference-between-sjlj-vs-dwarf-vs-seh
>>>> [6] http://stackoverflow.com/questions/7751640/undefined-reference-to-gxx-personality-sj0
>>>> [7] http://rubyinstaller.org/downloads/
>>>>
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>
>>>
>
More information about the R-devel
mailing list