[R-SIG-Win] Rtools44

Tomas Kalibera tom@@@k@||ber@ @end|ng |rom gm@||@com
Tue Mar 19 17:05:55 CET 2024 

On 3/19/24 15:33, Avraham Adler wrote:
> For completeness, I compiled R from source using the native BLAS, and
> make check fails again, this time on "aregexec". Running "agrep" from
> within Rgui causes it to crash to desktop. Could it be something to do
> with this: https://github.com/wch/r-source/commit/ba486d898c2698c2afb678abdab807510327541e?
> But that didn't affect aregexc or adist? This is beyond my
> understanding, but I'm happy to do any tests you need to help identify
> what is going on.

I can reproduce the problem (without LTO, without openblas), just by 
"-O3 -march=native" and running

agrep("lasy", "1 lazy 2")

which crashes R.

The crash happens in do_agrep from agrep.c, when calling tre_regaexec:

         } else {
             const char *s = translateChar(STRING_ELT(vec, i));
             if(mbcslocale && !mbcsValid(s))
                 error(_("input string %lld is invalid in this locale"),
                       (long long)i+1);
             rc = tre_regaexec(&reg, s, &match, params, 0);  // <=== 
here (line 242)
             vmaxset(vmax);
         }

    0x00007fff79c642da <+1290>:  mov    0x50(%rsp),%rdx
    0x00007fff79c642df <+1295>:  test   %eax,%eax
    0x00007fff79c642e1 <+1297>:  je     0x7fff79c6522a <do_agrep+5210>
    0x00007fff79c642e7 <+1303>:  movl   $0x0,0x20(%rsp)
    0x00007fff79c642ef <+1311>:  mov    0x38(%rsp),%rcx
    0x00007fff79c642f4 <+1316>:  mov    %r15,%r8
    0x00007fff79c642f7 <+1319>:  lea    0x80(%rsp),%r9
    0x00007fff79c642ff <+1327>:  vmovdqu 0xb0(%rsp),%ymm4
=> 0x00007fff79c64308 <+1336>:  vmovdqa %ymm4,0x80(%rsp)
    0x00007fff79c64311 <+1345>:  vzeroupper
    0x00007fff79c64314 <+1348>:  call   0x7fff79d183d0 <tre_regaexec>

And it happens because of incorrect alignment in an AVX2 instruction. In 
the above, vmovdqa is used as a second step of copying the regparams_t 
structure to the stack (to be passed as 4th argument of tre_regaexec). 
But GCC uses 0x80(%rsp) for that, which happens only to be 16-byte 
aligned, but not 32-byte aligned. AVX2 requires this to be 32-byte 
aligned in vmovdqa instruction, and hence the program segfaults.

This is a long standing bug in GCC that has been reported long time ago: 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412

It must be a coincidence that you haven't run into this with GCC 12.3 
and earlier (so earlier versions of Rtools for Windows).

I think the simplest and most reliable work-around is to not use AVX2 
with GCC on Windows.

Best
Tomas

>
> Thank you,
>
> Avi
>
>
>
> On Mon, Mar 18, 2024 at 11:58 PM Avraham Adler <avraham.adler using gmail.com> wrote:
>> I ran "tools::testInstalledBasic("both")" from within Rterm, and it
>> failed on reg-tests-1a, and doesn't that last call in the FAIL look
>> familiar? Three times isn;t coincidence. What could be causing
>> agrep/adist to fail?
>>
>> Avi
>>
>>> a <- c("NA", NA, "BANANA")
>>> na <- NA_character_
>>> a1 <- substr(a,1,1)
>>> stopifnot(is.na(a1)==is.na(a))
>>> a2 <- substring(a,1,1)
>>> stopifnot(is.na(a2)==is.na(a))
>>> a3 <- sub("NA","na",a)
>>> stopifnot(is.na(a3)==is.na(a))
>>> a3 <- gsub("NA","na",a)
>>> stopifnot(is.na(a3)==is.na(a))
>>> substr(a3, 1, 2) <- "na"
>>> stopifnot(is.na(a3)==is.na(a))
>>> substr(a3, 1, 2) <- na
>>> stopifnot(all(is.na(a3)))
>>> stopifnot(agrep("NA", a) == c(1, 3))
>> On Mon, Mar 18, 2024 at 11:52 PM Avraham Adler <avraham.adler using gmail.com> wrote:
>>> Hello, Tomas.
>>>
>>> I ran check again and this time it failed on base. Base failed at
>>> "agrep" and utils failed at "adist" if that triggers any ideas.
>>>
>>> Thank you,
>>>
>>> Avi
>>>
>>> On Mon, Mar 18, 2024 at 11:49 PM Avraham Adler <avraham.adler using gmail.com> wrote:
>>>> Hello, Tomas.
>>>>
>>>> I had hoped that my problem was that I was linking to OpenBLAS built
>>>> under Rtools43, but sadly that is not the case, as I built OpenBLAS
>>>> using Rtools44 and I still get the error in 'utils'. I do have
>>>> Rtools43 installed on this machine, but I am doing eveything from the
>>>> Rtools44 bash. My MkRules.local follows:
>>>>
>>>> USE_ATLAS = YES
>>>> ATLAS_PATH = C:/R/OPB/OPB_03.26_1T_44
>>>> EOPTS = -march=native -pipe -mno-rtm
>>>> LTO = -flto=1 -fuse-linker-plugin
>>>> LTO_OPT = -flto=1 -fuse-linker-plugin
>>>> LTO_FC = -flto=1 -fuse-linker-plugin
>>>> LTO_FC_OPT = -flto=1 -fuse-linker-plugin
>>>> QPDF = C:/R/qpdf-11.9.0-msvc64
>>>> OPENMP = -fopenmp
>>>>
>>>> I also make the following change to Makefile.win in /src/extra/blas as
>>>> I have been doing for more than a decade:
>>>>
>>>> --- /c/r/trunk/src/extra/blas/Makefile.win    2024-01-24
>>>> 18:34:42.755255900 +0000
>>>> +++ /c/r/Makefile.win    2024-01-24 18:39:39.716458000 +0000
>>>> @@ -12,7 +12,7 @@
>>>>   ../../../$(BINDIR)/Rblas.dll: blas00.o ../../gnuwin32/dllversion.o
>>>>       @$(ECHO) -------- Building $@ --------
>>>>       $(DLL) -s -shared $(DLLFLAGS) -o $@ $^ Rblas.def \
>>>> -       -L../../../$(IMPDIR) -lR  -L"$(ATLAS_PATH)" -lf77blas -latlas
>>>> +       -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -fopenmp -lopenblas
>>>>   else
>>>>   ../../../$(BINDIR)/Rblas.dll: blas.o blas2.o cmplxblas.o cmplxblas2.o
>>>> ../../gnuwin32/dllversion.o
>>>>       @$(ECHO) -------- Building $@ --------
>>>>
>>>> The utils-Ex.Rout.fail is 1188 lines long and I don't see any obvious
>>>> point of failure. The last few lines are:
>>>>> cleanEx()
>>>>> nameEx("adist")
>>>>> ### * adist
>>>>>
>>>>> flush(stderr()); flush(stdout())
>>>>>
>>>>> ### Name: adist
>>>>> ### Title: Approximate String Distances
>>>>> ### Aliases: adist
>>>>> ### Keywords: character
>>>>>
>>>>> ### ** Examples
>>>>>
>>>>> ## Cf. https://en.wikipedia.org/wiki/Levenshtein_distance
>>>>> adist("kitten", "sitting")
>>>>       [,1]
>>>> [1,]    3
>>>>> ## To see the transformation counts for the Levenshtein distance:
>>>>> drop(attr(adist("kitten", "sitting", counts = TRUE), "counts"))
>>>> ins del sub
>>>>    1   0   2
>>>>> ## To see the transformation sequences:
>>>>> attr(adist(c("kitten", "sitting"), counts = TRUE), "trafos")
>>>>       [,1]      [,2]
>>>> [1,] "MMMMMM"  "SMMMSMI"
>>>> [2,] "SMMMSMD" "MMMMMMM"
>>>>> ## Cf. the examples for agrep:
>>>>> adist("lasy", "1 lazy 2")
>>>>       [,1]
>>>> [1,]    5
>>>>> ## For a "partial approximate match" (as used for agrep):
>>>>> adist("lasy", "1 lazy 2", partial = TRUE)
>>>> The build works under Rtools43. Should I uninstall both versions of
>>>> Rtools, my current R installation, and its library and try again? I'm
>>>> doubtful that will help as my "active" R installation is in a
>>>> completely different directory, but I am willing to try. If there is
>>>> any output or other tests I can do, please let me know. Or, if you
>>>> think I should raise this on r-devel.
>>>>
>>>> Thank you,
>>>>
>>>> Avi
>>>>
>>>> On Mon, Mar 18, 2024 at 4:08 AM Tomas Kalibera <tomas.kalibera using gmail.com> wrote:
>>>>> Hello Avi,
>>>>>
>>>>> On 3/17/24 17:53, Avraham Adler wrote:
>>>>>> Hello, Tomas.
>>>>>>
>>>>>> As always, thank you for your incessant hard work. I have compiled
>>>>>> R-devel 86144 using Rtools44 and it completes normally. However, it
>>>>>> fails very early in make check-devel. Specifically, I get the output
>>>>>> below, and have received it more than once. Is this something to raise
>>>>>> on R-devel or is it an Rtools44-specific issue?
>>>>> I can't tell from this output. Could you please try to get a more
>>>>> specific error output?
>>>>> Could you please share your compiler options (e.g. MkRules.local) and
>>>>> indeed if you made any modifications to the code?
>>>>>
>>>>> Also, as always, it is worth making sure that all code has been rebuilt
>>>>> using the new Rtools - e.g. delete any old package library for R-devel
>>>>> (4.4) you may have on your system.
>>>>>
>>>>> The testing done by myself and CRAN only covers the default compilation
>>>>> options as specified in the make files.
>>>>>
>>>>> Best
>>>>> Tomas
>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Avi
>>>>>>
>>>>>> $ make check-devel
>>>>>> Testing examples for package 'base'
>>>>>> Testing examples for package 'tools'
>>>>>>     comparing 'tools-Ex.Rout' to 'tools-Ex.Rout.save' ... NOTE
>>>>>>     1046,1047d1045
>>>>>>     < Warning in file(con, "r") :
>>>>>>     <   file("") only supports open = "w+" and open = "w+b": using the former
>>>>>>     1050,1051c1048,1049
>>>>>>     <  $ file    : chr ""
>>>>>>     <  $ title   : chr ""
>>>>>>     ---
>>>>>>     >  $ file    : chr "grid.Rnw"
>>>>>>     >  $ title   : chr "Introduction to grid"
>>>>>> Testing examples for package 'utils'
>>>>>> Error: testing 'utils' failed
>>>>>> Execution halted
>>>>>> make[3]: *** [Makefile.win:29: test-Examples-Base] Error 1
>>>>>> make[2]: *** [Makefile.common:208: test-Examples] Error 2
>>>>>> make[1]: *** [Makefile.common:193: test-all-basics] Error 1
>>>>>> make: *** [Makefile:333: check-devel] Error 2
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 14, 2024 at 4:45 AM Tomas Kalibera <tomas.kalibera using gmail.com> wrote:
>>>>>>> Dear R Windows developers,
>>>>>>>
>>>>>>> there is now a new toolchain for R for Windows, Rtools44. It is now used
>>>>>>> by R-devel and is intended for R 4.4.0.
>>>>>>>
>>>>>>> Compared to Rtools43, it uses GCC 13 and updates other core components.
>>>>>>> See https://cran.r-project.org/bin/windows/Rtools/rtools44/news.html for
>>>>>>> a detailed list of changes.
>>>>>>>
>>>>>>> All users of R-devel who need to compile R packages with source code in
>>>>>>> C, C++ or Fortran should install Rtools44. From the user perspective,
>>>>>>> Rtools44 works the same way as Rtools43. See
>>>>>>> https://cran.r-project.org/bin/windows/base/howto-R-devel.html for
>>>>>>> instructions on how to build R-devel and packages for this version of R.
>>>>>>>
>>>>>>> It is recommended to re-install packages that need compilation to avoid
>>>>>>> potential incompatibilities with code built using Rtools43.
>>>>>>>
>>>>>>> Rtools44 also includes an experimental version for 64-bit ARM machines,
>>>>>>> using LLVM 17 (and clang, flang-new, lld, libc++) - the aarch64 version
>>>>>>> of Rtools has its own installer and distribution tarballs.
>>>>>>>
>>>>>>> Best
>>>>>>> Tomas
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> R-SIG-windows mailing list
>>>>>>> R-SIG-windows using r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-windows

More information about the R-SIG-windows mailing list