[R-SIG-Win] Rtools44

Tomas Kalibera tom@@@k@||ber@ @end|ng |rom gm@||@com
Tue Mar 19 17:15:17 CET 2024 

On 3/19/24 17:05, Tomas Kalibera wrote:
> On 3/19/24 15:33, Avraham Adler wrote:
>> For completeness, I compiled R from source using the native BLAS, and
>> make check fails again, this time on "aregexec". Running "agrep" from
>> within Rgui causes it to crash to desktop. Could it be something to do
>> with this: 
>> https://github.com/wch/r-source/commit/ba486d898c2698c2afb678abdab807510327541e?
>> But that didn't affect aregexc or adist? This is beyond my
>> understanding, but I'm happy to do any tests you need to help identify
>> what is going on.
>
> I can reproduce the problem (without LTO, without openblas), just by 
> "-O3 -march=native" and running
>
> agrep("lasy", "1 lazy 2")
>
> which crashes R.
>
> The crash happens in do_agrep from agrep.c, when calling tre_regaexec:
>
>         } else {
>             const char *s = translateChar(STRING_ELT(vec, i));
>             if(mbcslocale && !mbcsValid(s))
>                 error(_("input string %lld is invalid in this locale"),
>                       (long long)i+1);
>             rc = tre_regaexec(&reg, s, &match, params, 0);  // <=== 
> here (line 242)
>             vmaxset(vmax);
>         }
>
>    0x00007fff79c642da <+1290>:  mov    0x50(%rsp),%rdx
>    0x00007fff79c642df <+1295>:  test   %eax,%eax
>    0x00007fff79c642e1 <+1297>:  je     0x7fff79c6522a <do_agrep+5210>
>    0x00007fff79c642e7 <+1303>:  movl   $0x0,0x20(%rsp)
>    0x00007fff79c642ef <+1311>:  mov    0x38(%rsp),%rcx
>    0x00007fff79c642f4 <+1316>:  mov    %r15,%r8
>    0x00007fff79c642f7 <+1319>:  lea    0x80(%rsp),%r9
>    0x00007fff79c642ff <+1327>:  vmovdqu 0xb0(%rsp),%ymm4
> => 0x00007fff79c64308 <+1336>:  vmovdqa %ymm4,0x80(%rsp)
>    0x00007fff79c64311 <+1345>:  vzeroupper
>    0x00007fff79c64314 <+1348>:  call   0x7fff79d183d0 <tre_regaexec>
>
> And it happens because of incorrect alignment in an AVX2 instruction. 
> In the above, vmovdqa is used as a second step of copying the 
> regparams_t structure to the stack (to be passed as 4th argument of 
> tre_regaexec). But GCC uses 0x80(%rsp) for that, which happens only to 
> be 16-byte aligned, but not 32-byte aligned. AVX2 requires this to be 
> 32-byte aligned in vmovdqa instruction, and hence the program segfaults.
>
> This is a long standing bug in GCC that has been reported long time 
> ago: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412
>
> It must be a coincidence that you haven't run into this with GCC 12.3 
> and earlier (so earlier versions of Rtools for Windows).
>
> I think the simplest and most reliable work-around is to not use AVX2 
> with GCC on Windows.

You can also use "-Wa,-muse-unaligned-vector-move" option for the GNU 
assembler (add to optimization options), which will make the assembler 
use instructions that don't require memory to be aligned. Whether it 
would still be beneficial for performance to use AVX2 in the end or not 
probably depends on the code.

Best
Tomas

>
> Best
> Tomas
>
>>
>> Thank you,
>>
>> Avi
>>
>>
>>
>> On Mon, Mar 18, 2024 at 11:58 PM Avraham Adler 
>> <avraham.adler using gmail.com> wrote:
>>> I ran "tools::testInstalledBasic("both")" from within Rterm, and it
>>> failed on reg-tests-1a, and doesn't that last call in the FAIL look
>>> familiar? Three times isn;t coincidence. What could be causing
>>> agrep/adist to fail?
>>>
>>> Avi
>>>
>>>> a <- c("NA", NA, "BANANA")
>>>> na <- NA_character_
>>>> a1 <- substr(a,1,1)
>>>> stopifnot(is.na(a1)==is.na(a))
>>>> a2 <- substring(a,1,1)
>>>> stopifnot(is.na(a2)==is.na(a))
>>>> a3 <- sub("NA","na",a)
>>>> stopifnot(is.na(a3)==is.na(a))
>>>> a3 <- gsub("NA","na",a)
>>>> stopifnot(is.na(a3)==is.na(a))
>>>> substr(a3, 1, 2) <- "na"
>>>> stopifnot(is.na(a3)==is.na(a))
>>>> substr(a3, 1, 2) <- na
>>>> stopifnot(all(is.na(a3)))
>>>> stopifnot(agrep("NA", a) == c(1, 3))
>>> On Mon, Mar 18, 2024 at 11:52 PM Avraham Adler 
>>> <avraham.adler using gmail.com> wrote:
>>>> Hello, Tomas.
>>>>
>>>> I ran check again and this time it failed on base. Base failed at
>>>> "agrep" and utils failed at "adist" if that triggers any ideas.
>>>>
>>>> Thank you,
>>>>
>>>> Avi
>>>>
>>>> On Mon, Mar 18, 2024 at 11:49 PM Avraham Adler 
>>>> <avraham.adler using gmail.com> wrote:
>>>>> Hello, Tomas.
>>>>>
>>>>> I had hoped that my problem was that I was linking to OpenBLAS built
>>>>> under Rtools43, but sadly that is not the case, as I built OpenBLAS
>>>>> using Rtools44 and I still get the error in 'utils'. I do have
>>>>> Rtools43 installed on this machine, but I am doing eveything from the
>>>>> Rtools44 bash. My MkRules.local follows:
>>>>>
>>>>> USE_ATLAS = YES
>>>>> ATLAS_PATH = C:/R/OPB/OPB_03.26_1T_44
>>>>> EOPTS = -march=native -pipe -mno-rtm
>>>>> LTO = -flto=1 -fuse-linker-plugin
>>>>> LTO_OPT = -flto=1 -fuse-linker-plugin
>>>>> LTO_FC = -flto=1 -fuse-linker-plugin
>>>>> LTO_FC_OPT = -flto=1 -fuse-linker-plugin
>>>>> QPDF = C:/R/qpdf-11.9.0-msvc64
>>>>> OPENMP = -fopenmp
>>>>>
>>>>> I also make the following change to Makefile.win in 
>>>>> /src/extra/blas as
>>>>> I have been doing for more than a decade:
>>>>>
>>>>> --- /c/r/trunk/src/extra/blas/Makefile.win    2024-01-24
>>>>> 18:34:42.755255900 +0000
>>>>> +++ /c/r/Makefile.win    2024-01-24 18:39:39.716458000 +0000
>>>>> @@ -12,7 +12,7 @@
>>>>>   ../../../$(BINDIR)/Rblas.dll: blas00.o ../../gnuwin32/dllversion.o
>>>>>       @$(ECHO) -------- Building $@ --------
>>>>>       $(DLL) -s -shared $(DLLFLAGS) -o $@ $^ Rblas.def \
>>>>> -       -L../../../$(IMPDIR) -lR  -L"$(ATLAS_PATH)" -lf77blas -latlas
>>>>> +       -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -fopenmp 
>>>>> -lopenblas
>>>>>   else
>>>>>   ../../../$(BINDIR)/Rblas.dll: blas.o blas2.o cmplxblas.o 
>>>>> cmplxblas2.o
>>>>> ../../gnuwin32/dllversion.o
>>>>>       @$(ECHO) -------- Building $@ --------
>>>>>
>>>>> The utils-Ex.Rout.fail is 1188 lines long and I don't see any obvious
>>>>> point of failure. The last few lines are:
>>>>>> cleanEx()
>>>>>> nameEx("adist")
>>>>>> ### * adist
>>>>>>
>>>>>> flush(stderr()); flush(stdout())
>>>>>>
>>>>>> ### Name: adist
>>>>>> ### Title: Approximate String Distances
>>>>>> ### Aliases: adist
>>>>>> ### Keywords: character
>>>>>>
>>>>>> ### ** Examples
>>>>>>
>>>>>> ## Cf. https://en.wikipedia.org/wiki/Levenshtein_distance
>>>>>> adist("kitten", "sitting")
>>>>>       [,1]
>>>>> [1,]    3
>>>>>> ## To see the transformation counts for the Levenshtein distance:
>>>>>> drop(attr(adist("kitten", "sitting", counts = TRUE), "counts"))
>>>>> ins del sub
>>>>>    1   0   2
>>>>>> ## To see the transformation sequences:
>>>>>> attr(adist(c("kitten", "sitting"), counts = TRUE), "trafos")
>>>>>       [,1]      [,2]
>>>>> [1,] "MMMMMM"  "SMMMSMI"
>>>>> [2,] "SMMMSMD" "MMMMMMM"
>>>>>> ## Cf. the examples for agrep:
>>>>>> adist("lasy", "1 lazy 2")
>>>>>       [,1]
>>>>> [1,]    5
>>>>>> ## For a "partial approximate match" (as used for agrep):
>>>>>> adist("lasy", "1 lazy 2", partial = TRUE)
>>>>> The build works under Rtools43. Should I uninstall both versions of
>>>>> Rtools, my current R installation, and its library and try again? I'm
>>>>> doubtful that will help as my "active" R installation is in a
>>>>> completely different directory, but I am willing to try. If there is
>>>>> any output or other tests I can do, please let me know. Or, if you
>>>>> think I should raise this on r-devel.
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Avi
>>>>>
>>>>> On Mon, Mar 18, 2024 at 4:08 AM Tomas Kalibera 
>>>>> <tomas.kalibera using gmail.com> wrote:
>>>>>> Hello Avi,
>>>>>>
>>>>>> On 3/17/24 17:53, Avraham Adler wrote:
>>>>>>> Hello, Tomas.
>>>>>>>
>>>>>>> As always, thank you for your incessant hard work. I have compiled
>>>>>>> R-devel 86144 using Rtools44 and it completes normally. However, it
>>>>>>> fails very early in make check-devel. Specifically, I get the 
>>>>>>> output
>>>>>>> below, and have received it more than once. Is this something to 
>>>>>>> raise
>>>>>>> on R-devel or is it an Rtools44-specific issue?
>>>>>> I can't tell from this output. Could you please try to get a more
>>>>>> specific error output?
>>>>>> Could you please share your compiler options (e.g. MkRules.local) 
>>>>>> and
>>>>>> indeed if you made any modifications to the code?
>>>>>>
>>>>>> Also, as always, it is worth making sure that all code has been 
>>>>>> rebuilt
>>>>>> using the new Rtools - e.g. delete any old package library for 
>>>>>> R-devel
>>>>>> (4.4) you may have on your system.
>>>>>>
>>>>>> The testing done by myself and CRAN only covers the default 
>>>>>> compilation
>>>>>> options as specified in the make files.
>>>>>>
>>>>>> Best
>>>>>> Tomas
>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Avi
>>>>>>>
>>>>>>> $ make check-devel
>>>>>>> Testing examples for package 'base'
>>>>>>> Testing examples for package 'tools'
>>>>>>>     comparing 'tools-Ex.Rout' to 'tools-Ex.Rout.save' ... NOTE
>>>>>>>     1046,1047d1045
>>>>>>>     < Warning in file(con, "r") :
>>>>>>>     <   file("") only supports open = "w+" and open = "w+b": 
>>>>>>> using the former
>>>>>>>     1050,1051c1048,1049
>>>>>>>     <  $ file    : chr ""
>>>>>>>     <  $ title   : chr ""
>>>>>>>     ---
>>>>>>>     >  $ file    : chr "grid.Rnw"
>>>>>>>     >  $ title   : chr "Introduction to grid"
>>>>>>> Testing examples for package 'utils'
>>>>>>> Error: testing 'utils' failed
>>>>>>> Execution halted
>>>>>>> make[3]: *** [Makefile.win:29: test-Examples-Base] Error 1
>>>>>>> make[2]: *** [Makefile.common:208: test-Examples] Error 2
>>>>>>> make[1]: *** [Makefile.common:193: test-all-basics] Error 1
>>>>>>> make: *** [Makefile:333: check-devel] Error 2
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 14, 2024 at 4:45 AM Tomas Kalibera 
>>>>>>> <tomas.kalibera using gmail.com> wrote:
>>>>>>>> Dear R Windows developers,
>>>>>>>>
>>>>>>>> there is now a new toolchain for R for Windows, Rtools44. It is 
>>>>>>>> now used
>>>>>>>> by R-devel and is intended for R 4.4.0.
>>>>>>>>
>>>>>>>> Compared to Rtools43, it uses GCC 13 and updates other core 
>>>>>>>> components.
>>>>>>>> See 
>>>>>>>> https://cran.r-project.org/bin/windows/Rtools/rtools44/news.html 
>>>>>>>> for
>>>>>>>> a detailed list of changes.
>>>>>>>>
>>>>>>>> All users of R-devel who need to compile R packages with source 
>>>>>>>> code in
>>>>>>>> C, C++ or Fortran should install Rtools44. From the user 
>>>>>>>> perspective,
>>>>>>>> Rtools44 works the same way as Rtools43. See
>>>>>>>> https://cran.r-project.org/bin/windows/base/howto-R-devel.html for
>>>>>>>> instructions on how to build R-devel and packages for this 
>>>>>>>> version of R.
>>>>>>>>
>>>>>>>> It is recommended to re-install packages that need compilation 
>>>>>>>> to avoid
>>>>>>>> potential incompatibilities with code built using Rtools43.
>>>>>>>>
>>>>>>>> Rtools44 also includes an experimental version for 64-bit ARM 
>>>>>>>> machines,
>>>>>>>> using LLVM 17 (and clang, flang-new, lld, libc++) - the aarch64 
>>>>>>>> version
>>>>>>>> of Rtools has its own installer and distribution tarballs.
>>>>>>>>
>>>>>>>> Best
>>>>>>>> Tomas
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> R-SIG-windows mailing list
>>>>>>>> R-SIG-windows using r-project.org
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-windows

More information about the R-SIG-windows mailing list