[R-pkg-devel] ERROR building MixAll on Windows platform

[Serge]

This post is a continuation of the post *[R-pkg-devel] Does dependencies up to date on the pretest 
CRAN infrastructure*

I made more (unsuccessful) tries:

- I installed a Windows 11 version in a VM on my compuiter and try to buid the MixAll package using 
Rtools42 and Rtools43 (it's quite easy, and funny, to do it on windows: you have just to rename 
C:\rtools42 as C:\rtools43).
The result is that MixAll is build using the 4.2 version and the buid failed using the 4.3 version.

- I installed the version 12.3 of gcc on ubuntu (the same version used on windows) and could build 
the package without problem

- Inspecting the log of the Rtools4.3 
(https://cran.r-project.org/bin/windows/Rtools/rtools43/news.html) and g++12..3 
(https://gcc.gnu.org/gcc-12/changes.html) does not give insight.

- The package is dependent of the rtkore package which use extensive use of templated class. As 
rtkore is a port for R of stk++, I try to compile the stk++ library on windows (using g++12). All 
tests are compiled without any troubles.

These attempts eliminate some causes, but don't give me any insight why MixAll (and blockcluster) 
failed to be build on the Windows-devel platform. It seems related to Rtools43. Does anyone else 
(using for exemple Rcppeigen) is experiencing this problem ?

Serge



Le 14/01/2024 à 18:50, Uwe Ligges a écrit :
> 
> 
> On 13.01.2024 15:01, Uwe Ligges wrote:
>> Fascinating, now it worked with the latest winbuilder submission 3 times in a row when I checked 
>> it manually. So maybe Ivan was right and there was a very demanding set of other packages 
>> compiling at the same time?
>> I don't know.
>>
>> Serge, Can you somply submit your latest winbuilder upload to CRAN?
> 
> Really, I inspected some more. The underlying issue is simple:
> The C++ compiler used under Windows asks for precomitted memory. If several processes are running at 
> the same time, a lot of memory is precomitted. And Windows does not use it for other processes, even 
> if almost nothing is actually used.
> So while the used memory may be around 50GB, all of the rest (of 756 GB including swap space) may 
> have been precomitted (but unused) and new processes failed to start correctly. Grrrr.
> 
> Best,
> Uwe Ligges
> 
> 
> 
> 
> 
>> Best,
>> Uwe Ligges
>>
>>
>>
>> On 13.01.2024 14:12, Uwe Ligges wrote:
>>> I can take a look, but not sure if I get to it before monday.
>>> I haven't seen it for any other packages recently.
>>>
>>> My suspicion is currently a strange mix of cmd.exe and sh.exe calls. But this is a very wild guess.
>>>
>>> Best,
>>> Uwe
>>>
>>> On 13.01.2024 14:08, Uwe Ligges wrote:
>>>>
>>>>
>>>> On 13.01.2024 10:10, Ivan Krylov via R-package-devel wrote:
>>>>> В Fri, 12 Jan 2024 21:19:00 +0100
>>>>> Serge <Serge.Iovleff using stkpp.org> пишет:
>>>>>
>>>>>> After somme minor midficiations, I make a try on the winbuilder site.
>>>>>> I was able to build the archive with the static library
>>>>>> but I get again a Bad address error. You can have a look to
>>>>>>
>>>>>> https://win-builder.r-project.org/bw47qsMX3HTd/00install.out
>>>>>
>>>>> I think that Win-Builder is running out of memory. It took some
>>>>> experimenting, but I was able to reproduce something like this using
>>>>> the following:
>>>>>
>>>>> 1. Set the swap file in the Windows settings to minimal recommended
>>>>> size and disable its automatic growth
>>>>>
>>>>> 2. Write and run a program that does malloc(LARGE_NUMBER); getchar();
>>>>> so that almost all physical memory is allocated
>>>>>
>>>>> 3. Run gcc -DFOO=`/path/to/Rscript -e 'some script'` & many times
>>>>>
>>>>> I got a lot of interesting errors, including the "Bad address":
>>>>>
>>>>> Warnings:
>>>>> 1: .getGeneric(f, , package) : internal error -4 in R_decompress1
>>>>> 2: package "methods" in options("defaultPackages") was not found
>>>>>
>>>>> 0 [main] bash (2892) child_copy: cygheap read copy failed,
>>>>> 0x0..0x800025420, done 0, windows pid 2892, Win32 error 299
>>>>>
>>>>> 0 [main] bash (3256) C:\rtools43\usr\bin\bash.exe: *** fatal error in
>>>>> forked process - MEM_COMMIT failed, Win32 error 1455
>>>>>
>>>>> -bash: fork: retry: Resource temporarily unavailable
>>>>>
>>>>> -bash: R-devel/bin/Rscript.exe: Bad address
>>>>
>>>> The above indeed happens if not sufficient memory would be available.
>>>> Important to know: This includes unused but committed memory which may be a lot.
>>>> But I doubt it is the case on winbuilder as the machines has 256GB or more (depending in the 
>>>> machine) and additionally 500GB swap space on SSD.
>>>>
>>>> Best,
>>>> Uwe
>>>>
>>>>
>>>>> Your package is written in C++, but that by itself shouldn't disqualify
>>>>> it. On my Linux computer, /usr/bin/time R -e
>>>>> 'install.packages("MixAll")' says that the installation takes slightly
>>>>> less than a gigabyte of memory ("912516maxresident k"), which is par
>>>>> the course for such packages. (My small Rcpp-using package takes
>>>>> approximately half a gigabyte by the same metric.)
>>>>>
>>>>> I'm still not 100% sure (if Win-Builder is running out of memory, why
>>>>> are you seeing "Bad address" only and not the rest of the carnage?),
>>>>> but I'm not seeing a problem with your package, either. If EFAULT is
>>>>> Cygwin's way of saying "I caught a bad pointer in your system call"
>>>>> (which, I must stress, is happening inside /bin/sh, not your package
>>>>> or even R at all), it's not impossible that Win-Builder is having
>>>>> hardware problems. Unfortunately, they take a lot of effort and
>>>>> downtime to diagnose and could be hiding anywhere from RAM to the power
>>>>> supply.
>>>>>
>>>>
>>>> ______________________________________________
>>>> R-package-devel using r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>>
>>> ______________________________________________
>>> R-package-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel