[R-pkg-devel] Does dependencies up to date on the pretest CRAN infrastructure

[Uwe Ligges]

On 13.01.2024 15:01, Uwe Ligges wrote:
> Fascinating, now it worked with the latest winbuilder submission 3 times 
> in a row when I checked it manually. So maybe Ivan was right and there 
> was a very demanding set of other packages compiling at the same time?
> I don't know.
> 
> Serge, Can you somply submit your latest winbuilder upload to CRAN?

Really, I inspected some more. The underlying issue is simple:
The C++ compiler used under Windows asks for precomitted memory. If 
several processes are running at the same time, a lot of memory is 
precomitted. And Windows does not use it for other processes, even if 
almost nothing is actually used.
So while the used memory may be around 50GB, all of the rest (of 756 GB 
including swap space) may have been precomitted (but unused) and new 
processes failed to start correctly. Grrrr.

Best,
Uwe Ligges





> Best,
> Uwe Ligges
> 
> 
> 
> On 13.01.2024 14:12, Uwe Ligges wrote:
>> I can take a look, but not sure if I get to it before monday.
>> I haven't seen it for any other packages recently.
>>
>> My suspicion is currently a strange mix of cmd.exe and sh.exe calls. 
>> But this is a very wild guess.
>>
>> Best,
>> Uwe
>>
>> On 13.01.2024 14:08, Uwe Ligges wrote:
>>>
>>>
>>> On 13.01.2024 10:10, Ivan Krylov via R-package-devel wrote:
>>>> В Fri, 12 Jan 2024 21:19:00 +0100
>>>> Serge <Serge.Iovleff using stkpp.org> пишет:
>>>>
>>>>> After somme minor midficiations, I make a try on the winbuilder site.
>>>>> I was able to build the archive with the static library
>>>>> but I get again a Bad address error. You can have a look to
>>>>>
>>>>> https://win-builder.r-project.org/bw47qsMX3HTd/00install.out
>>>>
>>>> I think that Win-Builder is running out of memory. It took some
>>>> experimenting, but I was able to reproduce something like this using
>>>> the following:
>>>>
>>>> 1. Set the swap file in the Windows settings to minimal recommended
>>>> size and disable its automatic growth
>>>>
>>>> 2. Write and run a program that does malloc(LARGE_NUMBER); getchar();
>>>> so that almost all physical memory is allocated
>>>>
>>>> 3. Run gcc -DFOO=`/path/to/Rscript -e 'some script'` & many times
>>>>
>>>> I got a lot of interesting errors, including the "Bad address":
>>>>
>>>> Warnings:
>>>> 1: .getGeneric(f, , package) : internal error -4 in R_decompress1
>>>> 2: package "methods" in options("defaultPackages") was not found
>>>>
>>>> 0 [main] bash (2892) child_copy: cygheap read copy failed,
>>>> 0x0..0x800025420, done 0, windows pid 2892, Win32 error 299
>>>>
>>>> 0 [main] bash (3256) C:\rtools43\usr\bin\bash.exe: *** fatal error in
>>>> forked process - MEM_COMMIT failed, Win32 error 1455
>>>>
>>>> -bash: fork: retry: Resource temporarily unavailable
>>>>
>>>> -bash: R-devel/bin/Rscript.exe: Bad address
>>>
>>> The above indeed happens if not sufficient memory would be available.
>>> Important to know: This includes unused but committed memory which 
>>> may be a lot.
>>> But I doubt it is the case on winbuilder as the machines has 256GB or 
>>> more (depending in the machine) and additionally 500GB swap space on 
>>> SSD.
>>>
>>> Best,
>>> Uwe
>>>
>>>
>>>> Your package is written in C++, but that by itself shouldn't disqualify
>>>> it. On my Linux computer, /usr/bin/time R -e
>>>> 'install.packages("MixAll")' says that the installation takes slightly
>>>> less than a gigabyte of memory ("912516maxresident k"), which is par
>>>> the course for such packages. (My small Rcpp-using package takes
>>>> approximately half a gigabyte by the same metric.)
>>>>
>>>> I'm still not 100% sure (if Win-Builder is running out of memory, why
>>>> are you seeing "Bad address" only and not the rest of the carnage?),
>>>> but I'm not seeing a problem with your package, either. If EFAULT is
>>>> Cygwin's way of saying "I caught a bad pointer in your system call"
>>>> (which, I must stress, is happening inside /bin/sh, not your package
>>>> or even R at all), it's not impossible that Win-Builder is having
>>>> hardware problems. Unfortunately, they take a lot of effort and
>>>> downtime to diagnose and could be hiding anywhere from RAM to the power
>>>> supply.
>>>>
>>>
>>> ______________________________________________
>>> R-package-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel