[Bioc-devel] [External] Memory error from sesame build

Hervé Pagès hp@ge@@on@g|thub @end|ng |rom gm@||@com
Mon Nov 29 23:51:19 CET 2021


On 29/11/2021 09:25, Zhou, Wanding wrote:
> Thanks for the input Herve. I saw your post but can you help me confirm 
> that mine is on the 32-bit windows since it says mingw32 (64-bit)?
> 
> Windows Server 2012 R2 Standard / x64
> 
> * using platform: x86_64-w64-mingw32 (64-bit)

This shows the Windows arch that R uses by default on Windows. But we 
use --force-multiarch when running 'R CMD check' on our Windows builders 
to force 'R CMD check' to run all the examples, unit tests, and 
vignettes in the two supported Windows archs: 32-bit and 64-bit.

The error was happening during the following step:

   ** running examples for arch 'i386'

and not during the following step:

   ** running examples for arch 'x64'

So yes, this is a Windows 32-bit only error.

> 
> http://bioconductor.org/checkResults/release/bioc-LATEST/sesame/tokay2-checksrc.html 
> <http://bioconductor.org/checkResults/release/bioc-LATEST/sesame/tokay2-checksrc.html>
> 
> Does that mean all the examples added together cannot use over 3G 
> memory?

Yes.

> I still don't think I am using that much but I will remove some 
> examples and see that solves the problem

Here is one way to see how much memory 'R CMD check' consumes when 
running the examples:

1. Run 'R CMD check' on the package. This collates all the examples in 
sesame.Rcheck/sesame-Ex.R.

2. Comment out the last line (quit('no')) in sesame.Rcheck/sesame-Ex.R.

3. From R do: source("sesame.Rcheck/sesame-Ex.R", echo=TRUE). At the 
same time, monitor memory usage in another terminal e.g. by running 
'top' on Linux/macOS.

4.Once all the examples have finished, do gc().

This is what I get on my laptop when doing the above with sesame 1.13.3:

 > gc()

             used   (Mb) gc trigger   (Mb)  max used   (Mb)

Ncells  15062911  804.5   24349446 1300.5  24349446 1300.5

Vcells 278380669 2123.9  473158124 3610.0 473157073 3610.0


That's a memory consumption of 4.9G which is confirmed by the 'top' 
command I was running in another terminal.

It would be awesome if 'R CMD check' could do the above for us i.e. 
report the amount of memory used by the examples. Any volunteer to 
request this feature on the R-devel mailing list?

Hope this helps,
H.

> 
> Very best,
> 
> Wanding
> 
> 
>> On Nov 29, 2021, at 12:15 PM, Hervé Pagès <hpages.on.github using gmail.com 
>> <mailto:hpages.on.github using gmail.com>> wrote:
>>
>> Hi Wanding,
>>
>> A process cannot allocate more than 3Gb of memory on 32-bit Windows. 
>> See my other post on this list for more information about this:
>>
>> https://stat.ethz.ch/pipermail/bioc-devel/2021-November/018663.html 
>> <https://stat.ethz.ch/pipermail/bioc-devel/2021-November/018663.html>
>>
>> Cheers,
>> H.
>>
>>
>> On 22/11/2021 05:17, Zhou, Wanding wrote:
>>> Hi,
>>> Recently we are experience some memory issue on both development and 
>>> release (mostly Windows) that causes the sesame package build to 
>>> fail. We are trying to address it but it seems to run fine locally on 
>>> our side.
>>> This is the error on the development branch
>>> Setting options('download.file.method.GEOquery'='auto')
>>> Setting options('GEOquery.inmemory.gpl'=FALSE)
>>> Error: processing vignette 'sesame.Rmd' failed with diagnostics:
>>> C stack usage  7977908 is too close to the limit
>>> And this is the error on the release branch
>>>> data = data.frame(estimate=c(runif(10, 0, 10)))
>>>> plotLollipop(data)
>>> Error in grid.newpage() :
>>>   'Calloc' could not allocate memory (27541715 of 1 bytes)
>>> Calls: <Anonymous> -> print.ggplot -> grid.newpage
>>> Let me know if you are aware of something or have a clue. Any insight 
>>> will be super helpful!
>>> Thanks!
>>> Zoom link: https://us02web.zoom.us/j/4248839577 
>>> <https://us02web.zoom.us/j/4248839577>
>>> ----
>>> Wanding Zhou, Ph.D.
>>> Assistant Professor
>>> Office: CTRB 9050
>>> 3501 Civic Center Blvd
>>> Philadelphia, PA 19104
>>> Lab: https://zhou-lab.github.io/ <https://zhou-lab.github.io/>
>>> Phone (o):  (215) 590-0473
>>> [[alternative HTML version deleted]]
>>> _______________________________________________
>>> Bioc-devel using r-project.org <mailto:Bioc-devel using r-project.org> mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel 
>>> <https://stat.ethz.ch/mailman/listinfo/bioc-devel>
>>
>> -- 
>> Hervé Pagès
>>
>> Bioconductor Core Team
>> hpages.on.github using gmail.com <mailto:hpages.on.github using gmail.com>
> 

-- 
Hervé Pagès

Bioconductor Core Team
hpages.on.github using gmail.com



More information about the Bioc-devel mailing list