[Bioc-devel] memory exhausted on tokay2

[Pages, Herve]

Hi Aaron,

I used 'top' on merida1 (one of our 2 Mac builders, MacOS El Capitan) to monitor memory usage of 'R CMD check methimpute_1.5.1.tar.gz' and also observed a pick at more than 6Gb there, like on my Linux laptop (Ubuntu 16.04 LTS). Note that memory usage stays relatively low (< 1Gb) for most of the time 'R CMD check' is running but starts to increase significantly when it reaches the "checking examples" step, with the pick happening at the very end of that step but for a very short time (about 2 sec.). On Linux I hit the space bar every sec in the terminal where I run 'top' to force it to refresh the displayed stats with enough frequency, otherwise I would probably miss the pick. On merida1 I don't have to do anything because the 'top' command there automatically refreshes the displayed stats at very short intervals.

H.

On 4/11/19 11:53, Aaron Taudt wrote:
Dear Hervé,

thank you for your reply and the explanations. I have checked the R CMD check on my system (MacOS Mojave) and the whole check does not exceed 1Gb of RAM. I am also surprised why it would need that much, because all the examples are pretty slim, so even if all objects are kept, I am wondering why it needs that much RAM on Windows/Linux.
If you can give me some more hints I'd be glad.

Cheers,
Aaron



Am So., 7. Apr. 2019 um 00:03 Uhr schrieb Pages, Herve <hpages using fredhutch.org<mailto:hpages using fredhutch.org>>:
Hi Aaron,

Maybe the particular example (plotting) where the "memory exhausted"
error occurs is not particularly memory-intensive. However keep in mind
2 important things:

   1) 'R CMD check' runs all the examples from all the man pages in the
same R session. This means that memory used (and not released) by other
examples will reduce the memory available for the example being
currently run. So even though your plotting examples use less than 1 Gb
of RAM, the 'top' command on my Linux laptop indicates that a full 'R
CMD check' on the methimpute package uses about 6 Gb of RAM!

   2) We use the --force-multiarch option when running 'R CMD check' on
the Windows build machines. This forces 'R CMD check' to run all the
examples from all the man pages twice: once in 32-bit mode and once in
64-bit mode. Note that 32-bit Windows limits the amount of memory that a
single process can use to about 3Gb. This would explain why the plotting
examples fail only when run in 32-bit mode (i386 arch). All the examples
run ok in 64-bit mode (x64 arch).

The solution is to try to reduce the memory footprint of your examples
in general, not just the plotting examples. Maybe there is an example
somewhere that creates a big object. Note that because all the examples
are run in the same session, this object will persist when other
examples are run. Removing the object (with 'rm(object)') might help.

The very last resort would be to mark the package as not supported on
32-bit Windows but hopefully we won't need to do that.

Hope this helps,

H.

On 4/6/19 04:10, Aaron Taudt wrote:
> Dear bioconductor-devel,
>
> I am trying to fix an "Error: memory exhausted (limit reached?)" error that
> arises during Rcheck on tokay2 (Windows Server 2012 R2 Standard / x64) .
> The package builds and passes Rcheck just fine on all other hosts. Is this
> something I can fix? The function in question is not particularly
> memory-intensive.
>
> Regards,
> Aaron Taudt
>
>       [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=i7VUGpP2oVdclQ-zgbh2Nfj5Q4KDtdvne_1mdlA6Sao&s=drLmF9Z_F3ETYHXRyi-kZsSZhCwDaphEEcsYMEEqvIY&e=

--
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
Phone:  (206) 667-5791
Fax:    (206) 667-1319


--
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
Phone:  (206) 667-5791
Fax:    (206) 667-1319


	[[alternative HTML version deleted]]