[Bioc-devel] memory exhausted on tokay2

[Aaron Taudt]

Dear Hervé,

thank you (and Martin) for the advice. I was able to locate the culprit: it
was a data.table::fwrite( ) statement, the behaviour of which must have
changed since the last release. I put this example in a NOT RUN block.

Kind regards,
Aaron




Am Mi., 17. Apr. 2019 um 12:41 Uhr schrieb Martin Morgan <
mtmorgan.bioc using gmail.com>:

> maybe as an aid to debugging this, after running R CMD check on
> <pkg>_1.5.1.tar.gz one can run just the examples with
>
>   R -f pkg.Rcheck/pkg-Ex.R
>
> or interactively with something like
>
>   R
>   > source("pkg.Rcheck/pkg-Ex.R", echo = TRUE, max = Inf)
>
> Martin
>
> On 4/17/19, 5:31 AM, "Bioc-devel on behalf of Pages, Herve" <
> bioc-devel-bounces using r-project.org on behalf of hpages using fredhutch.org> wrote:
>
>     Hi Aaron,
>
>     I used 'top' on merida1 (one of our 2 Mac builders, MacOS El Capitan)
> to monitor memory usage of 'R CMD check methimpute_1.5.1.tar.gz' and also
> observed a pick at more than 6Gb there, like on my Linux laptop (Ubuntu
> 16.04 LTS). Note that memory usage stays relatively low (< 1Gb) for most of
> the time 'R CMD check' is running but starts to increase significantly when
> it reaches the "checking examples" step, with the pick happening at the
> very end of that step but for a very short time (about 2 sec.). On Linux I
> hit the space bar every sec in the terminal where I run 'top' to force it
> to refresh the displayed stats with enough frequency, otherwise I would
> probably miss the pick. On merida1 I don't have to do anything because the
> 'top' command there automatically refreshes the displayed stats at very
> short intervals.
>
>     H.
>
>     On 4/11/19 11:53, Aaron Taudt wrote:
>     Dear Hervé,
>
>     thank you for your reply and the explanations. I have checked the R
> CMD check on my system (MacOS Mojave) and the whole check does not exceed
> 1Gb of RAM. I am also surprised why it would need that much, because all
> the examples are pretty slim, so even if all objects are kept, I am
> wondering why it needs that much RAM on Windows/Linux.
>     If you can give me some more hints I'd be glad.
>
>     Cheers,
>     Aaron
>
>
>
>     Am So., 7. Apr. 2019 um 00:03 Uhr schrieb Pages, Herve <
> hpages using fredhutch.org<mailto:hpages using fredhutch.org>>:
>     Hi Aaron,
>
>     Maybe the particular example (plotting) where the "memory exhausted"
>     error occurs is not particularly memory-intensive. However keep in mind
>     2 important things:
>
>        1) 'R CMD check' runs all the examples from all the man pages in the
>     same R session. This means that memory used (and not released) by other
>     examples will reduce the memory available for the example being
>     currently run. So even though your plotting examples use less than 1 Gb
>     of RAM, the 'top' command on my Linux laptop indicates that a full 'R
>     CMD check' on the methimpute package uses about 6 Gb of RAM!
>
>        2) We use the --force-multiarch option when running 'R CMD check' on
>     the Windows build machines. This forces 'R CMD check' to run all the
>     examples from all the man pages twice: once in 32-bit mode and once in
>     64-bit mode. Note that 32-bit Windows limits the amount of memory that
> a
>     single process can use to about 3Gb. This would explain why the
> plotting
>     examples fail only when run in 32-bit mode (i386 arch). All the
> examples
>     run ok in 64-bit mode (x64 arch).
>
>     The solution is to try to reduce the memory footprint of your examples
>     in general, not just the plotting examples. Maybe there is an example
>     somewhere that creates a big object. Note that because all the examples
>     are run in the same session, this object will persist when other
>     examples are run. Removing the object (with 'rm(object)') might help.
>
>     The very last resort would be to mark the package as not supported on
>     32-bit Windows but hopefully we won't need to do that.
>
>     Hope this helps,
>
>     H.
>
>     On 4/6/19 04:10, Aaron Taudt wrote:
>     > Dear bioconductor-devel,
>     >
>     > I am trying to fix an "Error: memory exhausted (limit reached?)"
> error that
>     > arises during Rcheck on tokay2 (Windows Server 2012 R2 Standard /
> x64) .
>     > The package builds and passes Rcheck just fine on all other hosts.
> Is this
>     > something I can fix? The function in question is not particularly
>     > memory-intensive.
>     >
>     > Regards,
>     > Aaron Taudt
>     >
>     >       [[alternative HTML version deleted]]
>     >
>     > _______________________________________________
>     > Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing
> list
>     >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=i7VUGpP2oVdclQ-zgbh2Nfj5Q4KDtdvne_1mdlA6Sao&s=drLmF9Z_F3ETYHXRyi-kZsSZhCwDaphEEcsYMEEqvIY&e=
>
>     --
>     Hervé Pagès
>
>     Program in Computational Biology
>     Division of Public Health Sciences
>     Fred Hutchinson Cancer Research Center
>     1100 Fairview Ave. N, M1-B514
>     P.O. Box 19024
>     Seattle, WA 98109-1024
>
>     E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
>     Phone:  (206) 667-5791
>     Fax:    (206) 667-1319
>
>
>     --
>     Hervé Pagès
>
>     Program in Computational Biology
>     Division of Public Health Sciences
>     Fred Hutchinson Cancer Research Center
>     1100 Fairview Ave. N, M1-B514
>     P.O. Box 19024
>     Seattle, WA 98109-1024
>
>     E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
>     Phone:  (206) 667-5791
>     Fax:    (206) 667-1319
>
>
>         [[alternative HTML version deleted]]
>
>     _______________________________________________
>     Bioc-devel using r-project.org mailing list
>     https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>

	[[alternative HTML version deleted]]