[Bioc-devel] memory exhausted on tokay2

Wed Apr 17 12:41:18 CEST 2019

maybe as an aid to debugging this, after running R CMD check on <pkg>_1.5.1.tar.gz one can run just the examples with

  R -f pkg.Rcheck/pkg-Ex.R

or interactively with something like

  R
  > source("pkg.Rcheck/pkg-Ex.R", echo = TRUE, max = Inf)

Martin

On 4/17/19, 5:31 AM, "Bioc-devel on behalf of Pages, Herve" <bioc-devel-bounces using r-project.org on behalf of hpages using fredhutch.org> wrote:

    Hi Aaron,

    I used 'top' on merida1 (one of our 2 Mac builders, MacOS El Capitan) to monitor memory usage of 'R CMD check methimpute_1.5.1.tar.gz' and also observed a pick at more than 6Gb there, like on my Linux laptop (Ubuntu 16.04 LTS). Note that memory usage stays relatively low (< 1Gb) for most of the time 'R CMD check' is running but starts to increase significantly when it reaches the "checking examples" step, with the pick happening at the very end of that step but for a very short time (about 2 sec.). On Linux I hit the space bar every sec in the terminal where I run 'top' to force it to refresh the displayed stats with enough frequency, otherwise I would probably miss the pick. On merida1 I don't have to do anything because the 'top' command there automatically refreshes the displayed stats at very short intervals.

    H.

    On 4/11/19 11:53, Aaron Taudt wrote:
    Dear Hervé,

    thank you for your reply and the explanations. I have checked the R CMD check on my system (MacOS Mojave) and the whole check does not exceed 1Gb of RAM. I am also surprised why it would need that much, because all the examples are pretty slim, so even if all objects are kept, I am wondering why it needs that much RAM on Windows/Linux.
    If you can give me some more hints I'd be glad.

    Cheers,
    Aaron

    Am So., 7. Apr. 2019 um 00:03 Uhr schrieb Pages, Herve <hpages using fredhutch.org<mailto:hpages using fredhutch.org>>:
    Hi Aaron,

    Maybe the particular example (plotting) where the "memory exhausted"
    error occurs is not particularly memory-intensive. However keep in mind
    2 important things:

       1) 'R CMD check' runs all the examples from all the man pages in the
    same R session. This means that memory used (and not released) by other
    examples will reduce the memory available for the example being
    currently run. So even though your plotting examples use less than 1 Gb
    of RAM, the 'top' command on my Linux laptop indicates that a full 'R
    CMD check' on the methimpute package uses about 6 Gb of RAM!

       2) We use the --force-multiarch option when running 'R CMD check' on
    the Windows build machines. This forces 'R CMD check' to run all the
    examples from all the man pages twice: once in 32-bit mode and once in
    64-bit mode. Note that 32-bit Windows limits the amount of memory that a
    single process can use to about 3Gb. This would explain why the plotting
    examples fail only when run in 32-bit mode (i386 arch). All the examples
    run ok in 64-bit mode (x64 arch).

    The solution is to try to reduce the memory footprint of your examples
    in general, not just the plotting examples. Maybe there is an example
    somewhere that creates a big object. Note that because all the examples
    are run in the same session, this object will persist when other
    examples are run. Removing the object (with 'rm(object)') might help.

    The very last resort would be to mark the package as not supported on
    32-bit Windows but hopefully we won't need to do that.

    Hope this helps,

    H.

    On 4/6/19 04:10, Aaron Taudt wrote:
    > Dear bioconductor-devel,
    >
    > I am trying to fix an "Error: memory exhausted (limit reached?)" error that
    > arises during Rcheck on tokay2 (Windows Server 2012 R2 Standard / x64) .
    > The package builds and passes Rcheck just fine on all other hosts. Is this
    > something I can fix? The function in question is not particularly
    > memory-intensive.
    >
    > Regards,
    > Aaron Taudt
    >
    >       [[alternative HTML version deleted]]
    >
    > _______________________________________________
    > Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=i7VUGpP2oVdclQ-zgbh2Nfj5Q4KDtdvne_1mdlA6Sao&s=drLmF9Z_F3ETYHXRyi-kZsSZhCwDaphEEcsYMEEqvIY&e=

    --
    Hervé Pagès

    Program in Computational Biology
    Division of Public Health Sciences
    Fred Hutchinson Cancer Research Center
    1100 Fairview Ave. N, M1-B514
    P.O. Box 19024
    Seattle, WA 98109-1024

    E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
    Phone:  (206) 667-5791
    Fax:    (206) 667-1319

    --
    Hervé Pagès

    Program in Computational Biology
    Division of Public Health Sciences
    Fred Hutchinson Cancer Research Center
    1100 Fairview Ave. N, M1-B514
    P.O. Box 19024
    Seattle, WA 98109-1024

    E-mail: hpages using fredhutch.org<mailto:hpages using fredhutch.org>
    Phone:  (206) 667-5791
    Fax:    (206) 667-1319

    	[[alternative HTML version deleted]]

    _______________________________________________
    Bioc-devel using r-project.org mailing list
    https://stat.ethz.ch/mailman/listinfo/bioc-devel