[Bioc-devel] Cannot reproduce a regionReport 1.21.7 error on the Bioc 3.11 Linux builder

Martin Morgan mtmorg@n@b|oc @end|ng |rom gm@||@com
Fri Apr 24 12:26:30 CEST 2020

Thanks for the detective work Leo; to me it sounds like, for robust output (function more important than form) BiocStyle should not try to crop images.


On 4/24/20, 12:52 AM, "Bioc-devel on behalf of Leonardo Collado Torres" <bioc-devel-bounces using r-project.org on behalf of lcolladotor using gmail.com> wrote:

    Hi Bioc-devel again,

    I believe that I have found the source of my regionReport problem and
    a workaround. We'll know tomorrow if it goes through the Bioc builds
    but it works at

    The problem is related to
    https://github.com/ropensci/magick/issues/171 and similar issues. The
    workaround is to use `crop = NULL` on R Markdown code chunks to
    disable the default image cropping behavior by
    BiocStyle::html_document, and thus avoid cache issues from magick (R)
    and ImageMagick (not R).

    Both the Bioc-devel Linux machine and GitHub Actions using
    Bioconductor devel run into a problem where there's not enough
    resources in the cache for cropping the images produced by
    regionReport::derfinderReport(). Either (a) the Bioc-devel Windows and
    macOS machines don't have magick installed (so no cropping gets
    attempted), (b) use a compiled version of magick that ships with the
    right version of ImageMagick, or (c) they have more resources than the
    bioc Linux build machine and the bioc devel docker running on GHA

    In any case, disabling cropping circumvents this issue.

    Here's the full investigative report.

    ## A new lead

    I recently stumbled upon [1] and re-reading it now I see that they
    link to [2] which mentions that BiocStyle crops by default and that
    this can be turned off using `crop = NULL`.

    regionReport uses BiocStyle::html_document both for its vignettes as
    well as the reports it makes by default. Actually at [3], which was
    built using the Bioconductor devel docker & GitHub Actions (GHA), I
    see tons of messages that are similar to:

    ## The magick package is required to crop
    "basicExploration_files/figure-html/pvals-1.png" but not available.

    For reference, [4] is how that example from
    regionReport::derfinderReport() looks built using GHA. I copied the
    log of messages to a gist at [5]. From [6] I found [7] where Mike
    Smith explains to Aaron Lun the default behavior of BiocStyle and
    again suggests to use `crop = NULL` just like Yihui Xie mentioned
    earlier at [2]. I see at [8] that Kevin Rue-Albrecht also ran into a
    `crop = NULL` problem in iSEE.

    So I have a good lead now!

    ## Adding magick to Suggests

    We can see clearly that magick is not available on my GitHub Actions
    run using the Biocondutor devel docker: regionReport 1.21.7 doesn't
    mention magick on it's DESCRIPTION so it's never installed on GHA.
    However, magick is likely installed on the Bioconductor 3.11 Linux
    build machine and thus it is used for cropping. At [9] I bumped
    regionReport to 1.21.8 after adding magick to suggests. Actually, this
    is what Yihui Xie recommends BiocStyle could/should do in [2]. That
    crashed and burned on GHA Linux with Bioc-docker at [10], thus
    reproducing the Linux build machine error.

    Quitting from lines 224-229 (basicExploration.Rmd)
    35Error: processing vignette 'regionReport.Rmd' failed with diagnostics:
    36R: cache resources exhausted
    @ error/cache.c/OpenPixelCache/4083

    Actually, this error message is more informative than the one from the
    Bioc Linux builder as it gives the actual image name where it all
    burned down.

    ## Moving problematic code from the vignette to an example

    At [11] I moved the problematic code to an example instead of the
    vignette. On GHA, it failed again at [12] predictably on another image
    than in [10]. Also predictably, R CMD build worked this time, but R
    CMD check failed. However, now the I get the full logs that help me
    see the details much better and link it to code [13] (so if that was
    the bug source, I would know where to look).

    Quitting from lines 399-400 (basicExploration.Rmd)
    355Error in magick_image_trim(image, fuzz) :
    356 R: cache resources exhausted
    @ error/cache.c/OpenPixelCache/4083
    357Calls: derfinderReport ... <Anonymous> -> assert_image ->
    <Anonymous> -> magick_image_trim
    358Execution halted

    ## Using crop = NULL

    At [14] I dropped magick from the DESCRIPTION and added crop = NULL to
    the template Rmd files in regionReport. On GHA the Linux build ran
    well [15] and the resulting example report [overwrote 4] looks well =)

    ## Other OS?

    What about the other OS? Well, I guess that they have some extra
    resources than the Linux one. For example, locally on my macOS laptop
    regionReport 1.2.8 did build [16] with magick being available.

    Alternatively, the other Bioc-devel build machines (Windows, macOS)
    might not have magick installed, hence no cropping was attempted and
    no errors would have been detected, like in GHA's Linux regionReport
    1.2.7 build.

    I replied to Thomas Lumley about the lead idea at [17] before doing
    all these tests.


    [1] https://github.com/yihui/knitr/issues/1785#issuecomment-574723631
    [2] https://github.com/yihui/knitr/issues/1796#issuecomment-575371248
    [3] http://leekgroup.github.io/regionReport/articles/regionReport.html#derfinder-single-base-level-case-1
    [4] http://leekgroup.github.io/regionReport/reference/derfinderReport-example/basicExploration/basicExploration.html
    [5] https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016645.html
    [6] https://github.com/Bioconductor/BiocStyle/issues?q=crop
    [7] https://github.com/Bioconductor/BiocStyle/issues/65#issuecomment-552832630
    [8] https://github.com/iSEE/iSEE/commit/817d24d0aa092d04cbdfeb5daf5bfe60a711435a
    [9] https://github.com/leekgroup/regionReport/commit/14e7b714b7598add2b67d797668cecd121a726fe
    [10] https://github.com/leekgroup/regionReport/runs/614073194#step:8:36
    [11] https://github.com/leekgroup/regionReport/commit/c63ec2c737d8514f521ec64c66189ed8a8a772a2
    [12] https://github.com/leekgroup/regionReport/runs/614135852?check_suite_focus=true#step:8:354
    [13] https://github.com/leekgroup/regionReport/blob/c63ec2c737d8514f521ec64c66189ed8a8a772a2/inst/basicExploration/basicExploration.Rmd#L399
    [14] https://github.com/leekgroup/regionReport/commit/c7b2fc8dc086befd0539a8425cab81cc61baeb02
    [15] https://github.com/leekgroup/regionReport/runs/614159356?check_suite_focus=true#step:8:30
    [16] https://gist.github.com/lcolladotor/73db141aea3c7d9540ecd18b9b253e40
    [17] https://twitter.com/tslumley/status/1253505155523178496?s=20

    On Thu, Apr 23, 2020 at 10:03 PM Leonardo Collado Torres

    <lcolladotor using gmail.com> wrote:
    > Hi,
    > I went to google again and maybe
    > https://github.com/phw/peek/issues/112#issuecomment-296690302 is the
    > answer? It would require someone with access to the Linux builder to
    > check this particular ImageMagick config XML file.
    > I had linked to this thread earlier
    > https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016538.html
    > though back then I mentioned that it could be a disk space issue. That
    > was before the more recent thread about that ended with ggbio being
    > bumpted to 1.35.2 (for which using Bioconductor's devel docker
    > revealed the problem).
    > I also just asked on Twitter
    > https://twitter.com/fellgernon/status/1253502433231343617?s=20 to see
    > if anyone has seen a similar error.
    > Best,
    > Leo
    > On Thu, Apr 23, 2020 at 9:36 PM Leonardo Collado Torres
    > <lcolladotor using gmail.com> wrote:
    > >
    > > Hi Bioc-devel,
    > >
    > > I'm having trouble with my regionReport package on the Bioc-devel
    > > (3.11) Linux builds. Two weeks ago, using the Bioconductor devel
    > > docker I was able to reproduce an error that was solved by ggbio
    > > bumping their version number to 1.35.2 as described in [1]. I've tried
    > > my best to reproduce the error shown at [2] which is:
    > >
    > > Quitting from lines 224-229 (basicExploration.Rmd)
    > > Error: processing vignette 'regionReport.Rmd' failed with diagnostics:
    > > R: unable to get registry ID `cache:hosts' @
    > > error/registry.c/GetImageRegistry/202
    > >
    > > Recently [3], I bumped the regionReport version to 1.21.7 where I
    > > explicitly asked for ggbio 1.35.2 to be available. I suspected that
    > > the Linux builder still had version ggbio 1.35.1 installed. But that
    > > did not resolve the issue as the current build report is for
    > > regionReport version 1.21.7 and there's no error about ggbio 1.21.7
    > > not being available. Hm... the required versions for the dependencies
    > > does get checked at R CMD build (or even install), right?
    > >
    > > At [4] you can see that regionReport 1.21.7 can be successfully
    > > checked against Bioconductor devel's docker. This uses GitHub actions
    > > that has 7 GB of RAM available [5]. The memory resources (7GB vs 2.5
    > > GB? [6]), disk space (14 GB vs ?) and some system dependencies are the
    > > only differences I can think of (installed through [7], none seem to
    > > me to be related from what I see at [8]) besides the Linux builder
    > > differences vs the Bioconductor devel docker (dunno what they are).
    > >
    > > If you have any suggestions, please let me know.
    > >
    > > Best,
    > > Leo
    > >
    > > [1] https://stat.ethz.ch/pipermail/bioc-devel/2020-April/016538.html
    > > [2] https://master.bioconductor.org/checkResults/3.11/bioc-LATEST/regionReport/malbec2-buildsrc.html
    > > [3] https://github.com/leekgroup/regionReport/commit/5e7d956f71011f8551f2431f5af0e212d04be9f1
    > > [4] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:8:98
    > > [5] https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#supported-runners-and-hardware-resources
    > > [6] https://github.com/Bioconductor/Contributions/issues/1389#issuecomment-605024369
    > > 2.5 GB is used for the 32-bit windows Single Package Builder.
    > > [7] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:6:2
    > > [8] https://github.com/leekgroup/regionReport/runs/604161822?check_suite_focus=true#step:6:167
    > >
    > > Leonardo Collado Torres, Ph. D., Research Scientist
    > > Lieber Institute for Brain Development
    > > 855 N Wolfe St, Suite 300
    > > Baltimore, MD 21205
    > > http://lcolladotor.github.io

    Bioc-devel using r-project.org mailing list

More information about the Bioc-devel mailing list